Skip to content

Conversation

@rodrigobnogueira
Copy link

@rodrigobnogueira rodrigobnogueira commented Jan 15, 2026

Summary

This PR fixes issue #3734 by breaking reference cycles in BoundSyncStream and BoundAsyncStream that prevent timely garbage collection of SSL contexts.

Problem

When using httpx.Client or httpx.AsyncClient, the create_ssl_context() function is called for each Transport instantiation. Each SSL context can consume 10s or 100s of MB of memory.

The issue is exacerbated by reference cycles between Response objects and their associated BoundSyncStream/BoundAsyncStream instances:

response.stream → BoundSyncStream
BoundSyncStream._response → response  ← creates cycle!

This cycle prevents the garbage collector from immediately reclaiming response objects and their associated resources (including SSL contexts), leading to excessive memory usage.

Solution

Use weakref.ref to break the reference cycle. The stream now holds a weak reference to the response instead of a strong reference:

# Before
self._response = response

# After  
self._response_ref = weakref.ref(response)

When closing the stream, we safely dereference and only set elapsed if the response is still alive:

response = self._response_ref()
if response is not None:
    response.elapsed = datetime.timedelta(seconds=elapsed)

Changes

  • httpx/_client.py: Modified BoundSyncStream and BoundAsyncStream to use weakref.ref
  • tests/test_bound_stream.py: Added 9 new tests covering:
    • Normal behavior (elapsed is set correctly)
    • Graceful handling when response is already garbage collected
    • Verification that no reference cycle exists

Testing

  • All linting checks pass (ruff format, ruff check, mypy)
  • New tests pass (9 tests in test_bound_stream.py)
  • Model tests pass (831 tests)
  • ASGI tests pass (24 tests)
  • WSGI tests pass (12 tests)

Risk Assessment

Low risk because:

  1. The weakref is valid during normal operation (caller holds the response)
  2. We only skip setting elapsed if the response was already garbage collected (edge case)
  3. Existing functionality is fully preserved

Fixes #3734

Checklist

  • I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.

rodrigo.nogueira added 2 commits January 15, 2026 00:08
Use weakref.ref in BoundSyncStream and BoundAsyncStream to hold
the response reference, breaking the reference cycle that prevents
timely garbage collection of SSL contexts.
@rodrigobnogueira
Copy link
Author

duplicate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The create_ssl_context() call can consume a lot of memory

1 participant