feat: add idle timeout for StreamableHTTP sessions#1994
Draft
felixweinberger wants to merge 1 commit intov1.xfrom
Draft
feat: add idle timeout for StreamableHTTP sessions#1994felixweinberger wants to merge 1 commit intov1.xfrom
felixweinberger wants to merge 1 commit intov1.xfrom
Conversation
Member
|
Can't the task manage itself? I don't think we want the overhead of a background task that checks other tasks. This seems something the specification should mention. |
Add a `session_idle_timeout` parameter to StreamableHTTPSessionManager that spawns a background reaper task to periodically terminate and clean up sessions that have been idle beyond the configured threshold. - Background reaper scans sessions at half the timeout interval - Accounts for retry_interval to avoid reaping sessions between polling reconnections - Makes transport.terminate() idempotent for safe cleanup - Validates that session_idle_timeout is not used with stateless mode - Reaper removes sessions from tracking dicts before calling terminate() to prevent race conditions - Includes comprehensive test coverage including lifecycle verification Fixes #1283
felixweinberger
force-pushed
the
fweinberger/idle-timeout-termination
branch
from
debfd7d to
73f4cf2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
session_idle_timeoutparameter toStreamableHTTPSessionManagerthat enables automatic cleanup of idle sessions, fixing the memory leak described in #1283.Motivation and Context
Sessions created via
StreamableHTTPSessionManagerpersist indefinitely in_server_instanceseven after clients disconnect without sending DELETE. Over time this leaks memory. This was reported in #1283 and originally addressed in #1159 by @hopeful0 — this PR reworks that approach.Key design decisions:
_server_instances, crash cleanup), so this is the natural place for reaping. The transport stays unaware of timeout logic._server_instancesperiodically and terminates sessions that have been idle longer than the threshold.retry_interval(SSE polling) is configured, the effective timeout is at leastretry_interval_seconds * 3to avoid reaping sessions that are simply between polling reconnections.terminate()on the transport is now idempotent (early return if already terminated), making it safe to call from both the reaper and explicit DELETE requests.None(no timeout, fully backwards compatible). Docstring recommends 1800s (30 min) for most deployments.How Has This Been Tested?
6 new tests in
tests/issues/test_1283_idle_session_cleanup.py:terminate()idempotencyretry_intervalNoneAll existing tests pass unchanged.
Breaking Changes
None.
session_idle_timeoutdefaults toNone(no timeout).Types of changes
Checklist
Additional context
Based on the approach from PR #1159 by @hopeful0, reworked to move idle tracking to the session manager, remove
__aenter__/__aexit__and condition variables from the transport, and account forretry_intervalin the idle threshold.Closes #1283