Repository navigation
test(inference): pin the streaming request-slot lifecycle of the steer endpoints - #214
Open
Alexander230 wants to merge 1 commit into
Open
Alexander230 wants to merge 1 commit into
Alexander230 wants to merge 1 commit into
Conversation
|
@Alexander230 is attempting to deploy a commit to the Neuronpedia Team on Vercel. A member of the Team first needs to authorize it. |
Alexander230
force-pushed
the
fix/38-streaming-request-lock
branch
from
July 15, 2026 09:49
035a411 to
8b3431a
Compare
This was referenced Jul 15, 2026
Contributor
Author
|
CI status note, to save reviewer time triaging red checks — none of the failures on this PR are introduced by it:
Verification from a fork rehearsal of the exact workflow (Alexander230#1, runs 29406654972 / 29406654966, Python 3.11 + 3.12 on clean runners):
(The 5 integration failures are tokenizer/numeric drift, identical on |
Alexander230
force-pushed
the
fix/38-streaming-request-lock
branch
from
September 11, 2026 17:35
4a7406b to
6163a7a
Compare
…r endpoints /steer/completion and /steer/completion-chat release the admission slot when the handler returns and take a slot of their own inside the SSE generator (stream_lock) for the stream's lifetime. Nothing covered that ordering. These tests drive both endpoints with a stub eager backend and check that the handler's slot is free before the stream starts, that the generator holds one while streaming, that it is released on exhaustion and on early close, and that a non-streaming request generates under the handler's slot without taking a second one (which would deadlock the single mutex a non-vLLM pod runs with). Refs hijohnnylin#38. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Alexander230
force-pushed
the
fix/38-streaming-request-lock
branch
from
October 9, 2026 10:44
6163a7a to
16d1170
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rescoped after #237. The request-lock change this PR originally carried is superseded upstream:
with_request_lockadmits the handler, and the SSE generators take their own non-exclusive slot viastream_lockfor the stream's lifetime. What was still missing is a test that pins that lifecycle, so this PR is now tests only — no production code changes.apps/inference/tests/unit/test_steer_stream_slot.pydrives/steer/completionand/steer/completion-chatwith a stub eager backend (no model, no GPU) and checks that:Verified to bite: with
stream_lockmutated to never take a slot, 5 of the 6 cases fail; with the non-stream path made to take a stream slot, the deadlock is reported as aTimeoutErrorrather than a hang.ruffandpyrightclean.Refs #38 (closed as not planned).
🤖 Generated with Claude Code