Repository navigation
[PR] fix: release request input buffers before long-running responses - #4766
Open
ranxianglei wants to merge 7 commits into
Open
ranxianglei wants to merge 7 commits into
ranxianglei wants to merge 7 commits into
Conversation
added 2 commits
October 26, 2026 02:18
Three retention sites kept input-only data alive across the whole
response stream of large requests:
1. readBody never removed its data/end/error listeners, so the closure
holding every network chunk stayed attached to the request for its
whole life; a close before end left the promise dangling forever
(admission leak). All four settlement paths now detach their own
listeners and drop the chunk array; close-before-end rejects with an
explicit abort error instead of hanging.
2. rememberPluginMessages took the raw inbound Buffer AFTER forwarding
and re-parsed + re-walked it. The raw-wire hash and text-comparability
verdict are now computed BEFORE prepare mutates the parsed graph
(anchored on the original parse, gated on the plugin lane) and only
those two scalars cross to the existing post-forward bookkeeping
point; hash-gate semantics are unchanged.
3. handleRequest's frame retained the raw body, decoded text, parsed
graph and the overflow-refold preparation closure across the entire
response stream. forward() now takes an onRetriesSettled callback,
fired exactly once when every upstream attempt has finished (initial
send, role ladder, pck probe, image-count retry, overflow refold),
before the first response byte; handleRequest releases its input
cells there. Accepted outbound bodies (fake-completion/blind retries,
echo provenance) and the session pin stay through the response by
design.
Behavior change: a client disconnect mid-body now surfaces as a
rejected readBody ("request aborted before body completed") instead of
a permanently pending promise; the admission slot is released either
way. Previously the slot leaked until process exit.
Closes #4753
ranxianglei
changed the title
…nput (#4753) Review finding on this PR: affinityMessagesCache (chain-fingerprint / resume-detection memo) aliases parsed.messages or input/contents, so the inbound message array stayed pinned on the suspended handleRequest frame across the whole response stream even after parsed was released. Null it in releaseRequestInput alongside the other input cells. Verified: typecheck clean, tests/issue4753-input-lifetime.test.ts + tests/proxy-readbody.test.ts 30/30 green.
Owner
Author
(ai: vllm-qwen/qwen3.8-27b) 查重: 重复 #4756(同症状 + 同机制 + 同根因,先来后到 #4756 优先)。
标签: pr(原有)、duplicate(与 #4756 同机制并行)、bug(内存驻留缺陷)、scope/wiring(请求管道输入生命周期)、p2(功能可用,大请求长响应下慢性内存增长,有 OOM 风险但不破坏功能)。 独立审核(未复用 #4753/#4756 任何一侧的既有结论,diff 两边都重读了)本 PR 实现正确,逐项核实过:
本地(head 1544b5a):typecheck/build clean;issue4753-input-lifetime + proxy-readbody 30/30;refold 族 + plugin-fork + session-transient-memory + context-overflow 共 120/120;e2e-overflow-selfheal 3/3、fix-arm-headroom-clamp 3/3。 审核中发现一个真实缺口(已修,已推): 与 #4756 的对比(同机制家族):
八问
收工判定代码质量通过(缺口已补)。建议合 #4756(先来后到 + 覆盖更完整),前置条件:把它 merge 一次 origin/ci-fixes 清掉 5 片继承红(哨兵批量或作者自行均可);本 PR 已打 duplicate,等 owner 决定关闭。若 owner 偏好本 PR 更干净的 API,缺口已在分支上修复(head 139c9db),可直接用。两个 PR 我都不会执行关闭。 |
ranxianglei
added
bug
Contributor
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.6147-pr.4766.772f0f0This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr4766.tgz
npm install -g packageInstall instructions are refreshed on each push. |
added 4 commits
October 26, 2026 10:15
master window commits 293ae65/de5e03659 bumped kernel/package.json to 0.0.165 without regenerating the lockfiles; lock-sync re-resolution then rewrites the stale recorded versions (root 0.0.163, kernel 0.0.164) and fails. Pure recorded-version sync via npm install --package-lock-only; no dependency or semantic change.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4753
What
Three input-lifetime retention sites kept request-side memory alive across the entire response of large requests:
readBody (src/server.ts) never removed its data/end/error listeners, so a closure holding every network chunk stayed attached to the request for its whole life; a close before end left the promise pending forever (admission-slot leak). Now every settlement path (success / error / abort / oversize) detaches its own listeners and drops the chunk array, and close-before-end rejects with an explicit abort error.
rememberPluginMessages (src/plugin.ts) received the raw inbound Buffer after forwarding and re-parsed + re-walked it for the text-comparability verdict. The raw-wire hash and verdict are now computed before prepare mutates the parsed graph (anchored on the original parse, gated on the plugin lane), and only those two scalars cross to the existing post-forward bookkeeping point. Hash-gate semantics unchanged.
handleRequest's frame (src/server/handle.ts) retained the raw body, decoded text, parsed graph and the overflow-refold preparation closure across the entire response stream. forward() now takes an onRetriesSettled callback fired exactly once when every upstream attempt has finished (initial send, role ladder, pck probe, image-count retry, overflow refold), before the first response byte; handleRequest releases its input cells there.
Per the issue scope: accepted outbound bodies (fake-completion/blind retries, echo provenance) and the session pin stay through the response. No admission controls, no storage changes, no kernel changes, no config surface added.
Behavior change (disclosure)
Validation