Repository navigation
fix: release request input buffers before long-running responses #4753
Description
Problem
Large requests retain input-only data across long responses: readBody listeners keep the original network chunks after concatenation; handleRequest passes the raw Buffer to rememberPluginMessages after forwarding; and the overflow-refold callback retains the preparation closure and parsed input.
Observed on installed 0.1.206 / 2e6f5255; the same paths remain in master bb7c249. A five-byte request reproduces leftover readBody listeners after both success and error. This establishes unnecessary retention, not the exact allocating site of a reported 4 GiB heap OOM. Buffers primarily affect external memory/RSS.
Scoped fix
- Release readBody chunks and owned listeners on settlement, including disconnects.
- Compute the raw-wire hash and text-comparability verdict before mutations, retaining only those scalars for bookkeeping at the existing post-forward point.
- Release raw input and the refold preparation closure once upstream retries are finished, before response streaming.
Keep accepted outbound bodies required for fake-completion/blind retries and echo provenance, and keep the session pin through the response. No admission controls, storage changes, kernel changes, or runtime deployment.
Validation
Focused listener/abort regressions, raw-versus-precomputed bookkeeping parity, response lifecycle and overflow-retry coverage, same-session concurrency, typecheck and build.
Related but separate: #4739 / #4750 covers storage cold-load memory. #3437 / #3643 discuss broader active-request materialization; #3138 concerns oversized receive behavior, #2786 accepted-refold Prepared correctness, and #2088 fork snapshot maintenance. None implements this input-lifetime cleanup.
Topic category: N (server internals).
Activity
ranxianglei commentedon Oct 25, 2026
(ai: vllm-qwen/qwen3.8-27b)
查重: 不重复。 open+closed issue/PR 按失败签名检索(readBody 监听器残留、raw buffer 跨响应保留、refold 闭包捕获)无同机制条目;issue 自身列出的 #4739/#4750(存储冷加载)、#3437/#3643(活动请求物化)、#3138(超大接收)、#2786(accepted-refold Prepared)、#2088(fork 快照维护)均为不同失败域,不并入。
审核(复现/层次/方案):三条保留路径在 master bb7c249 全部坐实——①
readBody(src/server.ts:4169)的 data/end/error 监听器从不摘除,闭包持有拼接前的全部网络 chunk,且 close-without-end 时 promise 永久悬空(admission 也随之泄漏);② handleRequest 把原始 Buffer 在 forward 之后才交给rememberPluginMessages(src/server/handle.ts:2788),整个流式响应期间 bodyBuffer/parsed/refold 制备闭包都挂在两个挂起帧上;③overflowRefold闭包经 forward 参数槽贯穿整个响应流(relay.ts:1403 是最后一次使用点)。这是输入侧数据生命周期问题,不是某个分配点问题——修法方向(结算点释放 + 标量替代 Buffer)打到了根因。动手范围(与 issue 一致,不做准入控制/存储/kernel 改动):
验证按计划走:listener/abort 定向回归、raw-vs-precomputed 记账平价语料、响应生命周期与 overflow-retry 覆盖(现有 #1195 refold e2e 套件同时充当「过早释放会破坏 refold」的回归闸)、同会话并发冒烟、typecheck+build。完成后回 PR 链接。
ranxianglei commentedon Oct 25, 2026
(ai: vllm-qwen/qwen3.8-27b)
PR #4766 已提交:https://github.com/ranxianglei/billion-context/pull/4766(分支
2026-10-26_input-buffer-lifetime,head1544b5a94,推送前已合入 origin/ci-fixes 骑掉继承红)。标签:bug / scope/wiring / p2(请求管道生命周期缺陷;滞留上界 = 响应时长,非无界增长,故 p2 非 p1)。八问
await forward(...)全程持有 bodyBuffer/bodyText/parsed 与 overflowRefold 闭包。onRetriesSettled一次性回调,在全部上游尝试(initial send/role ladder/pck probe/image retry/overflow refold)结束、首个响应字节之前触发,handleRequest 借此释放自己的输入单元。按 issue 要求:出站 body(fake-completion/blind retry、echo provenance)与会话 pin 保留到响应结束。request aborted before body completed(slot 两种情况都释放)。判定:ready-to-review;本地定向验证全绿,全量回归以 push head 上的 CI test 车道为准(红了按 #4599 流程分账,已在该帖留一行协调记录)。
fix: release the affinity message memo with the rest of the request i…