Repository navigation
[PR] fix: arm emergency-shrink evidence from explicit in-stream SSE context overflow (#4950) - #4979
ranxianglei wants to merge 10 commits into
Conversation
…t overflow (#4950) An HTTP-200 SSE stream whose frame carries an EXPLICIT context-window overflow error (Codex response.failed/context_window_exceeded, generic error/context_length_exceeded, Anthropic prompt-too-long, Gemini token-limit phrasing) used to pass through without recording any evidence: inspectContextOverflow only classified HTTP status bodies, and the #2313 estimate-only probe branch has no consumed latch — so an over-limit session re-probed the same raw body forever instead of ever folding. - src/util.ts: structuredErrorFields + classifyStructuredOverflow — shape-tolerant {code,message} extraction from in-stream error frames classified against the existing CONTEXT_OVERFLOW_PATTERNS (+ window / counts parsing). Generic failures stay un-classified (#570). - src/overflow-arm.ts (new): armSessionOverflow — the shared arming unit extracted verbatim from the relay closure; every lane records identical evidence (judging-limit clamp #3738, origin stamp so the arm survives the #1933 F2 route gate, no-window payload-estimate bound, proven-completion wall). - src/plugin.ts: both plugin pipes classify their error frames; evidence arming is independent of refold capability; when nothing visible was forwarded yet and the hook is wired, the failure frame is swallowed and the folded same-request re-send takes over the stream (one-shot, shares the re-issue budget with empty-turn / zero-byte-cut retries). - src/server/relay.ts: refoldAndResend + onInStreamOverflow hook handed to both plugin-mode call sites; the 400/413 refold now shares the helper (behavior unchanged). The two session-less SSE call sites (native-compaction resetAfterSuccess, non-injected/title-gen) stay intentionally unarmed — usage accounting is off there by design. - src/loop/core.ts: the proxy-mode loop lane arms evidence on in-band overflow errors too (advisory try/catch; the zero-side-effect retry path is untouched). Tests: tests/issue4950-instream-overflow-arm.test.ts — classifier matrix (positive/negative), arming units (clamp/estimate/proven-wall/ origin), pipe-level takeover + passthrough + negative cases for both wires, and the e2e regression pin: after one armed probe forward the next round runs the compression pipeline (summary call) instead of re-probing raw (master: zero summary calls forever).
ranxianglei
changed the title
# Conflicts: # src/plugin.ts
(ai: vllm-qwen/qwen3.8-27b) 冲突已解决并推送。merge state: dirty → clean（当前 做了什么：把 验证：本地 typecheck 通过；PR 自身回归套件 结构化八问
判定：冲突解除完毕，等待 CI 转绿即可由 owner 合并。 |
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.9307-pr.4979.6592908This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr4979.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) Conflict resolved and pushed: origin/master@9b6ec2345 is now fully merged into the branch (head 4ae0039). The only textual conflict was the import block in src/plugin.ts — resolved as the union of both sides' imports (PR's |
(ai: vllm-qwen/qwen3.8-27b) 查重结论:不重复。 本 PR 即 #4950 的修复实现。与 PR 自述 related-but-different 一致且我核对属实:#2484/#4262 走 HTTP status-body 证据通道(不同通道)、#1023/#805 是「流错误逐字透传」(本 PR 保留该行为,只新增取证)、#3738 是 post-arm headroom clamp(被吸收进共享 arming 单元,行为保持)、#2313/#1195 是 probe/refold 机制(此处扩展而非改动)。无同症状+同机制+同根因的重复条目。 身份说明:这是本 daemon 前一会话已完成的 PR。我未重做,做的是独立复核——拉分支核对代码、本地定向测试、CI 状态、merge 冲突解决完整性。 结构化八问
CI 与 merge 完整性
收工判定独立复核通过:修复命中根因、机制正确、无回归、无缓存破坏、无设计转向、无新配置。代码质量达标,可合;建议待远程 CI 全量 test 分片转绿后由 owner 合并(agent 不合并不发布)。本次我未改任何代码——未发现需要修改的问题。 |
(ai: vllm-qwen/qwen3.8-27b) 本会话的独立复核已完成并已发布在上方 FLOOR 9(完整结构化八问报告,comment id 49496)。此处仅补一行状态确认,不重复全文。 做了什么:对 PR #4979(head 结果:修复命中根因(in-stream SSE 溢出帧现经共享 新增佐证(我复核之后):独立 判定:等待远程 CI 全量 test 分片转绿后由 owner 合并(agent 不合并不发布)。本次未改任何代码——未发现需要修改的问题。 |
…ming (#4950) Route-switch e2e pair requested against PR #4979's acceptance contract: - a session seeded on route A (usage baseline bound to A's origin) switched to route B: the #2313 F2 route gate demotes A's evidence on B, the probe forward gets 200 + in-band overflow, and that evidence binds to B - so B's next round folds instead of re-probing raw. - generic (non-overflow) errors on deliberately incomplete streams never arm the shrink and leave the settled baseline untouched (#570 discipline); a follow-up turn completes normally. Targeted: new suite 18/18; plugin-empty-turn-retry + issue2313-probe-forward 23/23; typecheck clean. Full suite is CI.
…9-head # Conflicts: # src/plugin.ts # src/server/relay.ts
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Merge-conflict resolution complete and pushed. What I did this run:
Conflicts resolved (3 files):
I verified this is pre-existing by checking out the PR head in a baseline worktree (pre-merge): it fails identically there (16/17 pass, same failure). So my merge did not introduce it. Root cause (test-design issue in the PR's own regression test): the test posts I did not modify this test — it is the PR author's regression pin, and fixing it is a design call (change the fake's error trigger from body-size to a marker / disable compression for this case / use a larger window). Flagging for the PR author / owner. Note: I could not verify the live CI state — the mirror check-cache for this PR is stale (fetched 2026-11-03, sha still 八问（中文，供人工 reviewer）：
判定：合并冲突已解决并推送（PR 不再是 dirty）；typecheck 与 PR 核心回归通过。剩余一个预存测试失败（非本次引入）需 PR 作者/owner 决策修复方式。 |
[ework] auto-fix stalled: test (windows-latest, 22, 2), test (ubuntu-latest, 22, 2), test (ubuntu-latest, 24, 2) — stopping for human review |
🔶 需要决策 Red CI confirmed (this run): 3 failures, all in shard 2 — Root cause (verified pre-existing, not from the merge): The PR's own regression test This is a test-design decision in the PR author's regression pin — not a small mechanical fix, so I did not fix it (auto-fix lane: small mechanical fixes only). Options:
Recommendation: Option A (reliable trigger) — cleanest, preserves the test's intent. Stopping per auto-fix lane rules (design decision needed). |
…so the probe forward reaches the upstream The e2e test for 'generic errors and incomplete streams never poison the window' was broken: the proxy folds the oversized body before forwarding, so the fake upstream's >100KB error trigger never fired and the test received a normal 'ok' SSE instead of the expected 'Rate limit reached' error. Seed a stuck meter (lastInputTokens=7_300_000, source=estimate) before the oversized forward, mirroring the working overflow test, so the probe path forwards the raw body and the upstream sees the >100KB payload. This is the same mechanism the sibling test at line 536 already uses.
…case (value is now the seeded estimate) The previous fix removed the 'settled baseline survives' assertion because the seeded stuck meter changes the expected source from 'usage' to 'estimate'. Restoring it with the correct value keeps the test asserting the meter is neither armed (overflow-arm) nor reset after a generic error.
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) CI red resolved — the pre-existing test failure is fixed and pushed. Previous runs resolved the merge conflict (merge commit The failure: The fix (option D — reuses an existing mechanism, no new design): seed a stuck meter ( Verified: Pushed: Note for owner / PR author: if you'd prefer one of the earlier options (A: marker trigger / B: disable compression for this case / C: larger window) over the seed-the-meter approach, say so and I'll switch — but the seed approach is the minimal change that preserves the test's intent and reuses a mechanism the file already uses. What remains / risk: the full-suite verdict is the remote CI on 八问（中文，供人工 reviewer）：
判定：CI 红已解（预存测试失败已修并推送），merge 冲突此前已解。typecheck 与目标回归 17/17 绿。待远程 CI 在新 head |
Closes #4950
What this fixes
An HTTP-200 SSE stream whose frame carries an EXPLICIT context-window overflow error (Codex
response.failed/context_window_exceeded, genericerror/context_length_exceeded, Anthropic prompt-too-long, Gemini token-limit phrasing) used to pass through with NO evidence recorded:inspectContextOverflowclassifies HTTP status bodies only, and the #2313 estimate-only probe branch has no consumed latch. An over-limit session therefore re-probes the same raw body forever and never folds (field incident: ~1.47M est tokens vs a 744K judging window on gpt-6.1-sol via DSH native + Codex backend).Changes
src/util.ts:structuredErrorFields+classifyStructuredOverflow— shape-tolerant{code,message}extraction from in-stream error frames, classified against the existingCONTEXT_OVERFLOW_PATTERNS(+ window/counts parsing). Generic failures stay un-classified (Learned context window (self-heal) treats any truncated upstream stream as a window overflow: false positives permanently shrink the session window, persisting across restarts #570 discipline: no mis-learning).src/overflow-arm.ts(new):armSessionOverflow— the shared arming unit extracted verbatim from the relay closure; every lane records identical evidence (judging-limit clamp fix: overflow-arm stored above the headroom-reserved judging limit → unbounded CONTEXT_WINDOW_EXCEEDED loop #3738, origin stamp so the arm survives the DSH: 切换模型后，preflight 在窗口远未满时提前触发压缩 #1933 F2 route gate, no-window payload-estimate bound, proven-completion wall).src/plugin.ts: both plugin pipes classify their error frames; evidence arming is INDEPENDENT of refold capability; when nothing visible was forwarded yet (zero-visible window) and the hook is wired, the failure frame is swallowed and the folded same-request re-send takes over the stream (one-shot; shares the re-issue budget with empty-turn / zero-byte-cut retries).src/server/relay.ts:refoldAndResend+onInStreamOverflowhook handed to both plugin-mode call sites; the 400/413 refold now shares the helper (behavior unchanged). The two session-less SSE call sites (native-compactionresetAfterSuccess, non-injected/title-gen) stay intentionally unarmed — usage accounting is off there by design.src/loop/core.ts: the proxy-mode loop lane arms evidence on in-band overflow errors too (advisory try/catch; the zero-side-effect retry path is untouched).Behavior changes (disclosure)
overflow-arm, origin-stamped); the next preflight foldsNo new config surface (no env/config/CLI changes). No kernel change. No persistence-format change.
Dup screening
Related-but-different: #2484/#4262 (HTTP status-body overflow arming — different evidence channel), #1023/#805 (stream errors pass through verbatim — still true here), #3738 (post-arm headroom clamp — preserved, now inside the shared arming unit), #4381 (same user-visible symptom, insufficient root-cause evidence at the time), #2313/#1195 (probe design + refold mechanics — extended here, not altered).
Tests
tests/issue4950-instream-overflow-arm.test.ts(new): classifier matrix (5 positive shapes / 4 negatives), arming units (clamp / estimate / proven-wall / origin), pipe-level takeover + passthrough + negative cases for both wires, and the e2e regression pin: after one armed probe forward, the NEXT round runs the compression pipeline (summary call) instead of re-probing raw — on master that is zero summary calls forever.