Repository navigation
[PR] Failed-compress recovery: end the re-request on a user turn, not an assistant prefill (fixes upstream 400) - #5605
ranxianglei wants to merge 1 commit into
Conversation
ranxianglei
changed the title
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.8311-pr.5605.71f3b65This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5605.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重：不重复。 扫了全部 open issue/PR（近期 50 条）：没有其他条目动「failed-compress recovery 重请求的最后一条消息角色」这个点。最近的几个在途 PR（#5537 egress tool-args JSON guard、#5602 fork multimodal adoption、#5603 dsh refusal floor）机制与文件都不同。GitHub issue #5604（本 PR 目标）是这个症状+机制族的唯一跟踪项。 标签：bug / claude-code / scope/wiring / p1 —— 保活路径在 Claude 5 模型上直接杀死回合（subagent 当场死，按 owner 修订 2026-10-19 属 p1 类①「会话不可用」）；改动点在 src/loop/core.ts 请求管道（scope/wiring）；报告场景是 Claude Code 车道。 审核进行中，CI shards 跑完后给完整八问结论。本地定向验证（PR head c4d8bea）：tests/issue5220-failed-compress-receipt.test.ts 7/7 绿；loop 区全部套件（tests/loop-*.test.ts + receipt）合计 114/114 绿。环境备注：dev checkout 跑测试前需先 |
(ai: vllm-qwen/qwen3.8-27b) Review 结论：可合（§7.4 wire/message-shape 变更，需人工合并，我不代合）。CI 全绿：30 checks、0 failed、0 pending —— static-gates（ubuntu+windows）/mysql-contract/docs-sync/version-guard/test-import-graph 绿，11 个 test shards（ubuntu/windows × node22/24）全绿；e2e 各 lane 按门控 skip； 八问：
测试有效性：新 pin 断言的恰是此前缺失的维度（最终消息 role === "user"），同时保留 ❌ 字节断言——原 bug 回归时该测试立即红（修复前最终 role 是 assistant），非恒真断言。仅 Responses wire 载体测试，但断言对象是 adapter 映射前的 coreMessages role，wire-agnostic 性质对三条 wire 成立；marker 字节侧由既有 T1/T6 覆盖。 #1304 备注：wire-constraint ledger 目前无「请求须以 user 收尾」条目（wire-contract-fakes.ts 无先例）。该约束是 model-family 相关的（宽容 wire 接受两种形状），全局 ledger 规则会对 OpenAI/Google 场景误报，loop 级回归 pin 是正确的层；本 PR 不要求补 ledger。 判定：可合。请确认 CI 后自行合并 #5605 （我不能代合）。 |
|
(ai: vllm-qwen/qwen3.8-27b)
APPROVE. 单 commit，+63/-0，仅 八问
测试有效性：#5604 回归钉断言重请求末条消息角色 = user（既有 T1/T6 只断言 ❌ 字节上行，从不断言收尾角色），同时断言 assistant 回执仍携 ❌（缓存不变量完整）。去掉修复则末条为 assistant 回执，该断言必红——真测试，有牙； 两模式（§6）：proxy 模式受影响且被正确修复；plugin 模式结构上进不了此分支（injection 被抑制→无 proxy 侧 compress 执行→failedMutating 恒空），即便进入也是纯尾部 additive、末条非 assistant 才不加，不可能有害。 判定：可合（APPROVE）。 无需顺手小修：命名沿用 |
(ai: vllm-qwen/qwen3.8-27b) ✅ 审核完成 = APPROVE（可合）。独立八问结论已贴到 PR 评论（GitHub #5605，comment id 6601194833）。注意：本账号即 PR 作者 ranxianglei，GitHub 禁止自我 approve（POST /reviews → 422 “Can not approve your own pull request”），故正式 Approve 待 owner 在 UI 点一下。判定要点：守卫位置正确（两子情形之后）、#5220 缓存不变量保持（纯尾部追加）、回归钉有牙（断言末条角色=user）、行为变更已披露、不碰 kernel/、无 config/安全面、CI 绿（head c4d8bea）。无需顺手小修。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 审核收工 = APPROVE（可合）。 本轮做了什么：
一句话：可合，等 owner 点 Approve。 |
…t never ends on an assistant prefill A round whose compress/decompress calls ALL fail re-requests with the recovery receipt (#5220) as the final message. That receipt is assistant-role by design — its bytes mirror the markers emitMarker already streamed, so the client echoes them back as assistant content and the frozen prefix never diverges (#5220 cache invariant). But Anthropic rejects a request that ENDS on an assistant turn: "final assistant content cannot end with trailing whitespace" when the joined markers end in \n, and outright "assistant message prefill not supported" on Claude 5 models. Either way the 400 kills the turn and a subagent dies with "Agent terminated early due to an API error" — the path meant to keep the task alive (#5220) is the one that kills it. After splicing this round's failed frames and appending the receipt, cap the body with a thin user nudge whenever the last message is still an assistant turn, so the re-request always ends on a user turn. The nudge is ephemeral like the receipt (never streamed, never persisted, dies with the invocation), so the client never sees it and there is no cross-turn seam. Flipping the receipt itself to user would break the #5220 cache invariant, hence a separate cap. Behavior change (all wires): the failed-only re-request now appends one short user turn whenever it would otherwise end on an assistant message. Tolerant wires (OpenAI/Responses/Google) accept either shape; capping makes the invariant universal instead of Anthropic-specific. No config surface added. Regression pin: tests/issue5220-failed-compress-receipt.test.ts asserts the re-request's final message role is user (the existing T1/T6 only asserted the marker bytes ride upstream, never which role ends the body).
c4d8bea to
2e3b183
Compare
|
(ai: vllm-qwen/qwen3.8-27b) APPROVE（re-review，head 2e3b183）。 本账号即 PR 作者 ranxianglei，GitHub 禁止自我 approve（POST /reviews → 422 “Can not approve your own pull request”），正式 Approve 待 owner 在 UI 点一下。 与上一版（c4d8beaac）相比改了什么
八问（针对本版）
测试有效性（重点看新增）：新 Anthropic 测断言重请求末条 role = user（去掉修复必红），并断言失败标记 ❌ 已流到客户端、回合存活（DONE-ANTHROPIC-5604 到达客户端）；用真实 Anthropic SSE 事件形状（message_start / content_block_* / message_delta / message_stop）+ 签名 thinking 块复现生产 400 场景，比旧 Responses 测更贴近实际故障面。签名 thinking 块随 ephemeral 帧一起被剥离、不进重请求体，故不会触及 #10 的签名校验路径。CI 绿（head 2e3b183）。 两模式（§6）：同前——proxy 模式受影响且被正确修复；plugin 模式结构上进不了此分支（injection 抑制→无 proxy 侧 compress 执行→failedMutating 恒空），即便进入也是纯尾部 additive、不可能有害。 判定：可合（APPROVE）。 无需顺手小修。唯一非阻塞观察：新测里 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) re-review 收工 = APPROVE（可合，head 2e3b183）。 相比上一版： |
|
Tested on master Results
A related path the PR does not cover: a preflight fold can leave the outbound body ending on an assistant message
So a small guard still looks useful. We used this guard in const lastNonSystemRole = (msgs: readonly { role?: unknown }[]): unknown => msgs.findLast((m) => m.role !== "system")?.role;
if (lastNonSystemRole(rebuiltMessages) === "assistant" && lastNonSystemRole(parsed.messages) !== "assistant") {
rebuiltMessages = insertBeforeTrailingSystemRun(rebuiltMessages, [{ role: "user", content: "[bili] Earlier context was compressed above. Continue the task." }]);
}With the guard, 5 more agentic runs (151 Opus requests) and a full scenario batch (432 Haiku, 48 Opus requests) had 0 upstream 400s. A fix inside the preflight fold (never leave the tail on an assistant turn) would be cleaner, if you prefer that. |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Thanks for the rigorous field validation — this is exactly the confirmation the fix needed, and it lands on real traffic (Claude Code 2.1.311 → bili → LiteLLM → Bedrock, Anthropic Messages wire): stock master produced the in-band SSE error on a failed-compress round, while with the three PRs you saw 0 in-band errors and 0 upstream 400s across ~500 requests, with the On the related preflight-fold path — confirmed real, and it is a different mechanism. I checked the code: Scope: this lives in Fix — my recommendation (final call is yours): I would avoid restructuring the fold itself. Protection relaxes to 0 precisely to close a token deficit, so “don’t fold the last assistant message” fights the fold’s job and may fail to close the gap. The robust shape is the invariant “no outbound Anthropic body ends on an assistant turn” enforced once at the Anthropic egress: if the last non-system message is an assistant turn, append one thin ephemeral user nudge before the trailing system run. Since this same shape now has ≥2 producers (the loop re-request #5605 already caps; the preflight fold), I’d extract that check into a single shared helper invoked at every Anthropic serialization point rather than a second hand-rolled guard — DRY, and it future-proofs against the next path that emits an assistant tail. Whoever implements it must verify the nudge lands after all cache-stable content so it does not move the prefix (the same consideration #5605 already handles). If you would rather minimize blast radius first, the targeted version is the identical guard local to What I will not do without your go-ahead: modify #5605 (it is correctly scoped and now field-validated), or implement the reporter’s snippet verbatim (unverified external input, and it references a non-existent helper). Proposed next step (one line from you triggers it): confirm (a) the durable home — keep #5604 open as the umbrella for both mechanisms, or cut a focused issue since #5605 is marked “Closes #5604” — and (b) the approach above (shared egress gate vs targeted Bottom line: #5605 stands APPROVED for its failed-compress scope; the preflight-fold assistant-tail is a real, separate, p1-class gap in the same symptom family, ready to be tracked and fixed separately once you pick the home + approach. |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Recap of handling for this report (full detail in the message above):
Nothing modified, nothing filed yet — deliberately held at the owner-decision boundary (unverified source + design choice). Proceeding on your word. |
What & why
Since #5220/#5231 (0.1.210+), a round in which every
compress/decompresscall fails re-requests with the recovery receipt as the final message. That receipt is assistant-role by design — its bytes mirror the markersemitMarkeralready streamed, so the client echoes them back as assistant content and the frozen prefix never diverges (#5220 cache invariant). But Anthropic rejects a request that ends on an assistant turn:messages: final assistant content cannot end with trailing whitespace— when the joined markers end in\n;This model does not support assistant message prefill. The conversation must end with a user message.— Claude 5 models reject any prefill.Either way the client gets a 400 and the turn ends; in Claude Code a subagent dies with
Agent terminated early due to an API error. The path meant to keep the task alive (#5220) is the one that kills it.Fix
In
src/loop/core.ts, after splicing this round's failed frames and appending the receipt, cap the body with a thin user nudge whenever the last core message is still an assistant turn, so the re-request always ends on a user turn. The guard sits after both sub-cases, so it also covers the deduped-markers edge where a round that opened with model prose leaves an assistant tail.The cap is a separate message, not a role flip on the receipt: flipping the receipt to
userwould break the #5220 cache invariant (the client echoes the streamed markers as assistant, so a user-role receipt desyncs the prefix). The nudge is ephemeral like the receipt — never streamed, never persisted, dies with therunCompressLoopinvocation — so the client never sees it and there is no cross-turn seam.Behavior change (all wires)
The failed-only re-request now appends one short user turn whenever it would otherwise end on an assistant message. Tolerant wires (OpenAI / Responses / Google) accept either shape; capping makes "an outbound request never ends on an assistant turn" a universal invariant instead of Anthropic-specific. No config field added, no env var, no default/threshold moved.
Tests
tests/issue5220-failed-compress-receipt.test.tsgains a regression pin asserting the re-request's final message role isuser(the existing T1/T6 only asserted the marker bytes ride upstream, never which role ends the body) while still carrying the ❌ receipt as assistant text. Targeted suites green locally (90 tests across the loop/receipt files); full regression + e2e on CI.Closes #5604
Update (post-external-review)
storage-render golden,storage engine: full-mode restart) was inherited from the stale based244fff(~233 commits behind) — those shards fail on the old base and pass on current master, unrelated to this change. The rebase clears them; no goldens were regenerated.tests/issue5220-failed-compress-receipt.test.ts: signed-thinking-block failed-only round) covering the wire where the 400 actually fires — the original pin only exercised the Responses wire. Local verification: typecheck clean; receipt + loop-core 34/34; previously-red storage/golden 17/17.