Repository navigation
Anthropic stream:false: a model compress call ends the client turn (receipt returned with end_turn, no re-request) #5957
Description
Symptom
A Claude Code session ends right after a fold. The final result is the compress receipt ([Compressed m00005–m00019 → 1 block(s), ~42144 tokens saved.] · b1 full summary ...), and no upstream request follows. In an agentic Java coding task this left an empty or partial diff.
It happens when the request reaches bili as stream:false. A compression proxy in front of bili can do this. For example, Headroom with CCR turns a client stream:true request into a buffered stream:false request once its retrieve tool is active ("CCR: stream:true request has headroom_retrieve available; using buffered stream:false upstream request").
Environment
- billion-context master
fe963d7f5, and 0.1.212 (daa082ead): same code. - Claude Code 2.1.311, Claude Opus 5.5, Anthropic Messages wire.
- Chain: Claude Code -> Headroom 0.40.0 (CCR on) -> bili -> LiteLLM -> AWS Bedrock.
Minimal reproducer
- Start bili with a mock Anthropic upstream that returns one
compresstool_use (as intests/ccr-v2.test.ts, "e2e CCR v2 non-stream"). - POST
/v1/messageswith"stream": false. - bili answers with a text block that holds the full compress result and
stop_reason: "end_turn". It sends no second upstream request. The existing test asserts exactly this shape.
With a real model the client stops there. With "stream": true the same response is re-requested and the turn continues.
Root cause
src/server/relay.ts:2441(0.1.212::2317): the non-stream Anthropic branch calls the one-shot rewriterrewriteJsonResponseAsync.src/stream.ts:1673-1684(rewriteJsonSteps; 0.1.212::1566): it replaces each proxytool_usewith a text block of its result. When no client tool call is present, it setsstop_reason = "end_turn"(:1684). Nothing re-requests upstream.- The stream lane re-requests after a proxy-only round (
src/loop/core.ts:1531,reRequest = proxyResults.length > 0 && realCalls === 0), so the model sees the result and continues.
This is the Anthropic analogue of #4019. PR #4037 adds a non-stream proxy-tool loop for OpenAI chat-completions only.
Fix approach
Add a non-stream proxy-tool loop for Anthropic, with the design of PR #4037:
- Proxy-only round: execute the calls through the existing rewriter, append the original assistant content and a
tool_resultuser turn to the forwarded body, and re-request. Repeat until the model answers or calls a client tool, bounded byMAX_LOOP_ROUNDS. - A round with a client tool call, or with no proxy call, keeps the current behaviour.
- No receipt text goes to the client on this lane, because the non-stream tag-echo strip removes marker lines. A prepended marker became an empty text block.
- Like PR [PR] fix: execute proxy-owned tools on the non-stream OpenAI path (#4019) #4037, the re-request resends the unfolded body. The fold takes effect from the next client request.
diff --git a/src/compress-loop-anthropic.ts b/src/compress-loop-anthropic.ts
new file mode 100644
index 000000000..35047b3c3
--- /dev/null
+++ b/src/compress-loop-anthropic.ts
@@ -0,0 +1,64 @@
+import { MAX_LOOP_ROUNDS } from "./loop/index.js";
+import { isProxyToolFor } from "./absorb.js";
+import { withSessionLock } from "./session.js";
+import { fetchWithRetry, UpstreamHttpError } from "./fetch-util.js";
+import { proxyDispatcher } from "./upstream-proxy.js";
+import { rewriteJsonResponseAsync, type RewriteCtx } from "./stream.js";
+
+type Block = { type?: string; name?: string; id?: string; text?: string };
+
+interface LoopOptions {
+ proxyUrl?: string;
+ signal?: AbortSignal;
+}
+
+interface RequestOptions {
+ url: string;
+ headers: Record<string, string>;
+ wireTransform?: (body: Record<string, unknown>) => Record<string, unknown>;
+ resign?: (headers: Record<string, string>, body: string | Buffer) => void;
+}
+
+export async function compressLoopAnthropicJson(
+ initial: Record<string, unknown>,
+ ctx: RewriteCtx,
+ options: LoopOptions,
+ requestBody: Record<string, unknown>,
+ requestOptions: RequestOptions,
+): Promise<Record<string, unknown>> {
+ let current = initial;
+ const isProxy = (b: Block) => b.type === "tool_use" && typeof b.name === "string" && isProxyToolFor(b.name, ctx.session, ctx.config);
+ for (let round = 1; round < MAX_LOOP_ROUNDS; round++) {
+ const content = Array.isArray(current.content) ? (current.content as Block[]) : [];
+ const proxyIdx = content.flatMap((b, i) => (isProxy(b) ? [i] : []));
+ if (proxyIdx.length === 0 || content.some((b) => b.type === "tool_use" && !isProxy(b))) break;
+ const original = structuredClone(content);
+ await withSessionLock(ctx.session, () => rewriteJsonResponseAsync(current, ctx, options.signal));
+ const rewritten = current.content as Block[];
+ const results = proxyIdx.map((i) => ({ call: original[i]!, text: rewritten[i]?.text ?? "" }));
+ const messages = Array.isArray(requestBody.messages) ? requestBody.messages : [];
+ requestBody.messages = [
+ ...messages,
+ { role: "assistant", content: original },
+ { role: "user", content: results.map((r) => ({ type: "tool_result", tool_use_id: r.call.id, content: r.text })) },
+ ];
+ const roundBody = JSON.stringify(requestOptions.wireTransform ? requestOptions.wireTransform(requestBody) : requestBody);
+ requestOptions.resign?.(requestOptions.headers, roundBody);
+ ctx.log(`[acp-proxy: non-stream anthropic proxy-tool round ${round}; re-requesting so the model sees the result]`);
+ const result = await fetchWithRetry(requestOptions.url, {
+ method: "POST",
+ headers: requestOptions.headers,
+ body: roundBody,
+ ...(options.proxyUrl ? { dispatcher: proxyDispatcher(options.proxyUrl) } : {}),
+ }, undefined, options.signal).catch((e) => {
+ if (e instanceof UpstreamHttpError) ctx.log(`[acp-proxy: non-stream anthropic loop upstream error ${e.status}: ${e.body.slice(0, 200)}]`);
+ throw e;
+ });
+ try {
+ current = (await result.response.json()) as Record<string, unknown>;
+ } finally {
+ result.clearTimer();
+ }
+ }
+ return current;
+}
diff --git a/src/server/relay.ts b/src/server/relay.ts
index d56c3ce81..0baa12570 100644
--- a/src/server/relay.ts
+++ b/src/server/relay.ts
@@ -19,6 +19,7 @@ import { applyCompatDropFields, dropCompatFieldsJson, resolveCompatDropFields }
import { applyCompatRoles, applyCompatRolesJson, detectRoleRejection, detectSystemPlacementError, hasOffHeadSystem, resolveCompatRoles, type CompatRoles } from "../compat-roles.js";
import { builtinCompatRoles, builtinRoleRestorePoints, withRestoredRoles } from "../kiro.js";
import { compressLoopResponsesJson } from "../compress-loop-responses.js";
+import { compressLoopAnthropicJson } from "../compress-loop-anthropic.js";
import { resolveCompress } from "../compress-settings.js";
import { buildAbsorbSystemPrompt, buildCompressHybridSystemPrompt, buildCompressSystemPrompt, withMarkerIntegrityNote, withMcpDialectNote, withProjectAttribution, withSummaryBudgetNote } from "../compress-tool.js";
import { detectGenerationBoundary, GENERATION_BOUNDARY_NOTE } from "../generation-boundary.js";
@@ -2393,6 +2394,16 @@ async function forwardCaptured(
{ url: upstreamUrl, headers: requestHeaders, wireTransform, resign: applyResign },
);
}
+ if (prepared.protocol === "anthropic") {
+ const requestBody = JSON.parse(typeof body === "string" ? body : body.toString("utf8")) as Record<string, unknown>;
+ json = await compressLoopAnthropicJson(
+ json,
+ ctx,
+ { proxyUrl, signal: clientAbort.signal },
+ requestBody,
+ { url: upstreamUrl, headers: buildForwardHeaders(headers), wireTransform, resign: applyResign },
+ );
+ }
// Capture upstream usage so tokenCount (which drives nudge +
// emergency-truncate) reflects reality for non-streaming
// sessions too. The streaming loops do this in their SSETests:
- New
tests/bug4-nonstream-anthropic-proxy-loop.test.ts: a proxy-only round re-requests, sends atool_resultfor the call id, and returns the continuation. A mixed round and a round without proxy calls do not re-request. tests/ccr-v2.test.ts"e2e CCR v2 non-stream" pinned the old one-shot shape. We changed its mock to answer atool_resultwith text. The test now checks that the receipt and the decompress content reach the model in the re-request, and that the client gets the final text.- Kernel 1,621/1,621 pass. The root suite shows no new failures.
npx tsc --noEmitis clean.
Measured (agentic Java coding task, Headroom CCR in front, Opus 5.5)
| Build | Model folds followed by a request | Runs with a non-empty final result and diff |
|---|---|---|
| 0.1.212 + local fixes for #4885/#5432/#5604 | 0/1 | 0/1 (final result = receipt) |
| master + #4898 + #5441 + #5605 | 0/1 | 0/1 (final result = receipt, empty diff) |
| same + this fix (+ a small trailing-assistant guard, see the PR #5605 comment) | 5/5 | 6/6 (the agent reports a passing build in every run) |
Scenario regression with the fix: 0 upstream 400s in 432 Haiku and 48 Opus requests. All 51 client-visible folds were followed by more output or a tool call.
Activity
ranxianglei commentedon Nov 8, 2026
(ai: vllm-qwen/qwen3.8-27b)
查重：不重复。 Closest items are #4019 (stream:false OpenAI chat-completions — proxy tools passed through unexecuted, answer lost) and its still-open PR #4037. Same failure family (the non-stream lane lacks the proxy-tool loop the stream lane has), different wire: this report is the Anthropic Messages lane, which #4037 explicitly does not touch. No open issue/PR or mirror [PR] row targets this thread.
Verified against master (
3ac3e95c) — the mechanism you describe is exactly what the code does:elserewriter at src/server/relay.ts:2413-2414 → one-shotrewriteJsonResponseAsync.if (converted && !sawRealToolUse) b.stop_reason = "end_turn"— each proxytool_usebecomes its receipt text and the turn ends; nothing re-requests upstream.reRequest = proxyResults.length > 0 && realCalls === 0) confirms the stream/non-stream asymmetry.Approach assessment: your fix direction is right — port the #4037 non-stream proxy-tool loop to the Anthropic lane with the same three-way gate (proxy-only round → execute + append original assistant content and a
tool_resultuser turn + re-request; mixed or proxy-free round → keep today's one-shot behavior untouched). Two implementation notes for my pass:tool_resultuser turn, never an assistant prefill.executeAnthropicProxyTool, so there is no double delivery.标签: bug · claude-code · scope/wiring · p1 — when a fold fires on a buffered (
stream:false) anthropic-wire turn, the agent's turn dies with zero output; core-promise failure under that condition, not cosmetic.Implementing now: new
src/compress-loop-anthropic.ts+ relay wiring + tests (a unit suite mirroringissue4019-nonstream-proxy-tools.test.ts, plus updating the ccr-v2 non-stream e2e, which currently pins the buggy shape). PR follows in this thread.ranxianglei commentedon Nov 8, 2026
(ai: vllm-qwen/qwen3.8-27b)
Fixed — PR: #5966 (labels applied: bug, p1; CI just kicked off — no check results in the mirror cache yet, I'll follow up when they land).
What changed. Non-stream (
stream:false) Anthropic responses that call a proxy tool are now handled by a small loop (src/compress-loop-anthropic.ts, wired intosrc/server/relay.tsfor the anthropic protocol), porting the streaming lane's decision (core.ts:1531):tool_resultuser turn are appended to the forwarded body, and the proxy re-requests upstream so the model actually sees the result.MAX_LOOP_ROUNDS; at the limit the last response degrades to the pre-fix shape (receipt + end_turn) — never an undeclared proxy tool_use handed to the client.Signed thinking blocks ride the replayed assistant turn verbatim (KDD #10, test-pinned); the continuation always ends on the tool_result USER turn (#5605); like #4037 the re-request resends the UNFOLDED body, so the fold materializes on the next client request (prefix-cache friendly).
Behavior change (disclosed). Old: a non-stream anthropic turn whose model called e.g.
compressended with the fold receipt as the turn's ONLY output (stop_reason: end_turn) — the model never saw the fold result. New: such a round makes one extra upstream request per proxy-call round and the client receives the model's real post-compress answer instead. Client-visible turn output changes from receipt text to the final answer; cost is one extra upstream call on affected turns only. This is the fix itself, not drift.Tests. New
tests/issue5957-nonstream-anthropic-proxy-loop.test.ts(8 tests, including negative gates: real client tool_use / plain text / mixed rounds → zero fetches, byte-identical passthrough).tests/ccr-v2.test.ts's non-stream e2e previously PINNED THE BUGGY SHAPE and now pins the fixed one: the client gets the answer (no tool_use block), the receipt rides the re-request's tool_result, unfolded history is resent on the re-request, the folded view materializes on turn-2 outbound, the range restore rides the turn-2 re-request. Local targeted runs: typecheck clean; issue5957 suite 8/8; ccr-v2 16/16; marker + image-ref-dispatch suites green. Full-suite verdict comes from CI.Review note: open PR #4037 touches the same relay.ts non-stream region (OpenAI lane) — no textual conflict at this base, but worth reviewing together if they merge close in time.
中文八问（供 reviewer）
stream:false)Anthropic 请求里,模型调用代理工具(compress/acp_status/search_context/decompress)时,一次性改写器把每个 tool_use 替换成结果文本并强制end_turn——压缩轮直接以"回执"作为该回合并唯一输出而结束,模型永远看不到折叠结果,回合内后续内容全部丢失。compressLoopAnthropicJson:纯代理工具轮用与一次性改写器完全相同的执行器执行(compress 走 applyConfiguredCompression,其余走 executeAnthropicProxyTool,原始结果文本与流式车道载体一致),把原 assistant 轮+tool_result user 轮追加到转发体后重请求上游;混合/无工具轮原样返回,交给未动的一次性改写器;每份上游响应只被一个执行器执行;MAX_LOOP_ROUNDS=10 封顶,到限降级为修复前形态。收工判定:代码已交付(PR #5966,基于当前 master e6cb37e 之上),待 CI 绿 + reviewer 过目即可合入。
ranxianglei commentedon Nov 8, 2026
(ai: vllm-qwen/qwen3.8-27b)
CI update for #5966 (head
e6cb37e, verified live just now): all green — 132 check runs on this SHA, zero failures, nothing still running; every workflow lane (CI test shards on ubuntu/windows × node 22/24, static-gates, architecture-docs, config-surface-guard, docs-coverage, gate, and all e2e lanes incl. hermetic registry + dsh plugin lane + real-codex E2E) has a completed/successful run. A first wave of runs was auto-cancelled by the repo's concurrency re-trigger right after PR open — each cancelled lane was superseded by a successful sibling run, so the cancellations are orchestration noise, not signal.PR is ready for review/merge (merge stays human-only per repo rules).
test(#5957): pin the MAX_LOOP_ROUNDS degradation gate for the non-str…
Merge pull request #5966 from ranxianglei/2026-11-08_nonstream-anthro…
fix: execute proxy-owned tools on the non-stream Anthropic path (ranx…
Merge upstream/master (5c3c268): adopt ranxianglei#6001 root README l…