Repository navigation
[PR] fix: repair non-JSON tool-call arguments at the OpenAI/Responses egress chokepoint (#5534) - #5537
[PR] fix: repair non-JSON tool-call arguments at the OpenAI/Responses egress chokepoint (#5534)#5537ranxianglei wants to merge 7 commits into
Conversation
…ss chokepoint (#5534) Issue #5534: the kernel's ref-keyed text replay (emergencyTruncateLatch) can stamp fold/checkpoint prose onto whatever message currently holds a latched ref. When that message is a tool-call core, coreToOpenai/coreToResponses emit the prose verbatim as function.arguments and strict-JSON upstreams reject every later request with parse_error.101 — permanently poisoning the session until some fold resets the latch. This PR adds the egress last line of defense suggested in the issue: validate every outbound tool_calls[*].function.arguments (OpenAI chat wire) and every type==function_call item's arguments (Responses wire) at the prepare chokepoints, covering both the kernel-processed path and the catch-fallback path that forwards parsed.messages verbatim. A payload failing JSON.parse is replaced with a valid placeholder object plus a [wire-guard] warn log carrying its first bytes for diagnosis. Parsing payloads (including empty strings) pass byte-exact, and the same array reference is returned when nothing was repaired — valid traffic is untouched, per the #1039 discipline of never rewriting possibly-legitimate bytes. Behavior change: previously non-JSON arguments produced by bili's own egress fault were forwarded verbatim -> upstream 500 + permanent session poison; now they are replaced with a valid placeholder + warn log. Model-produced malformed args on the loop/replay path were already repaired there (src/loop/core.ts MALFORMED_PROXY_TOOL_ARGS_REPLAY); this guard covers the prepare/history-rebuild path those never touch. Root cause (identity-less latch entries + cold-load ledger restamp renumbering under surviving entries) is a kernel change and owner-gated; see the decision request on issue #5534.
ranxianglei
changed the title
(ai: vllm-qwen/qwen3.8-27b) 查重：不重复 —— 本 PR 实现的是 #5534 里建议的 option b（出站最后一道防线），不是重复立项。检索 open+closed 的 issue/PR（按失败签名：egress guard / non-JSON arguments / latch cross-write）：
标签：bug, dsh, scope/wiring, p1（会话被永久毒化 = 用户不能用，按 owner 2026-10-19 修订属 p1 会话不可用类；lane 按报告环境取 dsh）。 开始独立 review。预先报告一个阻塞项：分支落后 master 632 个提交，且 |
# Conflicts: # src/server/prepare-openai.ts # src/server/prepare-responses.ts
Review finding: the useDecide branch builds its side call from a copy of the prefix taken BEFORE the main-body guard runs, so in a poisoned session on a strict-JSON upstream every decide round-trip still 500s and the nudge-decide arm fails for the whole poisoned stretch. Apply the same guard to the side call's prefix on both wires.
… resolution The merged import line carried imageRescaleFor — an identifier that does not exist (server.ts exports imageReserveFor). Every Responses-wire request died with ReferenceError -> 502 before turn processing. Restored the master spelling and aligned the import order with master.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.9172-pr.5537.c16d5f0This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5537.tgz
npm install -g packageInstall instructions are refreshed on each push. |
|
🔶 需要决策 PR #5537 覆盖了 steady 请求 + nudge-decide sidecall 两条出口，但独立审核发现同一机制还有第三个出口未覆盖：unified-loop 的 continuation 请求（proxy-mode 下 bili 服务端执行 compress 后的 round-2+ 重发）。 证据链（本次审核独立复核）：
后果评估：proxy-mode 车道（plain OpenAI/Responses 客户端、codex 等由 bili 驱动 compress 回路的车道；plugin-mode 正常流量不可达——工具注入被抑制，模型看不到 compress 工具）在 wedge 活跃窗口内（emergency truncate / cold-load restamp 之后、下一次成功 fold 清 latch 之前），continuation 请求仍会把脏 arguments 发给严格 JSON 上游 → 当轮 500。本 PR 合入后 steady 路径已安全，因此不再是「永久中毒」，但同机制残留一个更窄的失败面。Anthropic loop adapter 结构性安全（safeParse→{}），Google adapter 已有 parse fallback，缺口仅在 openai/responses 两个 loop adapter。 被卡 diff 面（若批准补齐）：src/loop/adapter-openai.ts:247、src/loop/adapter-responses.ts:330 各加一次对应 harden* 调用；需为 adapter factory 补 sessionId/log 管道（或复用 loggerLog 不带 session 前缀）——跨文件行为变更，按规则不代改，请 owner 拍板。 选项：
已打 needs-decision 标签。等 owner 拍板，不动代码。 |
ranxianglei
left a comment
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
APPROVE（评审结论以本评论为准） — 本 PR 与审核同账号提交，GitHub 禁止自评 approve，故结论以评论形式发布。 独立对照源码与 kernel 逐行核实：修复做到了它声称的事，覆盖两个受守卫 chokepoint 内的全部路径，无回归。CI 在 head 5eded5390 全绿（51/51 checks success/skipped，直接拉取确认）。一处相邻路径发现（unified-loop continuation 请求出口未覆盖）已单独走 🔶 需要决策（见后续评论）——不阻塞本 PR：那是先于本 PR 存在的暴露面、窄于报告的 bug，且报告复现路径已被完整修复。
八问
- 是什么问题: emergency fold 之后，裁剪出的 checkpoint 文本会被盖到出站请求 assistant tool_calls 的 arguments 上；严格解析 JSON 的上游直接 500（parse_error.101），客户端每轮重放历史 → 会话永久不可用。
- 什么原因导致的: kernel 的 emergency-truncate latch 只按 ref 编号存
{ref, text}，重放时不做身份校验，把裁剪文本盖到「当前持有该 ref」的消息上；冷启动 ledger restamp（restampRefsToLedger）会在存活 latch 条目下重新编号 ref，使旧裁剪落到无关消息——包括 tool-call core（其 text 就是 arguments JSON）。已对 kernel 源码核实（compress.tsemergencyTruncateNode、refs.ts、view-latch.ts），机制成立。 - 影响范围与严重程度: p1 可用性缺陷。触发条件：stale latch 条目绑定到 tool-call core + 严格 JSON 上游（OpenAI/Responses wire）。报告复现：DSH 车道 / Windows / infernix-serve。Anthropic 结构性安全（safeParse→{}），Google 已有 parse fallback。
- 怎么修复的: 两条 prepare 主出口做最后防线——出站前校验每个 tool-call arguments 字符串，非空且 JSON.parse 失败 → 替换为确定性合法占位符（主请求体 + nudge-decide sidecall，两 wire 全覆盖）。合法流量逐字节不变（no-op 返回原数组引用）。根因（latch 加身份字段）仍按 owner-gated 在 #5534 走决策，本 PR 独立交付 host 侧防线。
- 是否有回归问题: 无。逐项审计见下节。
- 是否破坏缓存: 不破坏。占位符是确定性常量，中毒会话跨轮发送稳定字节（前缀缓存一致）；合法流量字节不变。sidecall 与主请求同序 normalize→harden，前缀字节相等（cache-line parity 保持）。
- 是否改了设计方向: 不改。host 侧出口防御，kernel 根因另案。
- 是否引入新配置: 无。
回归审计（触碰路径的既有行为逐项）
- prepare-openai.ts: 单一渲染点 L337 + 单一出口 L538；守卫位于 normalizeStrictEchoReasoning 之后、body 组装之前 —— 覆盖 kernel 路径、catch fallback（parsed.messages 原样转发）、title-gen、snapshot-render 视图；其后所有变更只追加纯 user/system 消息，不会重新引入未守卫内容。✓
- prepare-responses.ts: 守卫在所有变换之后（L617-620）、唯一主 return（L712）之前；字符串型 input 不含 function_call 项；codex-forge 合成体（L814）不带历史。✓
- 两条 lane 的 decide sidecall: 与本 PR 声明一致（commit ea20aa5），且与主请求同操作序 → 缓存线一致。✓
- custom_tool_call.input 不触碰（free-form 契约保持）。✓
- tool-call 配对键（id/name/type/call_id）完整保留，仅替换 arguments 值。✓
- 日志面: 每次违规调用一条 warn，64 字节预览经 JSON.stringify 转义，含 sessionId/tool 名/#5534 标记；无敏感信息；有界。✓
- 测试有效性: 9 个测试与声明一致；#3617 双向门纪律齐备（该触发时触发 + 不该触发时不触发，含空串/object 型/undefined/custom_tool_call 负例）；机制复现走真实 kernel latch 重放路径而非 mock；断言有牙，bug 回归时会红。✓
- 行为变更披露（旧值→新值）: 旧 — 非 JSON arguments 逐字节转发 → 严格 JSON 上游 500 + 会话永久中毒；新 — 替换为
{"_bili":...}占位符 + warn 日志，会话向前自愈。合法流量无变化。PR 描述已披露，本审核确认披露完整准确。
已顺手修：PR description 中过期的测试文件名 tests/issue5536-tool-args-guard.test.ts → tests/issue5534-tool-args-guard.test.ts（rename commit 1d73dcc 后未同步描述；已 PATCH body，其余内容未动）。
判定: 可合并。
(ai: vllm-qwen/qwen3.8-27b) 状态确认：上一条 🔶 需要决策 是本审核自己发布的发现（unified-loop continuation 出口未覆盖 #5534 同一机制），非新问题。当前状态：评审结论 APPROVE 已在 PR #5537 发布（GitHub 禁止同账号自评 approve，故以正式评论形式）、CI 在 head 5eded53 全绿（51/51）、PR description 过期测试文件名已顺手修正、needs-decision 标签已打。除被卡部分外无其他待办——现停手等 owner 在 A（并入本 PR）/ B（另起小 PR）之间拍板，拍板前不动代码。 |
(ai: vllm-qwen/qwen3.8-27b) PR #5537 审核完成,结论:可合并(人工合并)。已把评审中发现的问题直接修进 PR 分支并推送,head 结构化八问
测试有效性:9 个测试按 #3617 双向门禁纪律——正向行用 reporter 的两个字节签名(checkpoint 散文 评审中追加的提交(全部在 PR 分支上,各自独立标注,贡献者原始提交保留):
查重结论(重申首条回复):不重复——同机制家族有 #5110(已关,kernel 侧 BLOCKED sentinel 修复尝试)与 #5217(open,latch 存于 BLOCKED sentinel ref 下),但层次与交付物不同;无其他在途出站 guard PR。标签:bug/dsh/scope/wiring/p1/pr(p1=会话彻底不可用类)。 判定:方向对(根因已钉死且 owner-gated,宿主侧防线独立可发)、代码与测试达标、CI 全绿、冲突已解——可合并,请人工合并 #5537 。kernel 侧根治待 #5534 决策,批准后另开。 |
…ake) Single conflict: src/server/prepare-responses.ts import block — master moved repairResponsesAssistantOrdering out of server.js into responses-ordering.js and added the context-analysis import; resolved as union of both sides, keeping the PR's hardenResponsesFunctionCallArguments util import. All other files auto-merged; verified merged tree = master + exactly the PR's guard additions.
[bot] 🏷 进度汇报（进程被强制重启后恢复，冲突已处理并推送）： 已完成
遇到的阻塞 接下来：镜像缓存盯 CI check 状态（master 红/绿先查 #4599 总控区分继承红与自有红），全绿后给七问总结。 |
[bot] 🏷 冲突处理完成，CI 全绿。本轮总结（七问）： 本轮做了什么
结构化八问
待办提醒：floor 6 的 A/B 决策（unified-loop continuation 出口守卫并入本 PR 还是另起小 PR）仍等 owner 拍板，本轮未动该部分，PR 范围保持不变。 判定：冲突已清、CI 全绿（30/30，0 failed），PR 就绪等人工合并：#5537 |
ranxianglei
left a comment
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Round-2 review of head 2c8af07411 (post master-sync merge): APPROVE（评审结论以本评论为准） — 本 PR 与审核同账号提交，GitHub 禁止自评 approve，故结论以评论形式发布。CI 在 2c8af07411 全绿（镜像缓存实读：30 checks，0 failed；e2e 车道按惯例 skip）。
What changed since round 1: exactly ONE commit — 2c8af07411 "Merge origin/master into PR #5537" (parents: old head 5eded5390 + master tip 58a8796). No new PR-side commits. Verified independently:
- Net contribution identical to round 1.
git diff --stat origin/master...HEAD→ same 4 files, same stat (prepare-openai +13, prepare-responses +11, util +82/-1, test +213). I re-read every guard hunk against current master and confirmed byte-equivalence to the round-1 reviewed code. - Merge integrity. The only real conflict (prepare-responses import block: master moved
repairResponsesAssistantOrderingintoresponses-ordering.js, added context-analysis import) resolved as union — PR'shardenResponsesFunctionCallArgumentsutil import kept, no master import lost. Master-side file churn (base..master-parent) vs merge-side delta (pr-parent..HEAD) are stat-identical (190+/21- across the 3 src files) → merged tree = master + exactly the PR's guard additions. Test file byte-identical old-head vs new-head. No conflict markers. - Placement still dominates all paths on the new master (line numbers shifted by ~100 lines of upstream churn above the guards):
src/server/prepare-openai.ts: side-call guard L510 (feeds decide sideBody), main guard L578 immediately afternormalizeStrictEchoReasoningand BEFOREconst rebuilt; single exit L658 returnsrebuilt. All mutations after the guard append plain user/system messages only. Catch fallback (kernel transform failed → forwardparsed.messagesverbatim) still covered because the guard runs after it.src/server/prepare-responses.ts: side-call guard L528 (normalizedItems → stableItems → sideBody AND yes-span directive[...stableItems]), main guard L630 behindArray.isArray(rebuiltInput)right after the strict-echo repair and BEFOREconst rebuilt; single main return L731 (L834 return is the codex-forge endpoint helper — synthetic body, no history). String-typed input can't carry function_call items.src/util.ts: both harden functions unchanged from round 1 (verified verbatim); type-onlyResponseInputItemimport; deterministic placeholder constant.
- No kernel/ or pi-subagents/ changes in the net diff (
git diff --name-only origin/master...HEAD= the 4 host/test files) → nokernel-change:declaration needed, kernel-guard CI unaffected, HARD-RULE authorization gate not triggered. - Loop-adapter finding status: the unified-loop continuation egress (
src/loop/adapter-openai.tsbuildRequest,src/loop/adapter-responses.tsbuildRequest) is STILL unguarded — the 🔶 A/B decision from floor 6 remains pending owner sign-off; this sync did not implement either option and the PR scope is unchanged. Per NO-OPEN-ENDS that finding stays CONVERTED TO A DECISION (already labeled needs-decision), not parked debt.
结构化八问
- 是什么问题：bili 自己的出站渲染会把非 JSON 散文写进 tool-call arguments——紧急截断 latch 按 ref 号重放裁剪文本，冷加载 ledger restamp 把 ref 重编号后旧裁剪落到无关消息（含 tool-call core），严格 JSON 上游（infernix-serve）整包 500，会话永久中毒。
- 什么原因导致的：latch 条目只存
{ref,text}无身份校验（kernel emergencyTruncateNode/view-latch），restampRefsToLedger 重编号不触碰 latch——本轮无变化（根因修复属 owner-gated kernel 变更，已在 issue #5534 挂 🟡 决策）。 - 影响范围与严重程度：p1 可用性缺陷——DSH/OpenAI/Responses 车道遇严格 JSON 上游时整会话不可用；Anthropic（safeParse→{}）与 Google（parse fallback）结构性安全。
- 怎么修复的：两条 prepare 主出口的最后防线（main body + nudge-decide side body，双 wire）：非空且 JSON.parse 失败的 arguments 替换为确定性占位符常量并 warn 留证；合法流量字节不变、无修复时返回原数组引用。
- 是否有回归问题：无。本轮增量仅为 master 同步合并，净贡献与已审核的第 1 轮代码逐字节等价；既有行为清单逐项复查：单渲染点+单出口属性保持、catch fallback 仍被覆盖、title-gen/snapshot-render 视图经同一出口、字符串 input 不含 function_call、codex-forge 合成体不经历史、custom_tool_call.input 不触碰、配对键保留、日志有界且无敏感值、测试 9 个双向门+真实 kernel 重放复现全部完好。
- 是否破坏缓存：否——占位符为确定性常量，合法路径零字节变化；sidecall 与主请求同序 normalize→harden，缓存行对齐关系不变。
- 是否改了设计方向：否——纯 egress last-line-of-defense，根因层（latch 身份字段）仍在 owner 决策中。
- 是否引入了新的配置：无。
判定：可合并（人工合并）。head 2c8af07411 就绪；唯一待决项仍是 floor 6 的 loop-adapter 守卫 A/B（needs-decision 标签在案）。
PR #5537 round-2 review done (head |
|
冲突并重新review |
…es import conflict (keep hardenResponsesFunctionCallArguments, drop master-removed makeFoldPointReasoningDroppable)
[bot] 🏷 本轮唤醒的「dirty」信号已过期——冲突实际已被解决，live 验证如下： Live 状态（本轮实测，非记忆）
本轮做了什么
结论：无需再动分支——冲突已清、guard 完整性已验证、无失败 check。剩余等待：CI 跑完 + owner 人工合并（https://github.com/ranxianglei/billion-context/pull/5537）+ floor 6 的 A/B 决策（unified-loop continuation 出口守卫并入本 PR 还是另起小 PR，仍待拍板，本轮未动）。 |
[bot] 本轮收到两条重复的「PR 冲突（dirty）」信号（01:18 / 01:26，均标 attempt 1/3）。按状态新鲜度纪律先 live 验证再动手——两条都是过期快照，无需任何操作： Live 证据（本轮实测）
结论：冲突线、红 CI 线对本 bot 都已无事可做。剩余纯 owner 动作：① 人工合并 #5537 ；② floor 6 的 A/B 决策（unified-loop continuation 出口守卫并入本 PR 还是另起小 PR）仍待拍板，本轮未动。 八问
收工判定：本轮零动作收工；等待 owner 人工合并 + floor 6 决策。若再有第三条同类信号，先核实状态是否真实变化再报。 |
[bot] 第 4 条同类「dirty」信号（02:04，attempt 1/3）。Live 验证结果与几分钟前完全一致——仍是过期快照，零动作：
判断：blocker 检测器在同步前的旧快照上反复重发同一信号（本任务已第 4 次收到相同内容）。分支侧无 bot 可做之事，等 owner 人工合并 + floor 6 A/B 决策。若状态真实变化（master 前进或出现红项）会立即处理；对纯重复信号只做一行确认、不再重复长报告。 |
(ai: vllm-qwen/qwen3.8-27b) 冲突状态（live 实测，非记忆）
重新 review（独立重审，非抄前轮结论） 结构化八问
测试有效性：9 个测试按 #3617 双向门禁——正向行用 reporter 两个字节签名（checkpoint 散文、tag 前缀）断言被修；负向行（8 种合法形状 + 非 assistant + 非数组）断言字节不变 + 同数组引用；机制复现行在 kernel 级完整复现链条（clip latch → 模拟冷启动 restamp 重绑定 → turn 2 重放污染）再断言 guard 在两条 wire 上都修复出站体。复现行钉住当前（bug 态）kernel 重放行为，已标 known kernel defect pin——kernel 身份修复落地时需同步更新，是有意耦合不是假测试。 已知缺口（披露，等 owner 拍板）：unified-loop continuation 出口（proxy-mode round-2+ 重发， 判定：冲突已清、CI 全绿（27 success / 3 skipped / 0 failed，head |
What
Refs #5534 — implements the egress last line of defense suggested there (option b).
The kernel's ref-keyed text replay (
emergencyTruncateLatch,kernel/src/compress.tsemergencyTruncateNode) stamps its clipped text onto whatever message currently holds the latched ref — the latch entry carries only{ref, text}, no identity check. A cold-load ledger restamp (restampRefsToLedger,src/persist.tsmaybeRestampToLedger) renumbers refs under surviving latch entries, so a stale entry can land on an unrelated message. When that message is a tool-call core,coreToOpenai/coreToResponsesemit itstextverbatim asfunction.arguments— fold/checkpoint prose where JSON belongs. Strict-JSON upstreams then reject every subsequent request withparse_error.101, and because the latch persists in state and self-perpetuates each turn (no fold can fire to reset it while every request 500s), the session is permanently poisoned until something else folds the range.This PR validates every outbound
tool_calls[*].function.arguments(OpenAI chat wire) and everytype == "function_call"item'sarguments(Responses wire) at the two prepare chokepoints:src/server/prepare-openai.ts— after strict-echo normalization, beforerebuiltassembly; covers both the kernel-processed path and the catch-fallback path that forwardsparsed.messagesverbatim.src/server/prepare-responses.ts— same placement; touchesfunction_callitems only (custom_tool_call.inputis free-form and left alone).A payload failing
JSON.parseis replaced with a valid placeholder object and logged once per violation as a[wire-guard]warn with the first 64 bytes for diagnosis. Parsing payloads (including empty strings) pass byte-exact, and the untouched input array reference itself is returned when nothing was repaired.Behavior change (disclosed)
{"_bili": ...placeholder...}+ warn log; the request succeeds. Valid traffic is byte-identical (the guard only fires on already-invalid bytes — model-produced malformed args on the loop/replay path were already repaired insrc/loop/core.ts; this covers the prepare/history-rebuild path those never touch). Google wire already had this guard (src/loop/adapter-google.ts); Anthropic is structurally safe (tool_use.inputis object-typed viasafeParse).Root cause status
The identity-less latch replay itself is a kernel change (identity fields on
EmergencyTruncateLatchEntry/ViewLatchEntry+ gated replay), which is owner-gated per the Kernel Boundary — decision request posted on issue #5534. This PR ships the host-side defense independently so poison can no longer reach the wire regardless of the kernel fix timeline.Tests
New
tests/issue5534-tool-args-guard.test.ts(9 tests): gate discipline both directions (#3617) — the reporter's two byte signatures (checkpoint prose startingT…, tag-prefixed<acp …>) repaired; valid JSON variants / empty / undefined / object-typed / non-assistant messages / custom_tool_call byte-exact untouched; plus a mechanism repro: issue-#3939-style clip latches, simulated cold-load restamp rebinds the latched ref to a fresh assistant tool-call core, turn 2 replays the clip onto it (bug reproduced at kernel level), then the guard repairs the outbound body on both wires.Config surface: none added.