Skip to content

[PR] fix: bound the failed-compress orphan sweep window at the commit-time ref frontier - #6142

Open
ranxianglei wants to merge 7 commits into
masterfrom
2026-11-10_sweep-window-upper-bound
Open

ranxianglei wants to merge 7 commits into
masterfrom
2026-11-10_sweep-window-upper-bound

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

kernel-change: issue #6116 requests exactly this behavior (owner-filed feature request; in-thread decision B confirmed — mechanism fix only, no prompt change).

What & why

The #3769 fold-gated orphan sweep removes failed compress calls at/after the pending fold's earliest start ref (pendingOrphanSweepMinRef) with no upper bound. A failure issued in the same assistant turn AFTER the successful compress call (mixed multi-tool turns — compress mixed with other tools, exactly the scenario raised in #6116) therefore gets wiped on the next pass too, destroying fresh error feedback the model needs to react correctly — the exact pathology #3769 exists to prevent ("failures must remain observable or a deterministic model re-issues the same no-op compress forever").

Fix: applyCompression now also records pendingOrphanSweepMaxRef = highestUsedIndex(state.messageRefs) at commit time. The successful call and any later same-turn calls only receive refs strictly above that frontier on the next assignRefs pass, so they stay out of the sweep window [minRef, maxRef]. Across multiple folds committing before one consuming turn the markers accumulate (running MIN / running MAX). The hide node consumes both one-shot; sync-blocks, cloneState and host hydration carry the new field.

Behavior change (old -> new)

  • Old: sweep removed every non-active compress failure with ref >= foldMinRef (unbounded upward), including failures created after the successful call.
  • New: sweep removes failures with foldMinRef <= ref <= foldMaxRef; failures at/above the success-call frontier stay visible until a later fold covers them.
  • Upgrade compat: states written before this change lack the max bound and keep the pre-fix unbounded behavior (a stray max without min is inert by construction). Pinned by test.

Cache impact: none — the sweep still runs in the same post-fold rewrite pass where the cached breakpoint already moved past foldMinRef.

Duplicate screening

查重: 不重复 — closest open work is #5607 (failed-compress recovery re-request shape), #5987 (failure accounting on rebase), #5336 (orphan-sweep CLI surface); all different mechanisms, none bounds the sweep window.

Tests

  • New kernel/tests/issue6116-sweep-window.test.ts (7 tests): inclusive window both ends + single-ref windows; legacy min-only -> unbounded (upgrade pin); max-without-min inert; real core.applyCompression writes both markers; THE issue scenario (same-turn failure after the success call survives); two folds accumulate the union window; full processTurn pipeline carries the bound through sync-blocks and consumes both one-shot.
  • Existing suites green: issue3769-orphan-sweep, hide-consumed-stub/mcp, orphan-fixes, storage-state-fields (round-trip extended with the new field), verif-subset-guard + verif-bug3769-orphan-window (guarded files changed per KDD release v0.1.4 #14).
  • typecheck + build clean on rebased master (58a8796).

Closes #6116

ci-check added 2 commits November 10, 2026 22:15
…ef frontier (#6116)

The #3769 fold-gated orphan sweep removed failed compress calls at/after the
pending fold's earliest start ref with no upper bound, so a failure issued in
the SAME assistant turn AFTER the successful compress call (mixed multi-tool
turns) was wiped on the next pass too — destroying fresh error feedback the
model needs to react correctly (the exact pathology #3769 exists to prevent).

applyCompression now also records pendingOrphanSweepMaxRef = highestUsedIndex
of the ref map at commit time: the successful call and any later same-turn
calls only get refs strictly above that frontier on the next assignRefs pass,
so they stay out of the sweep. Across folds committing before one consuming
turn the markers accumulate (running MIN / running MAX). The hide node
consumes both one-shot; sync-blocks, cloneState and hydration carry the new
field. States written before this change lack the max bound and keep the
pre-fix unbounded behavior (upgrade compat, pinned by test).

kernel-change: issue #6116 requests exactly this behavior; owner filed the
issue, in-thread decision B confirmed (mechanism fix only, no prompt change).
mergeState() is the single source of truth for persisted CompressionState
fields (#4192); add pendingOrphanSweepMaxRef to its literal so the transient
window bound survives the unified round-trip like its min counterpart, and
pin the round-trip in the storage field test.
@ranxianglei ranxianglei added enhancement New feature or request p2 scope/kernel labels Nov 10, 2026
@ranxianglei ranxianglei changed the title fix: bound the failed-compress orphan sweep window at the commit-time ref frontier [PR] fix: bound the failed-compress orphan sweep window at the commit-time ref frontier Nov 10, 2026
@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-10_sweep-window-upper-bound (797c5d8)

Latest build — install this exact version

npm install -g billion-context@0.0.9309-pr.6142.72283f8

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
797c5d8 billion-context@0.0.9309-pr.6142.72283f8 build
227d45e billion-context@0.0.9231-pr.6142.03dd2f8 build
4bcb78a billion-context@0.0.9175-pr.6142.550d4c1 build
61b7663 billion-context@0.0.8997-pr.6142.481b6a6 build
687c400 billion-context@0.0.8960-pr.6142.634cd1a build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6142.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷
(ai: vllm-qwen/qwen3.8-27b)

Review pass started on this PR (independent of the author-side report above).

查重: 不重复 — searched open+closed issues/PRs for the sweep-window mechanism: closest items are #5607 (failed-compress recovery re-request shape), #5987 (failure accounting on rebase), #5336 (orphan-sweep CLI surface) and the closed orphan-ACK family #4293/#4307/#4310 — all different mechanisms, none bounds the sweep window. This is the only open PR targeting #6116; no parallel second fix PR exists.

标签: bug + scope/kernel + p2(内核清理窗口语义修正:特定条件——同轮混合工具调用中成功压缩后又出现新失败——下丢新鲜错误反馈,非不可用、非缓存泄露,p2)。

Live state verified just now: PR open, head 2026-11-10_sweep-window-upper-bound @ 687c400 on base master@58a87962 (= current master tip, no rebase needed). CI at the last mirror snapshot (15:49 UTC): 0 failed / 1 pending (mysql-contract queued) / 2 codex jobs cancelled by a re-run — I will confirm the final green before concluding.

Next: pulling the branch, reading the full diff against master, regression-walking the touched paths, and running the targeted kernel suites locally. Full structured conclusion (八问) to follow in a second comment.

@ranxianglei ranxianglei added the bug Something isn't working label Nov 10, 2026
ci-check added 3 commits November 11, 2026 04:09
kernel/package.json was bumped 0.0.192 -> 0.0.193 but its own
package-lock.json was left at 0.0.192, and the version line in both
lockfiles lost its indentation. npm install --package-lock-only
re-normalizes all of it, so git diff --exit-code went red.

- kernel/package-lock.json: 0.0.192 -> 0.0.193 (top-level + packages.""),
  matching the already-declared kernel bump in this PR.
- package-lock.json: restore canonical 6-space indent on the acp-kernel
  version line (content unchanged).
Resolve the kernel-version conflict introduced by master advancing
0.0.192 -> 0.0.195 while this PR carried its own 0.0.193 bump:

- kernel/package.json, root package-lock.json, kernel/package-lock.json:
  the merged tree now holds master's 0.0.195 kernel PLUS this PR's
  sweep-window change, so the version moves to 0.0.196 (one bump past
  master's tip) rather than reusing either side's stale number.
- Restores canonical indentation on the acp-kernel version line that the
  original sed-style bump had stripped (the root cause of the lock-sync
  red: npm install --package-lock-only re-normalizes it, so git diff
  --exit-code went non-zero).

Kernel source (compress.ts / sync.ts / types.ts / hide-consumed.ts)
auto-merged clean against master; only the version/lockfile lines
conflicted.
…und key

#6116 adds pendingOrphanSweepMaxRef to the persisted CompressionState
(kernel/src/compress.ts, sync.ts, types.ts; hydrated in
src/state-hydration.ts). The four storage-render goldens pre-date that key,
so the per-top-level-state-key digest pin (stateKeyDigests) now sees one new
entry. In the scripted golden scenario the value is undefined (the one-shot
window bound is consumed together with its fold marker), so the entry carries
the shared "null" digest 74234e98afe7498f and no other field changes —
stateDigest, section digests, per-turn wire digests and state shape are all
byte-identical. Net diff: one added line per wire.

Regen: node --import tsx scripts/update-storage-render-goldens.ts
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Re-verified live, root-caused, fixed, and pushed. The lock-sync red from the earlier wake is already resolved (green again on the new head). This wake's real blocker was three red test shards, all failing the same four golden tests.

What was red (live, head 4bcb78ab14): test (windows-latest,22,5), test (ubuntu-latest,24,3), test (ubuntu-latest,22,3) — each failing storage-render golden (chat|anthropic|responses|google). The CI assertion named exactly one divergent state key: pendingOrphanSweepMaxRef=74234e98afe7498f (golden undefined); foldAt/editedAt/stateDigest/all section digests matched.

Root cause: this PR (#6116) adds pendingOrphanSweepMaxRef to the persisted CompressionState (kernel compress.ts/sync.ts/types.ts, hydrated in src/state-hydration.ts). The four storage-render goldens pre-date that key and were never regenerated, so the per-top-level-state-key digest pin (stateKeyDigests) now sees one new entry → all four wire goldens red → the three shards that run this file red. The PR was simply incomplete on the golden side — not a behavioral regression.

Fix: regenerated the four goldens with the sanctioned script (node --import tsx scripts/update-storage-render-goldens.ts). The diff is exactly one added line per wire — "pendingOrphanSweepMaxRef": "74234e98afe7498f" — inserted right after pendingOrphanSweepMinRef. The value is undefined in the scripted scenario (the one-shot window bound is consumed together with its fold marker), so it carries the shared "null" digest and no other field changes: stateDigest, section digests, per-turn wire digests, and stateShape are all byte-identical to the old goldens.

Verification:

  • Local targeted run of tests/golden-storage-render.test.ts: 5/5 pass (all four wires + the orphaned-file guard).
  • Diff review: 4 files, +4 lines, nothing else.
  • Pushed 227d45eb8f to 2026-11-10_sweep-window-upper-bound (clean fast-forward, no force-push; remote head was an ancestor).

CI now (live, single GitHub API read — the mirror cache is stale, fetched_at stuck at 2026-11-11T17:03Z on the old head): on head 227d45eb8f, lock-sync, version-guard, kernel-guard, architecture-docs, and both static-gates are all green; the 10 test shards are still in progress (including the three that were red). Given the local 5/5 pass and the one-line-per-wire diff, these should turn green; I'll confirm on the next wake if any shard is still red.


八问(中文,供人工 reviewer):

  1. 是什么问题:PR 给持久化状态新增了一个字段 pendingOrphanSweepMaxRef,但没重新生成早已存在的 4 个 storage-render golden 快照,导致「逐状态键摘要」钉住多出一个键,4 条 wire 的 golden 全红,连带 3 个 test shard 红。
  2. 什么原因导致的:根因是 PR 不完整(漏了 golden 再生成),不是行为回归——新增键在脚本化场景里值恒为 undefined(一次性窗口上界随 fold 标记一起被消费),JSON.stringify 会省略它,所以 stateDigest/各 section 摘要/逐轮 wire 摘要全都不变,唯一差异就是 stateKeyDigests 里多了一行。
  3. 影响范围与严重程度:仅影响 CI 的 3 个 test shard(跑 golden 测试的那几个),不影响任何运行时行为、不影响用户;属「CI 红、产品无感」,严重度低(流程瑕疵)。
  4. 怎么修复的:用仓库自带的正规脚本 update-storage-render-goldens.ts 重新生成 4 个 golden,每条 wire 只加一行 pendingOrphanSweepMaxRef: 74234e98afe7498f(共享的 "null" 摘要),其余字节完全不动。
  5. 是否有回归问题:无。diff 只有 4 个 golden 文件各 +1 行,没碰任何 src/kernel 逻辑;stateDigest 与逐轮 wire 摘要逐字节未变,证明 wire 与持久化行为都没漂移。
  6. 是否破坏缓存:不涉及。这是测试快照再生成,不碰缓存/重试/游标/去重键。
  7. 是否改了设计方向:不涉及。纯测试快照对齐,无架构/边界/数据流变更。
  8. 是否引入了新的配置:不涉及。无新 env/config/默认值。

收工判定:修复已推送(227d45eb8f),本地 5/5 通过,diff 最小化(每 wire 一行);guard/lock-sync 已绿,test shard 在跑,预期转绿——若仍有 shard 红,下次唤醒我会继续跟进。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Re-verified live — everything is already green; no work needed this run.

This event reported lock-sync red on head 61b7663623, but that's an old, superseded head (the original 3-commit PR head, an ancestor of the current tip). The earlier wake already fixed both the lock-sync red and the three red test shards.

Live state on the current head 227d45eb8f (verified this run — fresh mirror cache + a single GitHub check-runs read):

  • lock-sync, kernel-guard, version-guard, architecture-docs, both static-gates: success
  • all 11 test shards: success (incl. the three that were red: windows 22/5, ubuntu 24/3, ubuntu 22/3)
  • tally: 38 success / 14 skipped / 0 failed / 0 pending

Nothing to fix. Stopping without further work.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Final confirmation — PR #6142 is green and ready for human merge.

Re-verified live this run on the current head 227d45eb8f: all checks pass (38 success / 14 skipped / 0 failed / 0 pending) — lock-sync, kernel-guard, version-guard, architecture-docs, both static-gates, and all 11 test shards. The lock-sync red this wake reported was on the old, superseded head 61b7663623.

No further work from the CI lane. Ready for you to merge: #6142

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request p2 scope/kernel

Projects

None yet

1 participant