[PR] fix(boot): drop redundant synchronous corpus decode on pre-listen path (#4515) by ranxianglei · Pull Request #4539 · ranxianglei/billion-context

ranxianglei · 2026-10-24T11:59:54Z

What

Slice 1 of the #4515 fix (a full-scan of the historical corpus on the pre-listen boot path blows the 60 s native spawn budget → silent direct-connect on big cold stores). This PR ships the no-decision parts; the structural root-fix and config-surface parts are gated on owner decisions (see Remaining) and will land in this same PR before merge.

Changes

Skip the redundant synchronous corpus decode at maxSessions==0 (src/persist.ts bootBounded). At the Bound startup session loading and preserve historical recovery #2803 default (sessionWarmMax()==0) nothing is warmed, yet the old code decoded+parsed a 64 MB-decode window of the newest state files synchronously only to pop every candidate. In unified mode that bookkeeping is already covered by seedIndexedAliases (index rows) plus the post-boot background scan, so the window is skipped entirely when nothing is warmed.
Per-phase startup timing (src/session.ts initSessions, src/persist.ts bootBounded). Splits pre-listen cost into store open (index.db open / WAL recovery) vs bounded boot, and within boot into walk / parseWindow / indexUnion. A slow cold start now names which phase ate the budget instead of dying silently under it.
Test observability: lastBootSyncParsedFilesForTest() exposes how many state files the last bounded boot decoded synchronously (already-tracked parsedFiles.size).

Behavior change (flagged)

At maxSessions==0 the returned warm set is identical (empty). One nuance: label-resume of an unrecorded file (no index row) that previously landed in the sync window now resolves only after the background scan reaches it (sub-second, async) rather than synchronously at boot. Recorded sessions are unaffected (always index-seeded). Warm path (maxSessions>0) and legacy mode are untouched.

Tests

New tests/issue4515-boot-no-sync-warm.test.ts: pins nothing-decoded-synchronously at maxSessions==0, recorded-label resolves from the index, unrecorded-label fills in via the background scan. Negative-controlled (re-enabling the window makes the assertion fail).
Targeted suites green locally: bounded-storage-scale, bounded-storage, issue3199-overview-cold-scan, issue404-restore-timestamps (+ new). Full-suite verdict = CI test lanes on this head.

Remaining (same PR, gated on owner decision)

Decision A — structural root-fix: full-mode boot enumerates sessions from index rows only; legacy file walk becomes degrade-only fallback. Changes startup semantics → owner sign-off.
Decision B — safety net: make SPAWN_BUDGET_MS configurable (env > config > default 60000) + stage-aware timeout log. New config surface → owner sign-off.
Warm-loop over-budget handling (pure-index union loads before size-checking); background-scan idle-gating/cancellation.

References #4515

…t path At maxSessions==0 (the #2803 default) the bounded boot decoded+parsed a 64MB-decode window of the newest state files synchronously only to pop every candidate - pure waste on the path that gates listen() and, on big stores, a leading contributor to cold starts blowing the native spawn budget (#4515). In unified mode the whole corpus alias/canonical bookkeeping is already covered by seedIndexedAliases (index rows) plus the post-boot background scan, so the sync window is skipped entirely when nothing is warmed. Also add per-phase startup timing (store open vs bounded boot; walk / parseWindow / indexUnion) so a slow cold start names which phase ate the budget instead of dying silently under it. Behavior change (flagged): at maxSessions==0 the returned warm set is identical (empty), but label-resume of an UNRECORDED file (no index row) that previously landed in the sync window now resolves only after the background scan reaches it (sub-second, async) rather than synchronously at boot. Recorded sessions are unaffected (always index-seeded). Warm path (maxSessions>0) and legacy mode are untouched. Regression test pins: nothing decoded synchronously at maxSessions==0, recorded labels resolve from the index, unrecorded labels fill in via the background scan. Negative-controlled (re-enabling the window fails the assertion). References #4515

github-actions · 2026-10-24T12:01:53Z

📦 Built Package Artifact

Branch: 2026-10-24_boot-scan-off-critical-path (60d85af)

Latest build — install this exact version

npm install -g billion-context@0.0.6179-pr.4539.dd8ae30

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit	version	build
`60d85af`	`billion-context@0.0.6179-pr.4539.dd8ae30`	build
`4ecf66d`	`billion-context@0.0.6093-pr.4539.b48ba85`	build
`58e803b`	`billion-context@0.0.5997-pr.4539.0328dc2`	build
`25d7b45`	`billion-context@0.0.5481-pr.4539.9c73ef3`	build

Download artifact

Download the artifact from the Actions run
Extract the tarball and install:

tar xzf billion-context-pr4539.tgz
npm install -g package

Install instructions are refreshed on each push.

ranxianglei · 2026-10-24T13:11:44Z

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重：不重复。 本 PR 是 #4515（大 store 冷缓存启动超 60s spawn 预算 → 静默直连）的修复切片 1，与 #4515 线程 floor 2/3 已核实的机制一一对应；无同症状同机制的并行修复条目。检索范围：open+closed issue/PR，按失败签名（bootBounded 同步解码窗口 / pre-listen I/O / SPAWN_BUDGET_MS）。

标签：bug / pi / scope/storage / p1（与 #4515 对齐：p1 = 「出网前必压缩」核心承诺在大 store 冷重启下被静默整段关闭；pi 为复现车道，机制对所有 native 车道通用；scope/storage = persist/session 启动路径）。

开始按 PR Review 规范独立审核本 PR（审原 issue 方向 → 回归清点 → 代码 → 测试有效性 → CI），结论稍后按八问体例回贴。另注意 #4515 floor 5 的新证据（报告者两次失败阻塞在 index.db 打开阶段或更早）——本切片的分阶段计时正是为钉死该卡点，我会在审核结论里评估这一片对报告者实际故障的覆盖度。

ranxianglei · 2026-10-24T13:55:59Z

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

独立审核结论(按结构化八问)。审核对象:head 25d7b450d(单 commit),已拉分支逐行读 diff、本地定向验证、查 CI。

1. 是什么问题:默认配置(sessionWarmMax()==0)下,bootBounded 在 listen 之前仍会同步解码+解析最新 ~64MB 解码窗口的 state 文件,而解析出的候选随后全部被弹出——纯浪费的启动关键路径 I/O。大冷存储上这段耗时吃掉 launcher 的 60s spawn 预算 → 子进程被 SIGKILL → 插件静默直连(#4515)。

2. 什么原因导致的:热窗口解析在 bootBounded 里无条件执行,与「是否真的有人要预热」解耦;且启动各阶段无计时,慢冷启动在预算下静默死亡、无法定位是哪个阶段吃了时间。

3. 影响范围与严重程度:p1 可用性类(压缩能力对宿主整体不可用,直到宿主重启)。触发条件:大会话存储 + 冷启动 + unified 模式。本切片消除的是默认配置下的同步解码;warm 路径与 legacy 模式不受影响。注意:#4515 floor 5 的证据表明报告者的两次真实失败卡在更早的 index.db open/WAL 阶段(60s 内零日志行)——本切片单独不能恢复他的启动,但新增的 store open Xms 计时正是用来证实/证伪这一点的。

4. 怎么修复的:把同步热窗口解析 gate 到 maxSessions > 0;maxSessions==0 时完全跳过,未记录文件的 alias/canonical 记账交给既有的后台扫描(同一套 noteCanonical/upsertLegacyAlias 规则,总磁盘 I/O 不变);加 per-phase 计时(store open / walk / parseWindow / indexUnion)+ 测试钩子钉住「零同步解码」。

5. 是否有回归问题:无。逐项审过的既有行为:

maxSessions==0 返回的 warm 集:不变(空);
已记录会话的 label-resume:不变,仍由 seedIndexedAliases(index 行)同步解析,不依赖文件解析;
有意变更(已在 PR 描述披露):未记录文件的 label-resume 从「同步窗口内同步可解析」变为「后台扫描到达后可解析」——有界竞态,仅影响扫描尚未扫到的文件。一点措辞修正建议:PR 描述的「sub-second」偏乐观——后台扫描按 readdir 顺序而非 mtime 顺序遍历(walkJsonFiles 无排序),到达时间取决于文件系统顺序,建议改成「后台扫描到达后」;
legacy 模式(replicasAvailable() false):字节级不变;
warm 路径(maxSessions>0):逻辑逐行相同,仅移入 if 并加计时;
启动日志行格式变化(追加计数字段 + #4515 标记):全仓 grep 确认无测试/消费者断言旧字符串;
fail-fast 语义、去重键、错误处理:均未触碰。

6. 是否破坏缓存:不涉及。前缀缓存/重试/增量游标均不受影响;boot 期 alias 表只影响 label 解析时机。

7. 是否改了设计方向:本切片不改架构,是明确标注的 no-decision 部分。结构性根修(Decision A:full-mode 只从 index 行枚举会话、legacy walk 降为 degrade-only)与 SPAWN_BUDGET_MS 可配置化(Decision B)按作者计划仍在同一 PR 内等 owner 拍板。提醒:结合 #4515 floor 5,Decision A 的范围应覆盖 index.db open/WAL recovery 阶段,而不只是语料枚举——否则报告者的实际失败场景仍不在修复面内。

8. 是否引入了新的配置:本切片无新 env/config/默认值。(Decision B 若通过会带来配置面变更,届时需按 config-surface 纪律单独说明。)

验证证据:

本地定向:新测试通过;负控制实测有效(把 guard 改回无条件执行 → lastBootSyncParsedFilesForTest()==0 断言如预期失败);boot 相关 5 个套件(bounded-storage-scale / bounded-storage / issue3199-overview-cold-scan / issue404-restore-timestamps / issue4515)共 19 项全绿;
CI:第一波(11:59Z)红——7 个 shard 中唯一失败是 storage-engine-proofs 的 golden drive 预检报「node_modules/acp-kernel/dist 旧于 kernel/src(test: rebaseline storage-render goldens after remove-absorb (#3309) #3325)」,属 CI 构建顺序 flake,与本 PR diff 无关;第二波重跑同 head 全绿(CI + 全部 e2e 车道 + Theorem Lane)。红 run 日志同时可见新计时正常工作(walk 6ms, parseWindow 0ms, indexUnion 0ms / store open 0ms + bounded boot 15ms);
base 落后 master 414 commit;master 侧对 persist.ts/session.ts 的改动与 PR 触碰区域无语义重叠,GitHub mergeable=True——合并前建议 rebase + 重跑 CI。

判定:切片 1 代码质量通过(LGTM),负控制有牙、披露完整、CI 绿。按作者声明的计划,合并应等 Decision A/B 在同一 PR 内落地 + rebase 之后。两个待 owner 决策(A/B)我已在此线程外 #4515 有完整上下文,这里不重复选项对比,等你拍板。

…p-merge sweep)

master window commits 293ae65/de5e03659 bumped kernel/package.json to 0.0.165 without regenerating the lockfiles; lock-sync re-resolution then rewrites the stale recorded versions (root 0.0.163, kernel 0.0.164) and fails. Pure recorded-version sync via npm install --package-lock-only; no dependency or semantic change.

ranxianglei · 2026-10-31T00:35:18Z

方案被后续实现取代：启动路径的重复解码问题已由 C 类整合（#5198）与常驻最小化（D2）覆盖。冗余关闭。

ranxianglei · 2026-10-31T04:47:10Z

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

收到，按此收工。引用已对照 git 核实:C 类整合 = 已合并的 PR #5198(2026-10-30_classC-memory-consolidation,含 p4532/p4540/p4750/p4765/p5156);常驻最小化 D2 = #4788 D2 commit(单会话常驻内存封顶、turn 间释放、disk-proof 门)。

留一条事实记录(不影响你的决策,只是别让账目失真):当前 master 的 bootBounded 热窗口同步解码循环仍是无条件的(没有 maxSessions==0 跳过),且 C 类内 #4942 的范围说明明确写了「sqlite/file paths unchanged」——所以本 PR 移除的那几行(默认配置下无人预热时跳过同步解码)在 master 上原样存在;D2 管的是运行时常驻内存,不覆盖 pre-listen 的解码窗口。若判定残留可接受则无需动作;若日后想保留这个 gate,它是自包含的几行改动,随时可从本分支摘取。

关闭由你执行(规则:agent 不关单)。本 PR 我不再触碰。

ranxianglei changed the title ~~fix(boot): drop redundant synchronous corpus decode on pre-listen path (#4515)~~ [PR] fix(boot): drop redundant synchronous corpus decode on pre-listen path (#4515) Oct 24, 2026

ranxianglei added bug Something isn't working p1 pi scope/storage labels Oct 24, 2026

ranxianglei mentioned this pull request Oct 24, 2026

[PR] fix(#4516): degraded direct fallback — add the missing oversize refusal, self-heal retry, and transition logging #4546

Merged

ci-check added 4 commits October 26, 2026 10:18

merge ci-fixes into 2026-10-24_boot-scan-off-critical-path (#4599 hel…

58e803b

…p-merge sweep)

merge ci-fixes into 2026-10-24_boot-scan-off-critical-path (#4599 hel…

4ecf66d

…p-merge sweep)

Merge remote-tracking branch 'origin/ci-fixes' into sweep-wip

60d85af

ranxianglei mentioned this pull request Oct 27, 2026

[PR] fix(launcher): make the spawn health budget configurable (#4947) #4948

Merged

ranxianglei closed this Oct 31, 2026

ranxianglei mentioned this pull request Oct 31, 2026

"Agent Half-Dead" Cluster Breakthrough Entry Point + 48-Hour Active Surface A–N Classification Master Ledger (2026-10-29 | 『代理半死不活』簇攻坚入口 + 48小时活跃面 A–N 归类总账(2026-10-29~31) #5297

Open

This was referenced Nov 4, 2026

Service crashed, sessions exploded instantly. | 服务崩溃,会话瞬间爆炸 #5219

Open

[PR] fix(#5219): bound the bounded-boot parse window by decoded-file count #5634

Open

Navigation Menu

Repository navigation

[PR] fix(boot): drop redundant synchronous corpus decode on pre-listen path (#4515) - #4539

[PR] fix(boot): drop redundant synchronous corpus decode on pre-listen path (#4515)#4539
ranxianglei wants to merge 5 commits into
masterfrom
2026-10-24_boot-scan-off-critical-path

ranxianglei commented Oct 24, 2026

Uh oh!

github-actions Bot commented Oct 24, 2026 •

edited

Loading

Uh oh!

ranxianglei commented Oct 24, 2026

Uh oh!

ranxianglei commented Oct 24, 2026

Uh oh!

ranxianglei commented Oct 31, 2026

Uh oh!

ranxianglei commented Oct 31, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ranxianglei commented Oct 24, 2026

What

Changes

Behavior change (flagged)

Tests

Remaining (same PR, gated on owner decision)

Uh oh!

github-actions Bot commented Oct 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 Built Package Artifact

Latest build — install this exact version

Recent builds (commit → version)

Download artifact

Uh oh!

ranxianglei commented Oct 24, 2026

Uh oh!

ranxianglei commented Oct 24, 2026

Uh oh!

ranxianglei commented Oct 31, 2026

Uh oh!

ranxianglei commented Oct 31, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions Bot commented Oct 24, 2026 •

edited

Loading