[PR] fix(launcher): make the spawn health budget configurable (#4947)

CONFIGURATION.md

@@ -378,6 +378,8 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed
 | `BILI_CLAUDE_UPSTREAM` | string | unset | — | Upstream pin for the claude lane. |
 | `BILI_CODEX_STATE_MODE` | string | auto | — | codex state handling: auto (shared db via CODEX_SQLITE_HOME on modern codex, legacy overlay copies otherwise), shared, or legacy. Shared mode runs up to 16 concurrent sessions, one per-session overlay slot each; legacy keeps one exclusive overlay at a time. |
 | `BILI_ATTACH_HEALTH_DEADLINE_MS` | number | (built-in) | — | Deadline for attach health checks when joining an external daemon. |
+| `BILI_SPAWN_WAIT_MS` | number | (built-in) | — | Initial health-wait window (ms) for a freshly spawned proxy child before the launcher extends the wait. |
+| `BILI_SLOW_START_GRACE_MS` | number | (built-in) | — | Extra grace (ms) for a still-alive child that is slow to bind (slow store, AV scan, shared-store lock contention); the total spawn budget is the sum of the two. |
 | `BILI_ATTACH_EVIDENCE_GRACE_MS` | number | (built-in) | — | Grace period for attach ownership evidence. |
 | `BILI_PROVIDER_REWRITES` | string | unset | — | Provider URL rewrite rules applied before routing. |
 | `BILI_MCP_PROXY` | string | unset | — | Spawn channel: MCP proxy target for plugin-hosted tools. |
@@ -2000,6 +2002,8 @@ File keys resolve only when the matching env var is unset. Defaults in parenthes
 | `BILI_INHERITED_HTTP_PROXY` / `BILI_INHERITED_HTTPS_PROXY` / `BILI_INHERITED_ALL_PROXY` / `BILI_INHERITED_NO_PROXY` | Not user-facing — set automatically by the launcher when it spawns the proxy (#1012). The launcher strips the shell's proxy vars from both the client and the proxy child (clients must send to bili; the proxy must not have its model egress hijacked by a shell proxy), but it forwards the user's pre-strip proxy under these names so the proxy's **auxiliary egress** (MITM blind tunnels — client-side MCP/web traffic) can still ride the user's VPN. They feed only the blind-tunnel fallback tier: explicit routes / global `proxy` / `BILI_UPSTREAM_PROXY` / explicit `"upstreamProxyMode": "direct"` all still win, and a value pointing at bili's own port is dropped. Model egress is unaffected (stays direct unless explicitly configured). |
 | `BILI_ATTACH_HEALTH_DEADLINE_MS` | Health-wait deadline (ms) for dsh/opencode attach verification when the attach target is down but this process's model channel is **pinned** to it (routed `/bili/…` model traffic was observed against it): bili waits for the target to come back instead of spawning a second instance — a spawn would split the session (model traffic stays pinned, bili tools would 404 against the other instance). When the deadline elapses it fails loudly and keeps re-checking on every model request until the target returns (default `15000`). See #1365. |
 | `BILI_ATTACH_EVIDENCE_GRACE_MS` | Grace window (ms) during which a dsh/opencode attach verification probing a dead target waits for routed-channel evidence to appear before falling back to the legacy spawn path (covers the decision-before-first-request race: the probe fails at t≈0 while the first model request lands at t≈1s) (default `5000`). See #1365. |
+| `BILI_SPAWN_WAIT_MS` | Initial health-wait window (ms) for a freshly spawned proxy child before the launcher logs `still starting up` and extends the wait (default `20000`). Widen it when the proxy's pre-bind open path runs long: a large `index.db` opens whole-DB before bind (integrity check, WAL recovery — see #4939), and two lanes sharing one store cold-starting together contend for the SQLite write lock (observed 28s+ to first log line, 60s+ with zero output, on Windows with a 1.7 GiB store). Non-numeric or non-positive values fall back to the default (same `envMillis` contract as #1365). |
+| `BILI_SLOW_START_GRACE_MS` | Slow-start grace (ms) beyond `BILI_SPAWN_WAIT_MS` for a spawned child that is still ALIVE and starting (#2187: Windows spawn windows of 26-45s from AV / first-run disk activity; a large or contended shared store can push total pre-bind time past 60s). The total spawn budget is the sum of the two knobs; a child that EXITS still fails fast (default `40000`). |
 | `BILI_PERSIST` | Set `0` to disable session persistence (in-memory only, lost on restart). |
 | `BILI_PERSIST_DEBOUNCE_MS` | Debounce window for persistence writes to disk, in ms (default `500`). |
 | `BILI_PERSIST_TAIL_TOKENS` | Token budget for the persisted conversation snapshot (#401). The disk record stores the **folded view** (block summaries in place of compressed ranges) truncated to the newest messages within this budget — never the raw full history. Default `16384`; `0` disables message persistence entirely (block summaries and compressed originals still persist; `bili export` falls back to block-only rendering). Live in-memory sessions are unaffected — `bili export` of a live session is always complete. |

CONFIGURATION.zh-CN.md

@@ -378,6 +378,8 @@
 | `BILI_CLAUDE_UPSTREAM` | string | unset | — | claude 车道的上游钉死。 |
 | `BILI_CODEX_STATE_MODE` | string | auto | — | codex 状态处理方式：auto（新版 codex 走 CODEX_SQLITE_HOME 共享库，旧版回退 overlay 副本）、shared、legacy。shared 模式最多 16 个并发会话，每会话一个独立 overlay 槽位；legacy 保持一次仅一个独占 overlay。 |
 | `BILI_ATTACH_HEALTH_DEADLINE_MS` | number | (built-in) | — | 挂接外部守护进程时健康检查的截止时间。 |
+| `BILI_SPAWN_WAIT_MS` | number | (built-in) | — | 新派生的代理子进程在启动器延长等待之前的初始健康等待窗口（毫秒）。 |
+| `BILI_SLOW_START_GRACE_MS` | number | (built-in) | — | 对仍存活但仍在启动（慢速存储、杀毒扫描、共享存储锁竞争）的子进程的额外宽限（毫秒）；总启动预算为两者之和。 |
 | `BILI_ATTACH_EVIDENCE_GRACE_MS` | number | (built-in) | — | 挂接归属证据的宽限期。 |
 | `BILI_PROVIDER_REWRITES` | string | unset | — | 路由前应用的 provider URL 重写规则。 |
 | `BILI_MCP_PROXY` | string | unset | — | 派生通道：插件宿主工具的 MCP 代理目标。 |

LIFECYCLE.md

@@ -161,6 +161,7 @@
 | `BILI_ZONE_PORT` | USER zone 基址覆盖（#1660） |
 | `BILI_CLAUDE_NATIVE_PORT` / `BILI_ZCODE_PORT` | 显式 zone 端口覆盖（strict-port 语义，#1660） |
 | `BILI_ATTACH_EVIDENCE_GRACE_MS` / `BILI_ATTACH_HEALTH_DEADLINE_MS` | native attach 证据宽限窗 / 健康截止窗 |
+| `BILI_SPAWN_WAIT_MS` / `BILI_SLOW_START_GRACE_MS` | launcher spawn 初始健康等待窗 / LIVE 子进程慢启动宽限；总 spawn 预算 = 两者之和（#4947） |
 | `BILI_DSH_BIN` / `BILI_PI_BIN` | dsh / pi 可执行文件覆盖 |
 | `BILI_PLUGIN_AGENT` | `--plugin` 目标 agent 选择（默认 claude） |
 | `BILI_LOG_MASK_HOSTS` | 日志主机名掩码（默认开；`=0` 本地调试退出） |

src/launcher.ts

@@ -58,7 +58,7 @@ import {
 import { TWIN_ADOPT_EXIT_CODE } from "./listen-policy.js";
 import { tcpConnectable } from "./liveness.js";
 import { selfPackageRoot, isBiliPiEntry, ompPluginLoadedFrom, dshNativeInstalled, claudeNativeInstalled, piPackageStrings } from "./plugin-install.js";
-import { applyOmpFirstEventTimeout } from "./agent/native-bootstrap.js";
+import { applyOmpFirstEventTimeout, envMillis } from "./agent/native-bootstrap.js";
 import { logHostEvent } from "./agent/host-log.js";
 import { compactionDisablePatchRows } from "./agent/dsh-compaction-disable.js";
 import { log as teeLog, logTimestamp } from "./logger.js";
@@ -171,12 +171,17 @@ type BaseClientName = "claude" | "codex" | "pi" | "omp" | "opencode" | "hermes"
 
 const HEALTH_PATH = "/__bili/health";
 const HEALTH_POLL_INTERVAL_MS = 200;
-const SPAWN_WAIT_MS = 20000;
+// Env knobs (envMillis contract, #1365): large stores (whole-DB pre-bind open,
+// #4939) and two lanes sharing one index.db (write-lock contention on cold
+// start) have been observed exhausting the fixed budget, kill-spawn looping
+// into the same contention. Widen via env instead of patching dist; bad
+// values fall back to the defaults.
+const SPAWN_WAIT_MS = envMillis(process.env, "BILI_SPAWN_WAIT_MS", 20000);
 // #2187: grace beyond the initial wait for a LIVE child still starting.
 // Spawn-window startups have been observed taking 26-45s on slow Windows
 // machines (AV / first-run disk activity); the one-shot 20s budget misread
 // that as failure and left the lane degraded for the whole session.
-const SLOW_START_GRACE_MS = 40_000;
+const SLOW_START_GRACE_MS = envMillis(process.env, "BILI_SLOW_START_GRACE_MS", 40_000);
 // Total wait for a live child: initial window + grace. A child that EXITS or
 // fails to spawn still fails fast (the poll loop breaks on exit/error).
 export const SPAWN_BUDGET_MS = SPAWN_WAIT_MS + SLOW_START_GRACE_MS;

tests/issue4947-launcher-spawn-budget-env-defaults.test.ts

@@ -0,0 +1,16 @@
+// Invalid spawn-budget env values must fall back to the built-in 60s budget
+// (envMillis contract, #1365): a bad knob must never shrink or zero the
+// budget. Separate file so this node --test process resolves the module
+// constants with only the bad values set.
+
+process.env.BILI_SPAWN_WAIT_MS = "not-a-number";
+process.env.BILI_SLOW_START_GRACE_MS = "0";
+
+import test from "node:test";
+import assert from "node:assert/strict";
+
+const { SPAWN_BUDGET_MS } = await import("../src/launcher.ts");
+
+test("invalid BILI_SPAWN_WAIT_MS / BILI_SLOW_START_GRACE_MS fall back to the built-in budget", () => {
+    assert.equal(SPAWN_BUDGET_MS, 60000);
+});

tests/issue4947-launcher-spawn-budget-env.test.ts

@@ -0,0 +1,90 @@
+// Spawn-budget env knobs (#4939 and the cross-lane shared-store variant): a
+// large index.db opens whole-DB before bind, and two lanes sharing one store
+// contend for the write lock on cold start — pre-bind startup past the fixed
+// 60s budget makes the launcher kill-spawn loop into the same contention.
+// The knobs must resolve at module load, and the resolved budget must actually
+// be consumed by the wait loop. This file runs in its own node --test process
+// so the module-level constants pick up the env set below.
+
+process.env.BILI_SPAWN_WAIT_MS = "5000";
+process.env.BILI_SLOW_START_GRACE_MS = "3000";
+
+import test, { after } from "node:test";
+import assert from "node:assert/strict";
+import * as fs from "node:fs";
+import * as os from "node:os";
+import * as path from "node:path";
+import type { SpawnChild, SpawnFn } from "../src/launcher.ts";
+
+// Private XDG roots BEFORE importing the launcher: the starting marker is one
+// GLOBAL file per state dir, so any concurrent launcher activity in the shared
+// dir (other suites, other processes) makes this file wait behind it and burn
+// its budget ticks past the assertions below.
+const xdgRoot = fs.mkdtempSync(path.join(os.tmpdir(), "bili-issue4947-env-"));
+process.env.XDG_CACHE_HOME = path.join(xdgRoot, "cache");
+process.env.XDG_CONFIG_HOME = path.join(xdgRoot, "config");
+process.env.XDG_DATA_HOME = path.join(xdgRoot, "data");
+process.env.XDG_STATE_HOME = path.join(xdgRoot, "state");
+
+const { SPAWN_BUDGET_MS, ensureProxyRunning } = await import("../src/launcher.ts");
+
+function makeFakeChild(pid: number): SpawnChild {
+    return {
+        pid,
+        unref() {},
+        kill() {
+            return true;
+        },
+        on() {},
+    };
+}
+
+function fakeClock() {
+    let ticks = 0;
+    return {
+        now: () => ticks * 1000,
+        sleep: () => {
+            ticks += 1;
+            return Promise.resolve();
+        },
+        ticks: () => ticks,
+    };
+}
+
+test("BILI_SPAWN_WAIT_MS / BILI_SLOW_START_GRACE_MS resolve the spawn budget at module load", () => {
+    assert.equal(SPAWN_BUDGET_MS, 5000 + 3000);
+});
+
+test("the resolved env budget is consumed by the wait loop, not just exported", async () => {
+    const clock = fakeClock();
+    const spawnImpl: SpawnFn = () => makeFakeChild(42470);
+    await assert.rejects(
+        ensureProxyRunning(
+            // Private port: the port log (.tmp/bili-proxy-<port>.log) is shared
+            // per port across processes and rotates under writers (#4811).
+            { host: "127.0.0.1", port: 39787, passthrough: false, debug: false },
+            {
+                fetchImpl: async () => ({ ok: false }),
+                fetchHealthInfo: async () => ({ ok: false }),
+                spawnImpl,
+                now: clock.now,
+                sleep: clock.sleep,
+                readInstanceFile: () => undefined,
+            },
+        ),
+        new RegExp(`did not become healthy within ${SPAWN_BUDGET_MS}ms`),
+    );
+    assert.ok(clock.ticks() >= 8, `the env budget (8s) must be fully consumed, got ${clock.ticks()}s`);
+    // Upper bound vs the #2187 baseline: with the built-in 60s budget this same
+    // never-healthy scenario consumes >=59 ticks; a much smaller total proves
+    // the env budget (not the default) drove the deadline.
+    assert.ok(clock.ticks() < 59, `the wait must stay below the built-in 60s budget baseline (>=59 ticks), got ${clock.ticks()}s`);
+});
+
+after(() => {
+    delete process.env.XDG_CACHE_HOME;
+    delete process.env.XDG_CONFIG_HOME;
+    delete process.env.XDG_DATA_HOME;
+    delete process.env.XDG_STATE_HOME;
+    fs.rmSync(xdgRoot, { recursive: true, force: true });
+});

website/config-reference/env-only.yaml

@@ -186,6 +186,22 @@ entries:
     group: envonly
     desc_en: "Deadline for attach health checks when joining an external daemon."
     desc_zh: "挂接外部守护进程时健康检查的截止时间。"
+  - key: BILI_SPAWN_WAIT_MS
+    type: number
+    default: "(built-in)"
+    kind: env-only
+    audience: user
+    group: envonly
+    desc_en: "Initial health-wait window (ms) for a freshly spawned proxy child before the launcher extends the wait."
+    desc_zh: "新派生的代理子进程在启动器延长等待之前的初始健康等待窗口（毫秒）。"
+  - key: BILI_SLOW_START_GRACE_MS
+    type: number
+    default: "(built-in)"
+    kind: env-only
+    audience: user
+    group: envonly
+    desc_en: "Extra grace (ms) for a still-alive child that is slow to bind (slow store, AV scan, shared-store lock contention); the total spawn budget is the sum of the two."
+    desc_zh: "对仍存活但仍在启动（慢速存储、杀毒扫描、共享存储锁竞争）的子进程的额外宽限（毫秒）；总启动预算为两者之和。"
   - key: BILI_ATTACH_EVIDENCE_GRACE_MS
     type: number
     default: "(built-in)"