ranxianglei · Oct 24, 2026 · Oct 24, 2026 · Oct 24, 2026 · Oct 24, 2026 · Oct 24, 2026
diff --git a/CONFIGURATION.md b/CONFIGURATION.md
@@ -360,7 +360,7 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed
 | `BILI_STORAGE_BACKEND` | sqlite\|mysql | sqlite | — | Env tier for storage.backend (#3931): sqlite (default) or mysql (fully wired, #3388 steps 2+3 - saves ingest through the async shadow queue, sync reads ride the JSON envelope, replicas/ext/GC via the async twins; export too). Wins over the file tier. |
 | `BILI_MYSQL_DSN` | string | unset | — | MySQL DSN (mysql://user:pass@host:3306/db) for storage.backend="mysql" (#3388, #3931). Wins over the file key storage.mysql.dsn. Empty = the mysql backend cannot open; the documented error names both knobs. The pool opens lazily; closeQuietlyAsync drains the ingest queue and closes it. |
 | `BILI_STORAGE_ARCHIVE_AFTER_DAYS` | integer | 365 | — | GC age sweep window in days (#3463): sessions idle longer than this are archived (one JSON line appended to the single archive log (archive/archive.jsonl) + tombstone row + content/media refs released). 0 disables the age sweep; the manual entry POST /__bili/storage/gc always runs. Config-file key storage.archiveAfterDays is the persistent form. |
-| `BILI_STORAGE_QUICK_CHECK` | string | on | — | Storage integrity self-check at open (#4175): every index.db open runs PRAGMA quick_check BEFORE migrations; on failure the shadow layer quarantines the file set (index.db/-wal/-shm renamed to .corrupt-<ts>) and reopens fresh (JSON envelopes remain the recovery source). Set 0 only for boot-time-sensitive huge deployments where the check's cost matters. |
+| `BILI_STORAGE_QUICK_CHECK` | string | on | — | Storage integrity self-check (#4175/#4609), three lanes: unchanged since a passing check (size+mtime fingerprint, index.db.quickcheck) -> skipped; <=256 MiB -> PRAGMA quick_check inline BEFORE migrations, and on failure the shadow layer quarantines the file set (index.db/-wal/-shm renamed to .corrupt-<ts>) and reopens fresh (JSON envelopes remain the recovery source); >256 MiB -> the port binds FIRST and the scan runs in a background worker after listen (#4609: a 14 GB db took 260 s inline against the launcher's hard 60 s spawn budget) - a deferred failure disables unified reads for the process lifetime (envelope fallback) and writes a marker so the NEXT open quarantines the set. Set 0 to skip every check. |
 | `BILI_SNAPSHOT_SHADOW` | boolean | true (on) | — | Snapshot-render shadow observability (#3756 follow-up): compute the outbound snapshot view on every armed turn WITHOUT using it, classify the diff against the legacy outbound, and surface per-session counters in the web panel + the snapshot-shadow diagnostics topic. 0/false/off disables; 1/true/on enables; otherwise the config-file snapshot.shadow boolean applies (default on). The render lane itself stays rollout-gated (BILI_ROLLOUT_SNAPSHOT_RENDER), so the shadow lane never changes outbound bytes - it is the dry-run evidence that flips the rollout later. |
 | `BILI_CONFIG_FILE` | string | unset (XDG config path) | — | Override the config file path (path relocation). |
 | `BILI_SESSIONS_DIR` | string | unset (XDG state path) | — | Where session records are stored (path relocation). |

diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md
@@ -360,7 +360,7 @@
 | `BILI_STORAGE_BACKEND` | sqlite\|mysql | sqlite | — | storage.backend 的环境变量层（#3931）：sqlite（默认）或 mysql（已接线，#3388 第二+三步 —— 保存经异步影子队列入 mysql,同步读走 JSON 信封,replicas/ext/GC 走异步孪生;export 同样支持）。优先于文件层。 |
 | `BILI_MYSQL_DSN` | string | unset | — | storage.backend="mysql" 的 MySQL DSN（mysql://user:pass@host:3306/db）（#3388，#3931）。优先于文件键 storage.mysql.dsn。留空 = mysql 后端无法打开;文档化错误会列出两个旋钮。连接池惰性打开;closeQuietlyAsync 排空入队事务并关闭。 |
 | `BILI_STORAGE_ARCHIVE_AFTER_DAYS` | integer | 365 | — | GC 年龄清扫窗口（天，#3463）：闲置超过该时长的会话被归档（单文件归档日志 archive/archive.jsonl 追加一行 + 墓碑行保留标签 + 释放内容/媒体引用）。0 关闭年龄清扫；手动入口 POST /__bili/storage/gc 始终可用。配置文件键 storage.archiveAfterDays 为持久形式。 |
-| `BILI_STORAGE_QUICK_CHECK` | string | on | — | 存储开库自检（#4175）：每次 index.db 打开在迁移之前先跑 PRAGMA quick_check;失败时影子层把文件组（index.db/-wal/-shm 改名为 .corrupt-<ts>）隔离并重建新库（JSON 信封仍是恢复源）。仅当超大库的启动时间敏感时才设 0 跳过。 |
+| `BILI_STORAGE_QUICK_CHECK` | string | on | — | 存储完整性自检（#4175/#4609），分三档：自上次通过检查以来未变化（size+mtime 指纹，index.db.quickcheck）→ 跳过；≤256 MiB → 迁移之前内联跑 PRAGMA quick_check，失败时影子层把文件组（index.db/-wal/-shm 改名为 .corrupt-<ts>）隔离并重建新库（JSON 信封仍是恢复源）；>256 MiB → 端口先绑定、扫描在 listen 之后由后台 worker 线程执行（#4609：14 GB 库内联要 260 秒，远超启动器硬性的 60 秒 spawn 预算），后台检查失败则本进程生命周期内关闭统一读（回退信封加载）并写标记，下次开库时隔离文件组。设 0 跳过一切检查。 |
 | `BILI_SNAPSHOT_SHADOW` | boolean | true (on) | — | 快照渲染影子对账（#3756 后续）：每轮已折叠会话都用引擎现算一次出站快照视图但不使用，与 legacy 实际出站比对分类，逐会话计数进 web 面板与 snapshot-shadow 诊断主题。0/false/off 关闭；1/true/on 开启；否则读配置文件 snapshot.shadow 布尔（默认开）。渲染车道本身仍受灰度开关控制（BILI_ROLLOUT_SNAPSHOT_RENDER），影子车道绝不改变出站字节 —— 它是后续翻转灰度的干跑证据。 |
 | `BILI_CONFIG_FILE` | string | unset (XDG config path) | — | 覆盖配置文件路径（路径重定位）。 |
 | `BILI_SESSIONS_DIR` | string | unset (XDG state path) | — | 会话记录的存放目录（路径重定位）。 |

diff --git a/kernel/package-lock.json b/kernel/package-lock.json
diff --git a/package-lock.json b/package-lock.json
diff --git a/src/storage/quick-check-worker.ts b/src/storage/quick-check-worker.ts
@@ -0,0 +1,138 @@
+/**
+ * #4609: deferred integrity check for large index.db files.
+ *
+ * quick_check is O(db size) — 14 GB took 260 s in the field (#4609) — and both
+ * engines expose only synchronous APIs, so running it on the main thread
+ * post-listen would just move the multi-minute event-loop freeze onto live
+ * traffic. It therefore runs in a worker thread that opens its OWN readonly
+ * connection to the same WAL-mode file (concurrent reader + writer is the
+ * supported SQLite topology; nothing here ever takes a write lock).
+ *
+ * Outcome protocol (three-way, deliberately):
+ *   - worker posts a verdict message     → genuine pass/fail (fail feeds the
+ *                                          quarantine-marker path in store.ts)
+ *   - worker dies before posting         → INCONCLUSIVE (null): log and retry
+ *                                          at the next open. A dead worker
+ *                                          must NEVER be treated as corruption —
+ *                                          quarantining over an OOM is data loss.
+ *
+ * The worker is unref'd: it never holds the process open, so a shutdown that
+ * outruns the scan simply drops it (next open re-runs; no cache was written).
+ */
+import { Worker } from "node:worker_threads";
+import { log as loggerLog } from "../logger.js";
+import type { SqliteEngineName } from "./driver.js";
+
+export interface QuickCheckResult {
+    ok: boolean;
+    durationMs: number;
+    detail?: string;
+}
+
+interface WorkerVerdictMessage {
+    ok?: unknown;
+    durationMs?: unknown;
+    detail?: unknown;
+}
+
+/** Self-contained worker program (plain JS string — no module imports beyond
+ *  node builtins, so it bundles verbatim). argv layout: [anchor, dbPath,
+ *  engine]; `anchor` is the dist entry path the MAIN process resolved its
+ *  engine against — host-provided better-sqlite3 must resolve the same way
+ *  inside the worker (createRequire anchored at the same file). An EVAL
+ *  worker prepends the literal marker "[worker eval]" to process.argv, so
+ *  the payload is located AFTER that marker, not at a fixed index. */
+const WORKER_SOURCE = `"use strict";
+const { parentPort } = require("node:worker_threads");
+if (!parentPort) process.exit(4);
+const MARKER = process.argv.indexOf("[worker eval]");
+if (MARKER === -1) process.exit(4);
+const [anchor, dbPath, engine] = process.argv.slice(MARKER + 1);
+const started = Date.now();
+let db;
+try {
+    // Specifier through a constant: this string is bundled VERBATIM into dist,
+    // where scripts/check-dist-external.mjs text-scans for quoted-require forms;
+    // variable indirection is that gate's sanctioned invisible path (its header
+    // documents opaque specifiers as invisible by design). Do NOT inline the
+    // specifier back into the call — it re-trips the gate.
+    const SQLITE_SPEC = "node:sqlite";
+    if (engine === SQLITE_SPEC) {
+        const { DatabaseSync } = require(SQLITE_SPEC);
+        db = new DatabaseSync(dbPath, { readOnly: true });
+    } else {
+        const { createRequire } = require("node:module");
+        const BetterSqlite3 = createRequire(anchor)("better-sqlite3");
+        db = new BetterSqlite3(dbPath, { readonly: true });
+    }
+} catch {
+    // Engine load failure inside the worker is environmental, NOT an
+    // integrity verdict — exit without posting (inconclusive).
+    process.exit(3);
+}
+try {
+    const rows = db.prepare("PRAGMA quick_check").all();
+    const verdict = rows.map((row) => String(row.quick_check ?? "")).join(" ").trim();
+    parentPort.postMessage({ ok: verdict === "ok", durationMs: Date.now() - started, detail: verdict === "ok" ? undefined : verdict.slice(0, 400) });
+} catch (err) {
+    // Statement-level failure (e.g. NOTADB on a corrupt header) is a genuine
+    // integrity failure — same class as the sync path's catch in store.ts.
+    parentPort.postMessage({ ok: false, durationMs: Date.now() - started, detail: String(err).slice(0, 400) });
+} finally {
+    try { db.close(); } catch { /* best-effort */ }
+}
+`;
+
+const inflightChecks = new Map<string, Promise<QuickCheckResult | null>>();
+
+/** Runs (or reuses) the deferred quick_check for `dbPath`. Resolves to null
+ *  when the scan was inconclusive (worker death) — callers must NOT treat
+ *  null as corruption. Never rejects. */
+export function runDeferredQuickCheck(dbPath: string, engine: SqliteEngineName, anchorPath: string): Promise<QuickCheckResult | null> {
+    const existing = inflightChecks.get(dbPath);
+    if (existing !== undefined) return existing;
+    const promise = new Promise<QuickCheckResult | null>((resolve) => {
+        let settled = false;
+        const finish = (result: QuickCheckResult | null) => {
+            if (!settled) {
+                settled = true;
+                resolve(result);
+            }
+        };
+        let worker: Worker;
+        try {
+            worker = new Worker(WORKER_SOURCE, { eval: true, argv: [anchorPath, dbPath, engine] });
+        } catch (err) {
+            loggerLog("warn", `[storage] deferred quick_check could not start a worker for ${dbPath}: ${String(err)} — inconclusive, retried at the next open`);
+            finish(null);
+            return;
+        }
+        worker.unref();
+        worker.on("message", (msg: unknown) => {
+            if (typeof msg !== "object" || msg === null) {
+                finish(null);
+                return;
+            }
+            const m = msg as WorkerVerdictMessage;
+            if (typeof m.ok !== "boolean" || typeof m.durationMs !== "number") {
+                finish(null);
+                return;
+            }
+            finish({
+                ok: m.ok,
+                durationMs: m.durationMs,
+                ...(typeof m.detail === "string" && m.detail.length > 0 ? { detail: m.detail } : {}),
+            });
+        });
+        worker.on("error", (err) => {
+            loggerLog("warn", `[storage] deferred quick_check worker crashed for ${dbPath}: ${String(err)} — inconclusive, retried at the next open`);
+            finish(null);
+        });
+        worker.on("exit", () => finish(null));
+    });
+    inflightChecks.set(dbPath, promise);
+    void promise.then(() => {
+        inflightChecks.delete(dbPath);
+    });
+    return promise;
+}
diff --git a/src/storage/shadow.ts b/src/storage/shadow.ts
@@ -25,9 +25,7 @@ import { log as loggerLog } from "../logger.js";
 import { planIngest, planIngestDelta, loadLegacyView, completeBlockContents, type LegacySessionLike } from "./ingest.ts";
 import type { BlockView } from "../session.js";
 import { resolveStorageMode, resolveStorageWalJournalSizeLimit, type StorageMode } from "../knobs.ts";
-import { UnifiedStore, StorageIntegrityError, type CheckpointRow, type SessionSummaryRow, type WebSummaryRow } from "./store.ts";
-import { renameSync } from "node:fs";
-import path from "node:path";
+import { UnifiedStore, StorageIntegrityError, quarantineCorruptIndexSet, type CheckpointRow, type SessionSummaryRow, type WebSummaryRow } from "./store.ts";
 import { resolveBackendName, backendSelectionNotice, selectedBackendRaw, openBackend, type StorageBackend } from "./backend.ts";
 import { canonicalize, sha256 } from "./canonical.ts";
 import { recordSidecarInstance, recordSidecarOpenFailure } from "./runtime-status.ts";
@@ -190,9 +188,25 @@ export class UnifiedShadow {
             if (notice) loggerLog("warn", notice);
         }
         let store: UnifiedStore;
+        // #4609: when a DEFERRED (large-db) quick_check fails minutes after
+        // boot, the handle is live and cannot be quarantined in place
+        // (Windows EBUSY, #4347) — this sink flips unified reads off for
+        // the process lifetime instead (every load falls back to the JSON
+        // envelope, the recovery source #4175 designated); ingests stop
+        // too (record() early-returns on disabled) while envelopes keep
+        // receiving every save. The marker written by the store defers the
+        // actual quarantine to the next open. Declared OUTSIDE the open
+        // try/catch: the handler is attached after construction, below it.
+        const integrityFailureSink: { handler?: (detail: string) => void } = {};
         try {
+            const openOpts = {
+                walJournalSizeLimitBytes: walLimit,
+                onQuickCheckFailure: (detail: string) => {
+                    integrityFailureSink.handler?.(detail);
+                },
+            };
             try {
-                store = UnifiedStore.open(`${dir}/index.db`, undefined, { walJournalSizeLimitBytes: walLimit });
+                store = UnifiedStore.open(`${dir}/index.db`, undefined, openOpts);
             } catch (err) {
                 // #4175: quick_check failed — the file set is provably corrupt.
                 // Quarantine it (db/-wal/-shm → .corrupt-<ts>, same suffix
@@ -201,23 +215,12 @@ export class UnifiedShadow {
                 // (full mode re-imports organically on load). No auto-delete —
                 // the corrupt original is kept for forensics.
                 if (!(err instanceof StorageIntegrityError)) throw err;
-                const stamp = Date.now();
-                const quarantined: string[] = [];
-                for (const suffix of ["", "-wal", "-shm"]) {
-                    const from = `${dir}/index.db${suffix}`;
-                    const to = `${dir}/index.db${suffix}.corrupt-${stamp}`;
-                    try {
-                        renameSync(from, to);
-                        quarantined.push(path.basename(to));
-                    } catch {
-                        /* absent sidecar files are fine */
-                    }
-                }
+                const quarantined = quarantineCorruptIndexSet(dir);
                 loggerLog(
                     "error",
                     `[storage] index.db failed its integrity quick_check (${err.detail}) — quarantined ${quarantined.join(", ")} and reopened fresh; JSON envelopes remain the recovery source (#4175)`,
                 );
-                store = UnifiedStore.open(`${dir}/index.db`, undefined, { walJournalSizeLimitBytes: walLimit });
+                store = UnifiedStore.open(`${dir}/index.db`, undefined, openOpts);
             }
             // #3285: watermark trust does not cross process boundaries — a
             // restart-handoff overlap leaves the previous writer's rows AND
@@ -244,6 +247,14 @@ export class UnifiedShadow {
             return null;
         }
         const shadow = new UnifiedShadow(mode, store, legacyLoad ?? (() => null));
+        integrityFailureSink.handler = (detail) => {
+            shadow.stats.lastError = `integrity quick_check failed (deferred, large-db lane): ${detail}`;
+            shadow.stats.disabled = true;
+            loggerLog(
+                "error",
+                `[storage] unified sidecar DISABLED for this process lifetime: ${shadow.stats.lastError} — loads fall back to JSON envelopes (#4175 recovery source); the corrupt set is quarantined at the next open`,
+            );
+        };
         recordSidecarInstance(shadow);
         const engineTag = ` [engine: ${store.engine}/${store.engineSource}]`;
         const walTag = ` [wal-cap: ${walLimit}]`;