Repository navigation
[PR] fix: bound ordinary session loads and narrow lazy field reads - #4750
Conversation
ranxianglei
changed the title
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。检索了 open/closed issue 与全部在途 PR(100 条)——没有第二个 PR 触碰「普通加载全量物化消息行 / lazy 字段窄读」这条路径。相关但不重叠:#4532/#4540(验证侧的流式 digest/分批验证,不同机制)、#4522(ingestSession 只取 block 覆盖行,写路径)、#4619(quick_check 移出关键路径,启动路径)。本 PR 是 #4739 的修复 PR,一单一 PR。 标签: bug + scope/storage + p1(存储层、跨车道;大活跃历史下内存耗尽属 p1 口径)。 开始按 PR review 规范审:①方向核对 #4739 → ②回归清点(改动前既有行为逐项过)→ ③代码审查 → ④测试有效性 → ⑤diff 整洁度。PR head |
(ai: vllm-qwen/qwen3.8-27b) Review 结论:方向正确(修根因,非遮症状)、代码质量过硬标准、回归审计逐项通过;5 个红片共 14 个失败测试全部为 master 级继承性簇失败(A/B 字节级同签名实证),已按 CI 协调政策把 origin/ci-fixes(tip 1fa9373)合入分支并推送( 结构化八问
测试有效性(④):三个新测试文件有牙——字节级 parity 断言、spy 钉死(lazy 读取不触发任何整载/import/save/viewFor)、MySQL fake driver 断言精确快照 SQL 串 + 连接路由 + rollback/release 计数、仲裁真值表(6 行覆盖 drift-latch tie-break)并与公开 loadSync 选代交叉核对、页面失败恢复、空 vs 缺语义。storage-backend-contract 新增的真 MySQL readSnapshot 段已在 CI mysql-contract lane(mysql:8.0 service,job 绿)实际执行验证。 CI 处置(继承性实证):5 个红片的 14 个失败测试名(storage-render golden ×4、engine-proofs ×6、plugin-fork ×1、orphan-freeze ×1、opencode-inline ×1、blogfeed default probing ×1)与 #4599 在册簇逐一吻合。实证:4 个簇测试文件在干净 master(bb7c249)与 PR head 上跑出完全相同的失败(同一断言文本、同一 digest 漂移,66/77 同 11 败)——本 PR 零新增失败。merge ci-fixes 后本地定向:簇文件 77/77 过、PR 自测 28 pass/2 skip(better-sqlite3 驱动缺失)、web-blogfeed 8/8、launcher-opencode-inline 2/2。opencode-inline 为 Windows 在册 flaky 簇(仅单片出现),ci-fixes 尚无对应修复;若重跑再红属继承,已在 #4599 留言归口。 diff 整洁度(⑤/⑤b):12 文件每行均与目标相关;命名达意(LegacyViewBuilder/readResidentFieldsFor/buildRecordForSave);restoreBlockContents 抽公共消除两处重复;注释带 # 出处讲为什么;风格与仓库一致;无死代码/魔法数字。未发现需要补修 commit 的问题。 判定:代码与行为层面可合;等 CI 重跑绿(opencode-inline 若再红见 #4599)后请 owner 合并。 |
[bot] 🏷 进度更新(进程曾被看门狗重启,现汇报状态): Review 已完成(八问结论见上一条回复):方向正确、代码干净、回归审计通过;原 5 个红片 14 个失败全部为继承性簇(#4599 在册,A/B 字节级同签名实证),已把 origin/ci-fixes(1fa9373)合入分支推送(ba50d87),本地定向全绿。 新动态:ba50d87da 之上作者又推了两个 test-only commit(均针对我标记的 Windows 残留红):
两个 commit 本地实测 51/51 通过(Linux 路径)。 当前 CI:新 head |
[bot] 🏷 Review 收工总结(此前已发八问结论与进度更新,此为最终状态确认): 已完成:
当前状态:CI 正在新 head |
Preserve background integrity checks and their cache while retaining consistent sync/async read snapshots. Keep both MySQL snapshot and fork-clone contract tests, and combine Windows COMSPEC detection with exact launcher argument checks. Validation: 61 targeted tests passed, 3 optional backend cases skipped; TypeScript, build, and diff checks passed.
Windows SIGTERM terminates the child instead of running its graceful shutdown handler. Trigger the existing loopback shutdown route on Windows so the draining-marker regression exercises the same lifecycle as POSIX. Wait for child close before deleting its database, destroy sockets owned by the silent TCP fixture, and stop health polling when the child exits. Keep both regression scenarios and their behavioral assertions intact. Validation: the isolated CI-preloaded serial test file passes 6/6 and exits naturally in 11.58 seconds; npm run typecheck and npm run build pass.
Windows CI can reset an aborted probe connection instead of ending it. The silent TCP fixture did not handle errors on accepted sockets, so ECONNRESET became an uncaught test exception after the probe finished. Destroy reset sockets and add a real resetAndDestroy regression against the same fixture. Keep the existing port-owner and ladder assertions. Validation: the new regression reproduces ECONNRESET before the fix; the isolated CI-preloaded serial test file then passes 7/7 and exits naturally. Pure tsc and git diff --check pass.
[bot] (ai: vllm-qwen/qwen3.8-27b) 冲突已解决并推送。 把 master 合入 PR 分支(merge commit,不 rebase,保留外部贡献者历史),PR 恢复可合并,新 head 结构化八问:
交付: |
[bot] (ai: vllm-qwen/qwen3.8-27b) 状态确认(进程曾被看门狗重启,现复核完毕):冲突修复已完成并推送,无阻塞。
|
(ai: vllm-qwen/qwen3.8-27b) 已把最新 origin/master（ |
(ai: vllm-qwen/qwen3.8-27b) CI 失败诊断(自动修复车道 attempt 2/2)——结论:继承性 flake,非本 PR 引入,本分支无可修点,已归口 #4599;需 admin 重跑或等哨兵 ci-fixes→master。 结构化八问
下一步(均在本 agent 权限外,不再自行长查继承红):①admin 重跑 run |
ranxianglei#4538: sampled read-back verification in bounded batches of 32 (headers separately, blocks in persisted order, message hydration only at the verifier's pointer seqs); sync and async share one code path; SQLite and MySQL get bounded LIMIT/OFFSET + seq-IN reads. Conflict resolutions (p4540 base is pre-ranxianglei#5193/ranxianglei#4765/ranxianglei#4750; our side is the superset in every region): - src/storage/backend.ts: keep ours — p4540 added no content beyond ours (its sessionViewRows/readSnapshot/StorageReadView surface already present from ranxianglei#4765/ranxianglei#4750 merges); verified diff shows zero theirs-only lines. - src/storage/store.ts: union — p4540's getBlocks 'ORDER BY ord, block_id' total ordering (stable pagination across generations) onto our gc() batch-sweep constants, viewPointerSeqs, readSnapshot/readSnapshotAsync, activeForeignWriter, formatDbSize; comment kept ours' pluginSnapshotSeqs wording (documents the live field shape). - src/storage/mysql.ts: union — p4540's dedicated-pool reads with hydrateMediaAsync(conn) parameterization onto our v9 stamp + idx_content_refcount probe block + collectZeroRefIn lifecycle scoping; theirs-only lines audited line-by-line: all stale base versions of code our branch already carries (ranxianglei#5193 master state). - tests/plugin-fork.test.ts: keep ours' longer ranxianglei#4599 comment, replicasAvailable() gate (supersedes theirs' storageMode==="full"), and ours' ranxianglei#5140 raw-tail cold-restart test semantics (200, not 409); p4540's storageMode loop + env save/restore already on our side via ranxianglei#4765. - tests/storage-orphan-recovery.test.ts: take p4540's extra assert (alignedRef === "m00001") — pure addition, variable already defined.
…anxianglei#4765/ranxianglei#4750 The two PRs added different getBlockContents call shapes (paged rows for bounded view decode vs full narrow contents map for lazy-field restore); merged as one union return type, which broke caller-side narrowing in the read-snapshot test. Declare the overload pair so paged callers get the row array and no-arg callers get the map. storage-bounded-load's read counter monkey-patch now casts to the implementation signature it already matches at runtime (every counted call passes opts.limit=32).
Large ordinary cold loads hydrated every historical message and retained all raw block rows alongside the decoded view. Restoring dropped metadata or block contents also rebuilt an entire Session, twice when a save touched both lazy fields.
This change reads blocks in batches of 32 and fetches only the unique tail/plugin-snapshot message seqs. A shared incremental decoder preserves inactive blocks, archived/orphan originals, pointer order, repeated-message identity and tail-gap reporting. SQLite deferred read snapshots and dedicated async SQLite/MySQL connections keep all pages, CAS content and media hydration on one generation.
Lazy field getters now read just their requested indexed fields plus the existing envelope arbitration. A synchronous save shares that projection when both fields are dropped. Existing freshness/drift rules, missing-store failures and one/full normalization remain intact. The rollback-guard test helper now closes SQLite before deleting its temporary directory on Windows.
Validation:
The envelope still needs full parsing for source arbitration, and final original contents remain resident. This batch changes no schema, configuration or runtime installation and does not include request-lifetime, admission-control or streaming-write changes.
Fixes #4739