Skip to content

[PR] fix(storage): pre-hydration receipt no longer blocks the notice-store legacy→kv migration (#5761) - #5768

Open
ranxianglei wants to merge 2 commits into
masterfrom
2026-11-06_notice-store-hydration-race
Open

ranxianglei wants to merge 2 commits into
masterfrom
2026-11-06_notice-store-hydration-race

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes #5761

What

hydrateNoticeReceipts() (src/notice-store.ts, #5548) used cache !== null as its "already hydrated" gate — but addDismissedKeyInMemory lazily creates that same cache on first dismiss. So:

  • a dismiss/receipt landing between process start and the cli-boot hydration (void hydrateNoticeReceipts(), src/cli.ts:965) made hydration bail for the whole process: the one-time legacy JSON→kv migration was skipped (the file re-imports on every boot) and sibling lanes' kv rows stayed invisible in-process until restart;
  • variant: a dismissal landing inside the hydrate's await gap was clobbered by the final cache = merged.

Twin of the pattern already fixed in host-plugin-notice.ts on the #5757 line (commit 4d74613) — notice-store was missing the same fix.

How

Ports the #5757 pattern onto master's API surface (unifiedKvActive() + getStore() — the kvStore() probe introduced on that line is not merged yet, so this PR deliberately does not depend on it; #5757's branch touches neither of these files, so merge order is conflict-free either way):

  • hydrate UNIONs into whatever lazy cache exists instead of bailing; a new hydrated latch (distinct from lazy-init) gates re-entry;
  • the live-cache snapshot moves to just before the publish, with no awaits in between — a mid-hydration dismissal survives the final assignment (single-threaded guarantee);
  • publish only on success (see below).

Behavior changes (intentional)

  1. Hydrate-failure semantics: previously a failing listKvAsync still published the empty/partial map into cache, which killed the legacy-file fallback for the process lifetime and prevented any retry. Now the failure leaves cache null → file fallback stays live, next call retries. Advisory surface only; strictly more correct.
  2. Pre-hydration dismissal: previously = permanent migration skip + sibling rows hidden; now = union + migration runs. That is the fix itself.

No config-surface change, no wire-format/threshold/timing drift, no cache-invalidation impact, no new env/config keys.

Tests

Two regression cases in tests/notice-unified-store.test.ts mirroring the #5757 case shapes:

  • "a pre-hydration dismissal no longer blocks the migration" — fails on pre-fix code, passes with the fix;
  • "a dismissal landing during hydration survives the final publish" — gates BOTH kv list and set so the dismissal's own durable write cannot land before the list resolves and mask the loss via re-derivation from kv; fails on pre-fix code, passes with the fix.

Local: targeted suite 7/7 green; adjacent notice suites (issue5079-pck-default-off, issue5548-notice-dismiss-endpoint, local-announcements, notice-registry, wm-fastpath-intro-notice) 32/32 green; typecheck + build clean. Full-suite verdict via CI.

… legacy→kv migration (#5761)

A dismissal landing before the cli-boot hydration lazily created the receipts
cache, which made hydrateNoticeReceipts bail on `cache !== null` — skipping
the one-time legacy JSON→kv migration for that process's lifetime (re-import
churn on every boot) and hiding sibling lanes' kv rows until restart; a
dismissal landing inside the hydrate's await gap was clobbered by the final
cache assignment.

Twin of the host-plugin-notice fix that landed in the #5757 line (commit
4d74613), ported onto master's API surface (unifiedKvActive/getStore; the
kvStore() probe from that line is not merged yet):

- hydrate unions into the existing lazy cache instead of bailing; a new
  hydrated latch (distinct from lazy-init) gates re-entry
- the live-cache snapshot moves to just before the publish with no awaits in
  between, so mid-hydration dismissals survive the final assignment
- publish only on success: a failed listKvAsync now leaves the cache null
  (legacy-file fallback stays live, next call retries) instead of clobbering
  it with an empty/partial map for the process lifetime

Tests: two regression cases mirroring the #5757 shapes — both fail on the
pre-fix code and pass with it; adjacent notice suites green.
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-06_notice-store-hydration-race (0b2c949)

Latest build — install this exact version

npm install -g billion-context@0.0.9264-pr.5768.28ddb8f

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
0b2c949 billion-context@0.0.9264-pr.5768.28ddb8f build
cf85588 billion-context@0.0.8293-pr.5768.f9db48c build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5768.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix(storage): pre-hydration receipt no longer blocks the notice-store legacy→kv migration (#5761) [PR] fix(storage): pre-hydration receipt no longer blocks the notice-store legacy→kv migration (#5761) Nov 5, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

接单确认 — 这是 [PR] 类任务(GitHub PR #5768 → issue #5761),按流程做独立 review,不重做。

查重: 不重复。 核对了全部 open [PR] 行,没有第二个 PR 针对 #5761(notice-store hydration race)。关联的 #5757(host-plugin-notice.ts 同模式修复,commit 4d74613)已合入 master,与本 PR 是同一模式在不同文件上的两个实例——#5761 本身作为独立 issue 立项合理,本 PR 与其无重叠。

标签(建议): bug scope/storage p3(与 #5761 原标签一致:advisory 回执面缺陷,窗口内一次性迁移被跳过 + sibling kv 行进程内不可见,重启可恢复,非 p1/p2 级)。注:本镜像环境的 label API 端点未实现(shim 返回 not implemented),无法实际打标,在此报备。

开始独立审核:读原 issue → 逐行审 diff → 本地定向测试 → CI 状态。结论稍后按八问体例补在本线程。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

PR #5768 审核完成(单 commit cf85588,基于当前 master bfabcd1,0 commits 落后;独立拉分支核对,未照抄 PR 描述)。

① 是什么问题
统一存储实例上,一条在启动 hydration 之前落下的 dismissal 回执,会让「legacy JSON → 统一存储」的一次性迁移被整个进程跳过,其他车道已写入的回执也要重启后才可见。

② 什么原因导致的
hydrateNoticeReceipts() 用 cache !== null 当「已 hydrated」的门,但 addDismissedKeyInMemory 会在第一次 dismissal 时惰性创建同一个 cache——「本进程有人 dismiss 过」和「hydration 已完成」两种不同的状态被塞进一个变量,早期回执把门永久卡死。伴生症状:最终 cache = merged 赋值会覆盖 await 间隙里落下的 dismissal。修的是判定机制本身(门选错了状态),不是症状。

③ 影响范围与严重程度
仅 advisory 面(web 面板 banner/升级通知的关闭状态);触发条件 = 统一存储实例(shadow/full)且 dismissal 落在进程启动到 cli-boot hydration 之间的窗口(窄窗口,如刚启动就调管理端点)。最坏后果 = 丢一条回执 → banner 多显示一次、迁移推迟到下次启动。非数据丢失类、非缓存泄露类,issue 定 p3 合理。

④ 怎么修复的
引入独立 hydrated latch(hydration 完成 ≠ cache 非空)控制重入;hydrate 从「cache 非空即 bail」改为 UNION 进已存在的 lazy cache;live-cache 快照移到最终 publish 紧前方、中间无 awaits(单线程保证:此前落下的任何 dismissal 都活过最终赋值);仅成功才 publish。是 #5757 线既定模式的忠实移植,用 master 可用 API(unifiedKvActive() + getStore())——我核实了 #5753 commit 4d74613(含 kvStore() probe)确实尚未合入,且其分支 2026-11-05_json-state-to-unified 不触碰 src/notice-store.ts 与 tests/notice-unified-store.test.ts,合并顺序无冲突,PR 描述该说法成立。

⑤ 是否有回归问题
逐项清点既有行为:

  • legacy-only 实例(unified 未激活):gate 仍先 bail,纯文件路径,cache 永远 null——不变;
  • 正常 hydration:结果集(kv 行 ∪ 文件键)、迁移写入集(kv 中尚缺的文件键)、日志格式 imported N receipt(s)… 与修复前等价(旧代码逐键 .some() 扫 dbRows 的 O(n²) 成员检查改为 Set 查找,语义相同);
  • 幂等:hydrated latch 替代 cache !== null,仍 once-per-process(既有 "re-hydrate is a no-op" 测试钉住,通过);
  • 同步视图兜底(dismissedCache() 为 null → 读 legacy 文件):pck-notice/notices/announcements 三个消费方未动,hydration 前后语义均成立;
  • 驱逐上限(200,builtin 保护)、键校验(hash 形状/大小写锁)、fire-and-forget 持久写:全部未动。
    结论:无意回归零。有一处有意的失败路径语义变更,见下披露。

⑥ 是否破坏缓存
不涉及——模型侧 prefix cache 与压缩账本完全不受影响;涉及的只有本模块进程内回执缓存,其生命周期设计(进程内)未变。

⑦ 是否改了设计方向
否。移植既有 #5757/#5753 twin 模式到 master API 面;刻意不依赖未合入的 kvStore() probe(安全:即便 partial stub 触发异常也有 try/catch 兜底,warn-once 降级,绝不 crash)。

⑧ 是否引入了新的配置
无。无新 env/config 键,无阈值/时序/wire 变化。

两处有意的行为变更(旧→新 + 原因)

  1. hydration 失败:旧——listKvAsync 失败仍会把空/部分 map publish 进 cache → legacy 文件兜底全进程失效且永不重试;新——失败保持 cache null → 文件兜底存活、下次调用可重试。仅 advisory 面,严格更正确。
  2. pre-hydration dismissal:旧 = 永久跳过迁移 + 兄弟车道行隐藏;新 = union + 迁移照常执行。即修复本身。

验证证据(独立复现,不只信 PR 描述)

  • 回归测试有牙:临时检出 master 版 src/notice-store.ts 跑新套件 → 恰好两个新 case 失败(pre-hydration 挡迁移;mid-hydration 覆盖),既有 5 个通过;恢复 PR 版本 → 7/7 绿;
  • 定向套件:notice-unified-store 7/7;相邻 6 套件(issue5079-pck-default-off、issue5548-notice-dismiss-endpoint、local-announcements、notice-registry、wm-fastpath-intro-notice、remote-announcements)40/40;
  • typecheck clean,build clean @ cf85588(dist-import-annotations / dist-external gate OK);
  • 远程 CI(sha cf85588):30 checks,0 failed,0 pending——12 个 test shard(ubuntu/windows × node22/24 × 3)、static-gates×2、mysql-contract、kernel-guard、pi-subagents-guard、docs-sync、test-import-graph 全绿,gated e2e 车道按设计 skip;mergeable=true(无冲突),mergeable_state=blocked 为分支保护等待人工合并的正常状态。

一个观察(给 #5753 线,不阻本 PR):host-plugin-notice.ts 的 twin 实现(4d74613)把 live-cache 快照取在迁移循环的 awaits 之前,迁移窗口内落下的 dismissal 在那个文件里仍会被覆盖;本 PR 的 notice-store 版本把快照移到 publish 紧前方,严格更紧——#5753 合入时可顺手把同样顺序调整应用到 twin 侧。

判定:八问全过,diff 干净(2 文件,每行都与本修复相关),回归测试有牙且独立复现成立,CI 全绿——建议合并(合并权在你:https://github.com/ranxianglei/billion-context/pull/5768)。本次审核未推送任何 commit(未发现需要补修的缺陷)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: qwen/qwen3.8-27b)

APPROVE(建议合并)。 注:本账号即 PR 作者,GitHub 禁止对本人 PR 提交正式 approve 事件(API 返回「Can not approve your own pull request」),故审核结论以本 PR 评论记录,合并权在 owner。storage-scope advisory fix;正确、最小、真回归测试——独立对照 master bfabcd1ca 审(非照抄 PR 描述):两条新用例逐条推演旧代码必挂、新代码必过,所审 head 上 30 条 CI 全绿。

已顺手修: 迁移注释与真实语义不符——原注释称「raw junk stays file-only」,但 toImport 对 fileKeysOf 返回的每个未入 kv 的键都无条件 setKvAsync(该路径无格式校验,fileKeysOf 只过滤 typeof x === "string"),junk 实际也会被迁入 kv。已改为准确描述(best-effort 无格式过滤 + 视图直接并入 file keys,无论 kv 写是否落盘)。纯注释、单文件、commit 0b2c949bf;CI 将在新 head 重跑(仅注释改动,预期绿)。此修复不改变判定。

八问:

  1. 是什么问题:统一存储实例上,一条在启动 hydration 之前落下的 dismissal 回执会让「legacy JSON→kv」的一次性迁移被整个进程跳过(文件每轮重复导入),其它车道已写入的 kv 回执也要重启后才在本进程可见。

  2. 什么原因导致的:hydrateNoticeReceipts() 用 cache !== null 当「已 hydrated」的门,而 addDismissedKeyInMemory 会在首次 dismiss 时惰性创建同一个 cache——「本进程有人 dismiss 过」被误判成「hydration 已完成」,把门永久卡死。伴生:末尾 cache = merged 覆盖 await 间隙里落下的 dismissal;失败路径把空/部分 map 发布进 cache(杀掉文件兜底 + 挡住重试)。修的是判据本身(门选错了状态变量),不是症状。

  3. 影响范围与严重程度:仅 advisory 面(web 面板 banner / 升级通知的关闭态),非模型流量、非数据损坏。触发条件 = 统一存储实例且回执落在「启动→cli-boot hydration」窄窗口。最坏后果 = 丢一条回执 → banner 多显示一次 + 迁移推迟到下次启动。体验瑕疵级(p3)。

  4. 怎么修复的:引入独立 hydrated latch(完成 ≠ cache 非空)控制重入;hydrate 从「cache 非空即 bail」改为 UNION 进已有 lazy cache;live-cache 快照移到 publish 紧前方且中间零 await(单线程保证间隙内 dismissal 存活);仅成功才 publish。忠实移植 [PR] storage: move six JSON sidecar states onto the unified sqlite store (#5753) #5757 host-plugin-notice 同模式(两模块缓存形状不同,故移植模式而非照抄),且刻意不依赖尚未合入的 kvStore() probe。

  5. 是否有回归问题:逐项清点既有行为均保持——legacy-only 实例仍先 bail 走纯文件路径;正常 hydration 合并集(lazy ∪ kv ∪ file keys)与迁移写入集等价于旧逻辑(O(n²) .some() 成员检查改 Set 查找,语义相同);幂等 once-per-process 由既有测试钉住;三处消费方(pck-notice/notices/announcements)同步兜底语义前后均成立;驱逐上限 200 / builtin 保护 / 键校验 / fire-and-forget 持久写全部未动。有意变更(PR 已披露):① hydrate 失败从「发布空 map、永久杀文件兜底」变为「保持 cache=null、文件兜底存活可重试」(fail-safe 优于 fail-poison,方向正确);② pre-hydration dismissal 从「永久跳过迁移 + 隐藏兄弟行」变为「union + 照常迁移」(即修复本身)。

  6. 是否破坏缓存:不涉及模型侧 prefix/压缩缓存。仅本模块进程内回执 Map 的构建时机与失败回退变化,其进程内生命周期设计未改。

  7. 是否改了设计方向:无架构转向;仍是「一次 hydrate + 懒 cache + 文件兜底」三层结构,只修正 latch 判据与失败回退语义。

  8. 是否引入新配置:无新 env/config/默认值,无 wire/阈值/时序变化。

测试有效性:两条新增用例是真回归钉——用例1(启动前 dismissal 不再挡迁移)与用例2(hydrate 间隙内 dismissal 存活)逐条推演旧代码必挂、新代码必过;用例2 对 kv list/set 双向设闸,防止旧代码从 kv 反推掩盖丢失,设计到位。可选增强(非阻塞):可再加一条「listKvAsync 抛错 → cache 仍 null → 文件兜底存活」钉住披露的失败语义变更;属增强钉而非 bug 捕获器,不拦。

判定:storage-scope 公告面修复,正确、最小、测试有牙且独立复现成立,两处行为变更均已披露且方向正确,无 kernel/config/wire/安全影响,diff 干净(2 文件,每行都相关)。建议合并(合并权在你:https://github.com/ranxianglei/billion-context/pull/5768)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: qwen/qwen3.8-27b)

审核收工。本 pass 在前一 pass(floor 6)结论之上补了两件事:

  • 顺手修了一处迁移注释:src/notice-store.ts 原注释「raw junk stays file-only」与真实语义不符——toImport 对 fileKeysOf 返回的每个未入 kv 键都无条件 setKvAsync,该路径无格式校验(fileKeysOf 只过滤 typeof x === "string"),junk 实际也会被迁入 kv。已改为准确描述。纯注释、单文件、commit 0b2c949bf。→ 据此,floor 6「本次审核未推送任何 commit」的说法以本 push 为准更新。
  • 审核结论已记录到 GitHub:本账号即 PR 作者,GitHub 禁止对本账号 PR 提交正式 approve 事件(API 返回「Can not approve your own pull request」),故以 PR 评论形式给出完整八问 + APPROVE 判定 → [PR] fix(storage): pre-hydration receipt no longer blocks the notice-store legacy→kv migration (#5761) #5768 (comment)

CI:新 head 0b2c949bf 重跑中,当前 0 失败 / 2 进行中(仅注释改动,预期全绿)。

判定:建议合并——storage-scope 公告面修复,正确、最小、真回归测试;两处行为变更均已披露且方向正确,无 kernel/config/wire/安全影响。合并权在 owner(agent 无权合并)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

该PR已按根因归类纳入 [kernel-compression] 根因汇总 #6265,由汇总单统一跟踪处理;本单不再单独跟进,故关闭。如认为归类有误或问题仍未解决,请直接回复,会重新打开。

@ranxianglei

Copy link
Copy Markdown
Owner Author

状态更正(自动化脚本误操作): 本单此前被「纳入根因聚合」流程误关闭, 现重新打开。机器人工作已停, 本单由根因聚合线程统一跟踪, 根因修复落地后请手动关闭或在本单回复「close」。
[automated: re-open after bot-stop-only policy]

@ranxianglei ranxianglei reopened this Nov 12, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

更正说明:本单此前被归类聚合流程误关闭。按「只停机器人、不动 issue 本身」的口径重新打开;本单已被根因聚合线程纳入跟踪,机器人在本单上的工作已停止,不会再有自动处理。归类或纳入有误请直接回复指出。

@ranxianglei

Copy link
Copy Markdown
Owner Author

重新打开:此前被归类聚合脚本误关。按 owner 口径,纳入根因聚合的 issue 只停机器人、不关闭,由根因聚合线程统一跟踪,根因修复落地后统一关闭。
(reopened: absorbed issues must not be closed, only bot work stopped — owner policy 2026-11-12)

@ranxianglei

Copy link
Copy Markdown
Owner Author

(更正)此前按"清理"口径误关闭;按 owner 口径纠正:被根因聚合纳入的 issue 只停机器人、不关闭。本 issue 重新打开,机器人工作已停。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: qwen/qwen3.8-27b)

收到。本单已重新打开、机器人在本单上的工作已停止——我不再重复审核、不再推送 commit、不关闭,根因修复落地后由聚合线程统一收口。

审核交付物此前已全部完成,无需重做:

另按 owner 邀请("如认为归类有误请直接回复")提一点,请 owner 定夺:本 PR 是 storage 范围(src/notice-store.ts + tests/notice-unified-store.test.ts,advisory 通知回执 legacy JSON→kv 迁移的 hydration 竞态),完全不涉及 kernel/ 压缩代码,其关联 issue #5761 也是 storage 缺陷。因此归入「[kernel-compression] 根因汇总 #6265」看起来是聚合脚本的误分类——本 PR 的修复已独立完成并通过审核,并不依赖任何 kernel-compression 根因修复,按"根因修复落地后统一关闭"的逻辑它也不会被正确收口。建议改归 storage 类聚合、或按已审核状态单独推进。

机器人在本单上到此为止。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: qwen/qwen3.8-27b)

收到,状态更正已知悉。归类/纳入的疑问我已在上一条回复中指出(本 PR 为 storage 范围、不涉及 kernel/,疑为误分类),此处不重复。机器人在本单上的工作维持停止,等 owner 手动关闭或在本单回复「close」。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: qwen/qwen3.8-27b)

本单(PR #5768,storage 范围 advisory 通知回执 hydration 竞态修复)审核已完成,汇总如下:

已交付

当前状态

等 owner 手动关闭或在本单回复「close」。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

storage: notice-store (#5548) pre-hydration receipts block the one-time legacy→kv migration

1 participant