Skip to content

[PR] feat(obs): M1 Native Identity Observation Layer — Convert the #5571 collection table from static forensics to live measu | feat(obs): M1 原生身份观测层 — 把 #5571 采集表从静态取证变成实测数据(默认关) - #5573

Open
ranxianglei wants to merge 15 commits into
masterfrom
m1-native-identity

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

M1 原生身份观测层(#5571)

这是什么

每条经过 bili 的模型请求都自带客户端的原生身份(WB/codex/CC/opencode/Gemini 各自的 session id、消息级宿主 id、压缩信号——全是 wire 上的普通头/正文字段)。这个 PR 在 relay 单点咽喉加一个纯采集器,把这些信号按客户端家族识别出来、以 sha1-8 摘要形式记进 session ext(nidv:obs),让后续 M2 绑定表 / M3 re-baseline 建立在实测数据上,而不是二进制字符串推测上。设计稿与红线见 issue #5571 评论(thread: m1-design)。

改了什么(1 commit,+~1000 行含测试/文档)

  • src/server/native-identity-profiles.ts(新):五家客户端模式表 + 纯函数抽取。压缩文案做成一张带测试钉住的常量表——将来文案漂移改表即可,禁止别处硬编码。
  • src/server/native-identity-obs.ts(新):observeNativeIdentity() + 状态机(纯函数 mergeNidvState,可单测)+ ext 读写。整体 try/catch fail-open,常数成本查找,正文扫描有界(USER_TEXT_SCAN_CAP=3)。
  • src/server/handle.ts:一行调用,插在 sessionId 绑定后(enterSessionContext 之后)。设计稿原写 L396,body 解码/会话绑定实际更晚,按真实管线落位——这是与设计稿唯一的位置性修正。
  • src/rollout.ts + website/config-reference/rollout.yaml:开关 rollout.features.nativeIdentityObservation,默认关(percent 0),env BILI_ROLLOUT_NATIVE_IDENTITY_OBSERVATION=1|0。CONFIGURATION.md/zh-CN 重生成产物在本 PR 内(避免 [PR] docs: regenerate CONFIGURATION tables (guardrails.repetition drift breaks docs-sync) #5374 式 docs-sync 长红)。
  • TECHNICAL-NOTES.md/.zh-CN.md:新增章节。

红线保持(设计稿 §6 逐条)

零转发字节改动(模块只读)· 零决策(下游不得 gate 本数据,直到 M2/M3)· 零 agent lane 改动 · 未知客户端不写任何值(隐私)· 关闭时零 ext 写入且管线字节一致(显式 off 用例钉住)· 原始 id 不落盘不打日志(只用摘要,两个测试断言原始值不出现在 blob/日志)。

验证

  • 新测试 tests/issue5571-native-identity-obs.test.ts 9/9:profile 命中顺序钉死、五家 fixture 抽取、前缀锚定开头(中段嵌入不得命中)、双通道互验+畸形吞错、形状断言、稳定性矩阵(含重启续接)、事件 FIFO/4KB 预算、off/inert 全路径。
  • 相邻套件 fastpath + lane-decisions-matrix 118/118 绿。
  • tsc --noEmit 0 错(注:本机过期 node_modules 会假报 ~40 个存量错误,npm install 后清零——与本 PR 无关)。
  • gen-config-docs i18n 校验通过(246 keys EN/ZH 等集)。

合并后动作(owner)

你这台 WSL 盒开 BILI_ROLLOUT_NATIVE_IDENTITY_OBSERVATION=1 开始采集(dsh/CC/opencode/codex 混流),约一周后对照出口判据回填 #5571 表 → 解锁 M2。

…oint, five client profiles, digest-only session ext, default off

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

自审(46fabdd)——结论:可合并,CI 全 lane 绿 + owner 在 WSL 盒开开关即可开始采集。逐点复核记录:

  1. 单采集点核实:observeNativeIdentity 全仓仅 handle.ts 一处调用(rg 验证),位于 sessionId 绑定后、任何改写前;passthrough/bypass 早退分支在其之前 return,天然不采(正确:#1117/#920 的匿名直转流量不该被指纹化)。
  2. fail-open 不是注释而是结构:函数体整体 try/catch、内部零 await、store 不可用(replicasAvailable=false)与空 sessionId 双早退;profiles 抽取层自己也带独立 try/catch——双层兜底,坏行最差后果 = 无观测。
  3. 隐私断言双向钉住:测试同时断言原始值 不在 ext blob 里 且 不在日志行里(digest-only);unknown 客户端走 profile.id === "unknown" 早退,duck-store 断言零写入。
  4. 前缀匹配器只锚文本开头(startsWith,非 includes):中段引用压缩文案的对抗样本有独立用例,不命中。
  5. 非阻塞观察(留 M2 处理):① codex 头族(x-codex-window-id 等)目前是"存在即采集、拼写待实测",这正是本层要关掉的待抓包项,不做预判过滤是有意为之;② msgN/msgLastH 只记最后值+计数,wb 消息级 id 每轮必变,M2 绑定表会把它变成真信号——M1 阶段这是正确的最小形态。
  6. 文档链完整:rollout.yaml 条目 → CONFIGURATION.md/zh-CN 重生成产物同 PR(gen-config-docs i18n 校验 246 keys 通过),不会复现 #5374 式 docs-sync 长红。

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: m1-native-identity (4b4d024)

Latest build — install this exact version

npm install -g billion-context@0.0.9046-pr.5573.ac9f267

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
4b4d024 billion-context@0.0.9046-pr.5573.ac9f267 build
37cbd31 billion-context@0.0.9036-pr.5573.62ff795 build
a3e4aa6 billion-context@0.0.9025-pr.5573.a806fc1 build
7e910aa billion-context@0.0.7964-pr.5573.8ee3f6b build
0e83b21 billion-context@0.0.7954-pr.5573.85848bb build
f6968ee billion-context@0.0.7950-pr.5573.73c0d04 build
e1a3ef3 billion-context@0.0.7945-pr.5573.efb8c3f build
870265d billion-context@0.0.7939-pr.5573.bf0cca4 build
46fabdd billion-context@0.0.7935-pr.5573.44b7aa2 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5573.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title feat(obs): M1 原生身份观测层 — 把 #5571 采集表从静态取证变成实测数据(默认关) [PR] feat(obs): M1 Native Identity Observation Layer — Convert the #5571 collection table from static forensics to live measu | feat(obs): M1 原生身份观测层 — 把 #5571 采集表从静态取证变成实测数据(默认关) Nov 3, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 检索 open+closed issue/PR(关键词:native identity observation / nidv / #5571 采集表 / M2 绑定表 / re-baseline):无其他相关条目;本 PR 即 #5571 设计稿的实现,无并行在途工作。

标签: pr(已有)+ scope/wiring(relay 单点采集,属请求管线接线)+ p3(默认关的纯观测功能,存量用户零行为变化)+ dsh / claude-code / opencode / codex / gemini / workbuddy(模式表覆盖的五家客户端家族,dsh 为主流量车道)。理由:功能面精确对应这五家的 wire 身份信号。

审核进行中:全 diff 已通读完毕,正在跑定向测试与 CI 状态核对;有发现会直接在 PR 分支修掉,结论按八问体例回复。

…raw ids (#5571 M1)

Review of the M1 PR: the dual-channel mismatch event carried up to 64 chars
of the raw client session id (cc metadata.user_id.session_id) into the
persisted ext blob, breaking the stated privacy redline that only sha1-8
digests ever reach storage or logs. Evidence is now hdr=<h8> body=<h8>, and
the stability-matrix test pins that neither raw channel value appears in the
serialized state.
…gnments (#5571 M1)

CI static gate check-unused-exports flagged seven exports of the new modules:
six module-local types (NidvProfile, NidvCompactionSignal, NidvEvent,
NidvState, NidvStoreView, ObserveNativeIdentityArgs) lose their leading
export per the gate's own remedy, and NidvHeaders was dead code entirely —
deleted, its structural-typing rationale folded into detectNativeProfile's
doc. Also removes two self-assigning no-op statements from the fail-open
catch in extractNativeIdentity.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

🔶 需要决策
(ai: vllm-qwen/qwen3.8-27b)

背景:存在两个并行的 M1 实现 PR——本线程的 #5573(按 02:04 设计稿原文实现:单采集点、纯头抽取)与 #5576(按你 02:10 的更正实现:双采集点 + 有界文本扫描 + 派生记录键 n: + BILI_NATIVE_IDENTITY_PROFILES_JSON 扩展 env)。两者注册同一个 rollout 开关(nativeIdentityObservation)、写同一个 session ext 键(nidv:obs)、新增同名不同内容的文件——互斥,只能保留一个。

待决问题:保留哪个 M1 PR?

选项(各带代价):

推荐:A。
不决定的默认行为:两个 PR 都保持打开不合并;M2(绑定表/re-baseline)继续等稳定的 M1 实现。

另需确认一件小事:WB profile 表的压缩头拼写是 x-compacted,而 #5571 三处(issue 正文、设计稿 §3 表、L2a 评论)都写作 X-Compaction(值 manual|reactive|auto)。请对照源码/实抓确认哪个对——若 X-Compaction 才对,存活的那个 PR 要改表,否则 WB 显式压缩事件(最强信号类)在生产上永远不会触发。


复审结论(结构化八问)

  1. 是什么问题:不是缺陷——是 [Design] Client-Native Unified Identity Layer for Session/MsgID (Identity Collection Table + Incremental Parsing Refacto | [设计] 客户端原生的 session/msgid 统一身份层(身份采集表 + 增量解析重构) #5571 设计的 M1 原生身份观测层落地:每条过 bili 的模型请求自带客户端原生身份(WB/codex/CC/opencode/Gemini 各自的 session id、压缩信号),本 PR 在 relay 单点咽喉识别客户端家族、以 sha1-8 摘要记入 session ext(nidv:obs),默认关,让 M2/M3 建立在实测数据上。
  2. 什么原因导致的:不适用(功能项)。审查中发现作者初稿两处问题,均已直接在分支修好:①dual_mismatch 事件把最多 64 字节的原始 CC session id 写进 ext blob,违反 PR 自己声明的红线「原始 id 不落盘不打日志」(测试也只钉住了另一条路径);②模块内类型被多余导出触发 unused-exports 门禁,catch 里还有两行自赋值死代码。
  3. 影响范围与严重程度:默认关(percent=0),存量用户零行为变化;开启后每请求成本 = 一行 debug 日志 + 一次 session-ext 读写(仅统一存储实例),出站字节不变,下游零决策依赖本数据。无外部影响(纯观测面)。
  4. 怎么修复的:两个独立 commit——a978523a7 把 dual_mismatch 改为存双通道摘要(hdr=<h8> body=<h8>)+ 补断言原始值不出现在 blob;870265dcf 去掉多余 export、删除 no-op 自赋值。
  5. 是否有回归问题:逐项审过——出站字节(模块只读,off 路径字节一致有测试钉住);消息身份链(msgid/reconcile/snapshot-render 未触);ext 命名空间(现有键为 inc:* / pf:state / threads:mirror:,新键 nidv:obs 无冲突且遵循同一命名惯例);配置面(沿用既有 BILI_ROLLOUT_ 惯例,docs 已在 PR 内重生成);kernel/ 与全部 agent 车道未触;fail-open(整体单 try/catch,replicas 不可用或 legacy 存储时不写任何东西且不影响主管线)。无回归。待确认一项:WB 头拼写问题(见上)。
  6. 是否破坏缓存:不涉及——不碰前缀缓存/增量/去重键;观测数据只存 session ext,不被任何缓存路径引用。
  7. 是否改了设计方向:不转向——按 [Design] Client-Native Unified Identity Layer for Session/MsgID (Identity Collection Table + Incremental Parsing Refacto | [设计] 客户端原生的 session/msgid 统一身份层(身份采集表 + 增量解析重构) #5571 设计稿 M1 落地;唯一位置性偏差(设计稿写 L396,实际落在 handle.ts sessionId 绑定后 ~L1362)PR 正文已说明。与 [PR] Native-Identity Fastpath: Observation Layer (M1) + Identity-Only Hit Gate (v5) | 原生身份快路径:观测层+身份制判据(一次做完) #5576(你的更正版)的形态差异正是本次决策内容。
  8. 是否引入了新的配置:是——rollout.features.nativeIdentityObservation(默认 percent=0=关,config 支持 true/false/number)+ env BILI_ROLLOUT_NATIVE_IDENTITY_OBSERVATION(1|0,env 优先);CONFIGURATION.md 中英已在 PR 内重生成。迁移方式:无(默认关)。

CI 状态:旧头的 unused-exports 红已由 870265d 修复;新头 870265d 有 6 个 test 分片红——拉取两个分片日志核实,失败均为已知环境敏感族「storage engine: full-mode restart is invisible × 4 wires」(#4599 有档),与本 PR diff 足迹零交集,本地定向套件 143/143 绿 ⇒ 判定继承性/环境性。已按协议 merge origin/ci-fixes(d03c578,验证全绿)进分支并推送,新头 e1a3ef3,CI 重跑中,转绿后补报。

判定:代码侧 ready(本地门禁全绿、两处发现已修、隐私红线恢复);合并次序等你拍板保留哪个 M1。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重补充:本线程此前「查重不重复」的结论产生于 GitHub PR #5576(镜像线程 #5578)出现之前,现已过时——#5576 是同一 M1 交付物的第二个实现:配置面、nidv:obs ext key、sha1-8 PII 红线全部相同,两个不能都合。关键差异在采集点架构:incremental-lane 的 forward 以合成 anthropic 请求重入 handleRequest(src/incremental-lane.ts:605/892),合成请求头只带 x-bili-plugin 族 + authorization/anthropic-version/anthropic-beta,因此本 PR 的单 relay 点采集对 dsh/omp/opencode/pi 流量结构性不可见,#4491 msgIds 信封证据也采不到;#5576 的双点设计(relay + lane 入口)正是针对这一点,且已验证无双重记录。owner 决策项已在 #5578 提出(推荐保留 #5576、吸收本 PR 的 TECHNICAL-NOTES 文档节后关闭本 PR)——拍板前两边都保持 open。

NidvProfile.extraHeaders was declared and populated in the codex row but no code path reads it: detection keys x-codex-installation-id directly, extraction has no storage slot for extra headers, and no test asserts them. Remove the field and the row entry; the pending live-capture spellings stay in issue #5571's collection table until M2 decides what to actually capture.

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ai: vllm-qwen/qwen3.8-27b)

APPROVE — M1 observation layer is correct, safe-by-construction, and well-pinned. Reviewed end-to-end (full diff, integration points, tests, docs); no kernel/ touch, no version/lockfile churn.

What I verified

  • Red lines hold: single call site in src/server/handle.ts right after enterSessionContext (before processTurn), read-only, whole body in one try/catch (fail-open), default OFF (rollout.features.nativeIdentityObservation, percent 0 ⇒ off for every install; env fold to BILI_ROLLOUT_NATIVE_IDENTITY_OBSERVATION checked against rolloutEnvName). Off-path cost is one cached config lookup + early return; off ⇒ zero ext writes + zero logs is pinned by test 8.
  • Privacy: persistence stores sha1-8 digests only (nidvH8), raw id values live in request memory for microseconds, log lines print digests/truncations, unknown clients write NOTHING — all asserted (tests 4/9).
  • Store integration: setSessionExt/getSessionExt are sync and the replicasAvailable() gate makes legacy-only instances inert (verified against src/persist.ts); new ext key nidv:obs does not collide with inc:* / pf:* / threads:* namespaces.
  • Tests are real, both-directional: prefix-match fires at text START only (mid-text quoted copy must NOT fire), compact-endpoint suffix match, cc dual-channel match/mismatch/object-dialect/malformed, profile-flip & restart continuity over JSON round-trip, FIFO ≤32 events + 4KB blob cap, digest-only mismatch evidence regex-pinned. Local targeted run: 9/9 pass; typecheck clean.
  • Docs: CONFIGURATION.md/.zh-CN + website/config-reference/rollout.yaml seed rows are consistent with the implementation (default off, env name, digest-only semantics).

Small finding fixed on the spot (commit f6968ee9f): NidvProfile.extraHeaders was declared and populated in the codex row but consumed nowhere — detection keys x-codex-installation-id directly, extraction has no storage slot for extra headers, and no test asserts them. Removed the field + row entry; the pending live-capture spellings stay in issue #5571's collection table until M2 decides what to actually capture.

Non-blocking observations (no code change requested)

  1. shapeAssertAbsent matches markers over the WHOLE body text (bodyText.includes) — user-authored content that legitimately contains ses_/msg_/prt_ (e.g. pasted opencode docs) can produce spurious shape_assert_fail events. Noise is bounded (consecutive dedupe + 32-event FIFO + 4KB cap); strictness is a defensible data-quality choice for M2. Revisit if the event turns out to be noisy in practice.
  2. The codex fallback rule (responses wire + generic x-client-request-id presence) may label other Responses-API clients as codex. Observation-only impact (family mislabel); the unknown⇒no-write redline still holds.
  3. WB compaction header spelling (x-compacted/x-context-compacted here vs X-Compaction as written in #5571) is still unconfirmed pending live capture — already escalated on-thread. If X-Compaction is right, the WB explicit-compaction class simply never fires until the row is corrected; per the module's own convention that fix belongs in this profile row (+ tests), so it does not block M1.

Note for the owner: parallel PR #5576 implements the same M1 surface with dual collection points (it also covers incremental-lane traffic this single choke point structurally misses). Which of the two survives is an open on-thread decision — this review judges #5573 on its own merits.

(posted as a comment review — GitHub does not allow the author account to file a formal APPROVE on its own PR; the verdict stands as stated.)

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

审查完成,GitHub 已发评论式 review(APPROVE 结论——本账号即 PR 作者,平台不允许正式自批,故以 COMMENT 形式发布,判定不变):#5573 (review)

已顺手修:删除死字段 NidvProfile.extraHeaders(声明并在 codex 行填充,但无任何代码消费——检测直接用 installation-id 字面量、提取无对应存储槽、测试也未断言;待 live capture 的拼写留在 #5571 采集表)→ commit f6968ee(已推 m1-native-identity)。

结构化八问:

  1. 是什么问题:M1 原生身份观测层——在 relay 单点采集各客户端家族的 native 身份面(session/message/request id、压缩信号),以 sha1-8 摘要记入 session ext nidv:obs,把 [Design] Client-Native Unified Identity Layer for Session/MsgID (Identity Collection Table + Incremental Parsing Refacto | [设计] 客户端原生的 session/msgid 统一身份层(身份采集表 + 增量解析重构) #5571 静态取证表变成实测数据(供 M2 绑定表 / M3 re-baseline)。默认关。
  2. 什么原因导致的:不涉及(功能/观测层,非缺陷修复);动机是 M2/M3 需要实测事实而非二进制字符串。
  3. 影响范围与严重程度:rollout percent 0 ⇒ 开启前对所有安装零行为变化;开启后仅统一存储实例上有附加 ext 写,legacy-only 实例 inert;零转发字节改动(模块只读、下游不据此决策)。用户面影响:无。严重度:不适用(纯观测)。
  4. 怎么修复的(审查结论):独立验证红线全部成立——单调用点在 enterSessionContext 之后、只读、外层 try/catch fail-open、仅落 sha1-8 摘要、unknown 客户端不写、off ⇒ 零存储流量+零日志(测试 8/9 钉死);集成点逐一核对(persist 同步 ext API + replicasAvailable 门、rolloutEnvName 折叠 ⇒ BILI_ROLLOUT_NATIVE_IDENTITY_OBSERVATION、nidv:obs 键与 inc:/pf:/threads: 命名空间无冲突);本地定向跑 9/9 + typecheck 干净(fix 后头 f6968ee)。
  5. 是否有回归问题:审了触碰路径的既有行为(handle.ts 管道插入点、rollout 注册表、persist ext 命名空间、CONFIGURATION en/zh/yaml 文档)逐项保持,未见无意回归。高危模式逐项过:无语义漂移(不 gate 任何东西)、无 fail-fast 变吞错(fail-open 为设计且仅观测)、无阈值/时序/输出格式漂移(默认关,日志行仅在开启时出现)、前缀缓存/重试/增量游标不受影响。
  6. 是否破坏缓存:不涉及——不与任何 prefix-cache/pinning/stash 路径交互;ext 写只进 unified sidecar,不触 kstate/wire。
  7. 是否改了设计方向:否——纯附加观测车道,走 canonical rollout 机制;M2/M3 决策明确推迟("nothing downstream may gate on this data")。
  8. 是否引入了新的配置:是——rollout.features.nativeIdentityObservation(默认 false/0%,env BILI_ROLLOUT_NATIVE_IDENTITY_OBSERVATION=1|0),CONFIGURATION.md/.zh-CN + website seed 同步文档化,沿用既有 rollout 模式,无迁移。

非阻塞发现(已记入 GitHub review,未动代码):① shapeAssertAbsent 对整段 body 文本做子串匹配——用户内容恰好含 ses_/msg_/prt_(如粘贴 opencode 文档)会误报 shape_assert_fail 事件,噪音有界(dedup+FIFO≤32+4KB),严格度留给 M2 数据质量取舍;② codex 回落规则(responses + 通用 x-client-request-id)可能把其他 Responses 客户端标成 codex(仅观测层面误标);③ WB 压缩头拼写(x-compacted/x-context-compacted vs #5571 所写 X-Compaction)待 live capture 确认——floor 5 已升级,按模块自身惯例修法属于 profile 行本身,不阻塞 M1。

状态:PR head 现为 f6968ee(CI 重跑中;旧头 e1a3ef3 经镜像缓存确认绿:12 success / 18 skipped-条件车道 / 0 failed)。#5573 vs #5576 的留一关一裁决仍在 owner 手里(floor 5/7 已有记录,本账号不关闭任何条目)。等 owner 批准合并。

…026-11-03

Owner directive (issue #5571 thread): drop byte-stability from the hit gate;
verify on sessionid + msgid only, count stays diagnostic (already so since v4).

- Head element loop: under a shape-valid host msgid list, id/byte inequality
  demotes to a diagnostic warn stream (feeds M3 re-seeding signals); pure-count
  clients keep the pre-v5 byte gate unchanged (their ids carry no proof).
- Session continuity gate: new state field cs (h8 digest of the client-native
  session header: wb x-conversation-id / cc x-claude-code-session-id /
  opencode x-session-affinity); an identity swap riding one bili session is a
  sess-change miss; recorded identity follows the traffic, swaps always cost a
  legacy pass. readSaved whitelist extended for cs.
- System text has no msgid (flagged deviation in the ruling): head-drift stays
  gating in both modes.
#5403/#5418 v6 canonical-ledger unification. The knob governs what the
watermark hit decision does with a client structural rewrite whose shape
survived the width + origin gates: ON (default) takes one conservative
legacy pass and rides the increment passthrough-as-new-message from the
next turn (fresh ids append to the cumulative host-id ledger, no
suppression logic — contract R4/O3); OFF restores pre-v6 ride-with-
diagnostic for those turns. Nested under pluginFullResendWatermark so it
is dead weight outside the watermark cohort; count-only turns stay
byte-identical either way.

Docs follow the sibling-knob convention: CONFIGURATION(.zh-CN) feature +
env tables, website/config-reference/rollout.yaml entry (en+zh), and a
v6 paragraph in the Full-resend watermark fastpath TECHNICAL-NOTES
section (en+zh).
The watermark identity channel already carried the per-turn tail-id list;
this commit adds the CUMULATIVE ledger it needs — the set-diff baseline
for the canonical-ledger hit decision (contract R1/R2, owner rulings
2026-11-03):

- PFSavedState.lg: ordered real host ids ever observed on this session's
  channel, FIFO-capped (PF_LEDGER_CAP 2048 entries / PF_LEDGER_MAX_BYTES
  48KB serialized, oldest dropped first). '?' slots never recorded.
  MODEL-INDEPENDENT by design (R6): projections partition on
  (model × wire format), the logical ledger does not.
- Sole growth site is finalizeWatermarkTurn: grow only when the turn
  delivered a shape-valid list AND the new watermarkCompactionPassthrough
  knob is on AND the conversation did not swap (ledgerHold, contract R7
  — a cs-gated turn's ids describe a DIFFERENT conversation and must not
  fold into this session's baseline). Non-grow turns carry the stored
  ledger forward unchanged. Bootstrap/migration falls back to l.
- Append-only: no server-authored text ever enters (R2); entries are the
  client's own screened id surface (parseTailIds all-or-nothing).
- Fixes noteCreditedMiss arm-path dropping l on breaker trip — the
  breaker firing would have wiped exactly the ledger state rewrite-heavy
  clients rely on (l is now preserved alongside lg).
- FastpathOutcome gains optional ledgerHold (additive, consumed by
  finalize; produced in the next commit).
…cs (R1/R4/R5)

The watermark HIT DECISION now derives from the canonical ledger instead
of the fastpath's own parallel byte gates. Consuming snapshot-render's
full walk on the hit path would pay the O(N) parse this lane exists to
avoid, so the shim tail-ids SET-DIFF against pf:state.lg fronts the same
semantic contract as the primary cheap gate — host msgids are unique
forever, so a pure-append list is equivalent to the backward id-chain
anchored on the transcript succeeding (snapshot-render's id fast-path).

- New layer after the cs gate (R7 placement), id mode only, knob-gated:
  a REAL id arriving mid-list before a still-known id proves in-range
  structural mutation (client compaction/insertion); verdict = ONE
  conservative CREDITED legacy pass ('rewrite-passthrough'), then the
  increment rides passthrough-as-new-message from the next turn — fresh
  ids append naturally via finalize set-diff, NO suppression logic, never
  speculative rebuilds (R4/O3). Knob OFF skips the layer entirely: pre-v6
  ride-with-diagnostic semantics.
- Ids OUTSIDE [K-1..T-1] legitimately never reappear in traffic, so
  'vanishing from the list' is not a signal — only mid-list unknowns are.
  Index 0 (window origin) stays with the origin pin, which explicitly
  neutralizes an unknown no-claim origin (#5418 v3/#5571 v5).
- R5: body grew (T > K) but the increment carries no host id at all ⇒
  hit proceeds on the stashed snapshot (every verifiable gate agreed) +
  structured diagnostic, ledger append stays a natural no-op; model-
  switch/fork id-space questions deferred to research holds O1/O2.
- tail-ids-prev miss now carries the shape-valid list so its observed ids
  still enter the ledger (R2 completeness; count-only turns unaffected).
- cs-gate miss sets ledgerHold=true (finalize then holds lg — a swapped-in
  conversation's ids never fold into this session's baseline, R7).
- Header doctrine + PFSeed.model comment updated: m demoted from
  world-partition to projection-cache key over ONE logical ledger (R6);
  kept gates documented where alignment cannot cover them (width =
  shim-protocol shape/origin K; origin pin = gap-tolerant alignment would
  ride a reminted origin; head system-text = prompt has no msgid).
The cheap set-diff gate is the id fast-path of the canonical alignment —
this adds the confirmation layer where it costs nothing: on a watermark
MISS turn the full O(N) parse is already paid, so prepare-openai runs a
light backward id-chain over store.unifiedMessageIdRows (identity rows
ONLY — no content bytes loaded) and logs whether the client's live tail
anchors into the unified transcript. One log per session at most
(metadata latch); dormant unless the snapshotRender rollout cohort is
active with ≥1 active block — outside that cohort it is a blocks check.

probeTranscriptAlignment is deliberately standalone rather than extracted
from buildView: this module is render authority (#3812/#4241), and an
observability probe must not couple doctrine-critical walk code to a
diagnostics path. Strict-contiguous walk + MIN_SEAM_RUN floor mirror the
alignment's gap discipline; it reports reasons (no-active-blocks /
store-unavailable / transcript-error / transcript-empty / chain-not-started
/ seam-too-short) for M3 ledger-vs-transcript cross-checks.
…gh (R4/O3)

The structural-rewrite layer fires only on "unknown real id BEFORE a still-known
slot". Spell out WHY that is the sole id-provable mutation shape:
- lists span elements [K-1..T-1], so old ledger ids legitimately never
  reappear; "vanishing from traffic" is not a signal;
- an unknown at index 0 (the ORIGIN slot) can never reach this layer — the
  origin pin above has already verified or missed it (#5418 v3 / #5571 v5);
- a pure trailing run of fresh ids — incl. a full-history compaction summary
  minting fresh ids — is indistinguishable from ordinary appends, so it rides
  by DEFAULT per R4/O3 (passthrough-as-new-message; fresh ids enter the ledger
  via finalize; NO suppression logic).

Docs-only: the loop already implements cursor > i. Closes the review question
"which rewrites does v6 catch, which ride?" in the code where the answer lives.
Ref contract R4/R5 + O3 hold; #5403/#5418 v6 canonical-ledger unification.
…, R5 no-append, R6 projections, FIFO cap

Five new cases in tests/fastpath.test.ts (suite now 27), encoding the v6
contract face in host-id space (#5403/#5418 v6; contract R4/R5/R6 + O1–O3 holds):

1. rewrite-passthrough (R4/O3): seed a world [s,a,b,c] with a hit whose list
   [h-b,h-c] grows the ledger, then send T=6 with slots
   ["h-c","X1","h-b"] — unknown real id BEFORE a still-known one. Width law
   len===T-K+1 still holds and slot K-1 keeps its id, so every cheap gate
   passes; the mid-list proof fires: ONE conservative legacy pass, CREDITED
   (parity with the pre-v6 credited catches so pathological cycles stay
   breaker-bounded), ids still enter the ledger. The next turn rides clean on
   the grown ledger → hit(no-count). Convergence in one pass, no speculative
   rebuild.
2. Same turn with BILI_ROLLOUT_WATERMARK_COMPACTION_PASSTHROUGH=0: the v6
   layer is skipped entirely, the byte-compare covers only unchanged prefix
   positions ⇒ no warns, plain "hit" — exact pre-v6 behavior for that turn.
3. unidentifiable increment (R5): body grows but the increment slot is "?"
   (host minted no id) ⇒ hit WITHOUT ledger append + structured diagnostic
   "[pf-v6] unidentifiable-increment ... O1/O2"; the anonymous content still
   rides the replayed snapshot.
4. per-model projections share ONE logical ledger (R6): m-one hits, flip to
   m-two ⇒ ONE "seed-stale" legacy pass builds m-two's projection, then pure
   incremental; flipping back is symmetric. The ledger itself is
   model-independent.
5. FIFO cap: inflate the persisted lg to PF_LEDGER_CAP=2048 via ext, append
   one honest id ⇒ exactly the single oldest entry drops (oldest-first trim,
   newest kept) — bounded like the obs ≤4KB discipline.

Geometry notes (cost hours to derive, kept here for the next maintainer):
- shim lists span elements [K-1..T-1]; shape gate demands length T-K+1 where
  K = seed.stampT; slot[0] is the origin anchor (origin-pin vs last id of the
  previous turn's list, '?' neutral either side);
- parseTailIds REJECTS duplicate real ids within one list ('?' may repeat),
  so a mid-insertion case needs TWO distinct known ids in play;
- known-id re-emergence must sit at index ≥ 1 — index 0 can never be an
  unknown at this layer (the pin settles it first).

Ref contract R4/R5/R6, owner rulings 2026-11-03 (m02699/m02725/m02730 line);
#3756/#3658 alignment doctrine consumed as the id-fast-path domain per
implementation guidance item 2.
ci-check added 2 commits November 11, 2026 08:30
)

Union-of-intent resolutions:
- src/rollout.ts: keep all four feature entries (PR: nativeIdentityObservation,
  watermarkCompactionPassthrough; master: identityChannelHarvest)
- src/server/prepare-openai.ts: msgid pin block BEFORE the v6 transcript-alignment
  probe (anthropic-twin order), so the probe sees the post-pin view
- website/config-reference/rollout.yaml: master's desc_en/desc_zh format kept;
  both PR entries carried over with doc refs updated to post-migration
  technical-notes(.zh-CN).md names
- TECHNICAL-NOTES.md/.zh-CN.md, CONFIGURATION.md/.zh-CN.md: root files take
  master's redirect stubs (#5920 layout migration); PR content transplanted —
  watermark-section superset (incl. v6 paragraph + M1 section) spliced into
  docs/technical-notes.md/.zh-CN.md; docs/configuration.md/.zh-CN.md regenerated
  via tools/gen-config-docs.mjs (i18n ok, coverage ok)
- tests/rollout.test.ts: master's generic decisions test adapted for nested
  (base:) features — source 'off' when the base cohort excludes the install
  (watermarkCompactionPassthrough inside the 5% pluginFullResendWatermark grey)

Verified: typecheck clean; targeted suites 100/100 (issue5571-native-identity-obs,
fastpath, issue5403-watermark-fastpath, msgid-pin, openai-msgid-pin,
snapshot-render + chain, rollout + identity-unified, msgid-rule-parity).
…erge

- reference/architecture.md: list src/server/native-identity-obs.ts and
  native-identity-profiles.ts in the src/server/ tree (guard requires full
  src/ coverage; the map predates these modules from the #5571 M1 PR)
- src/server/snapshot-render.ts: TranscriptProbeResult is used only inside
  snapshot-render.ts (probeTranscriptAlignment return annotation) — drop the
  leading export per the unused-exports gate (INTERNAL_ONLY)
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 车道本轮处置完毕:合并冲突 + 两道静态门禁 + 一个测试 shard 失败,全部解决,head 37cbd311e5 全矩阵绿。七问如下:

1. 是什么问题
三个连续阻塞:① PR 分支与 master 冲突(7 个文件);② 冲突解决后 architecture-docs、unused-exports 两道门禁红;③ test (windows-latest, 22, 2) shard 挂一个用例。

2. 什么原因导致的
① 分支点之后 master 前进了 885 个提交,期间发生文档布局迁移(#5920:根目录 CONFIGURATION/TECHNICAL-NOTES 变成重定向 stub,正文移入 docs/)且 rollout 表新增 feature——与 PR 的新增条目正面相撞;② PR 新增的 src/server/native-identity-obs.ts / native-identity-profiles.ts 两个模块未列入 reference/architecture.md 的文件树(map 早于这两个模块存在,PR 分支上该门禁还不查 docs/ 布局,合入 master 后才暴露);TranscriptProbeResult 只在 snapshot-render.ts 内部使用却带 export;③ 经排查为单 runner 环境性时序抖动,非代码问题(证据链见第 6 问)。

3. 影响范围与严重程度
对用户行为零影响。冲突解决保持双方语义并集;两处门禁修复是文档补录 + 去掉一个无消费者的 export(INTERNAL_ONLY,函数签名返回类型不受影响);测试失败与 PR 代码路径无关。观测层功能仍默认关(knob percent:0),默认行为无任何漂移。

4. 怎么修复的

  • 冲突:merge origin/master 进分支,逐文件按意图并集解决——rollout.ts 四个 feature 全保留(nativeIdentityObservation、watermarkCompactionPassthrough、identityChannelHarvest + master 既有项);prepare-openai.ts 按 anthropic twin 的顺序先 msgid-pin 后 v6 probe;文档跟随新布局(rollout.yaml 重建 + gen-config-docs 重新生成 + technical-notes 严格超集拼接);tests/rollout.test.ts 适配嵌套 feature「base cohort 不含 install 时 source=off」的语义(保留断言牙口)。merge commit a3e4aa6a27。
  • 门禁:reference/architecture.md 树中补两行模块条目;snapshot-render.ts 的 TranscriptProbeResult 去掉 export。commit 37cbd311e5。(插曲:第一次编辑意外带入行首空格,把 box-drawing 树的深度解析打乱导致 300+ 条误报,已定位字节级原因并修正。)
  • 测试红:完成证据链后对单个失败 job 执行一次 rerun-failed-jobs,attempt 2 通过。

5. 是否有回归问题
无。typecheck 干净;定向套件 10 个文件 100/100(含适配后的 rollout.test.ts、msgid-rule-parity、fastpath、watermark、snapshot-render 链);CI 全矩阵(ubuntu/windows × node22/24 × 全部 shard)+ 所有 guard 全绿。唯一的行为面变化是 tests/rollout.test.ts 的断言从「所有 feature source=bucket」改为按 base cohort 判定,这是适配 master 既有的嵌套 feature 机制,不是行为变更。

6. 当前实时状态(本轮实测)
head 37cbd311e5,run 42939521178 结论 completed/success:12 个 test shard 全绿(windows-22 shard 2 为 rerun 后 attempt 2)、architecture-docs/unused-exports 及全部 guard 绿;合并冲突已清除(mergeable=True,现处 blocked=等 review/approval)。关于第③项的证据链:该测试及其全部代码路径(relay/stream-responses/server.ts/wire-exit-matrix)PR 相对基点零改动、与 master 字节一致;master(58a8796)自身全绿;本地跑该套件 8/8 绿;同 run 其余 11 个 shard 全绿;失败行耗时 21.7s 吃满 teardown 扩展窗口才断言 closed=0,而同进程姊妹行(openai-sse 234ms、anthropic-sse 83ms)秒过——典型负载下 socket close 延迟超过窗口。注:镜像缓存轮询当时停滞(冻结在推送前的 sha),以上状态用一次性 GitHub API 读取核实,非轮询。

7. 收工判定
三个阻塞全部消除,无需设计决策、未开新 issue(flake 结论已按纪律一行记录到 #4599)。PR 处于可合状态,剩人工 review/merge——我不代合。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

本轮处理了一次新的冲突唤醒:master 在我上次合并之后又前进了(#6132 移除 compressAtFoldPoint 特性 + #6160 e2e 测试修正),GitHub 因此把 PR 重新标成 dirty。已合并解决并推送,head 现为 4b4d024aa2,实时核验 mergeable=True(blocked=等 review,非冲突)。七问如下:

1. 是什么问题
不是代码问题,是基线漂移:master 新增 5 个提交(全部来自 compressAtFoldPoint 特性移除 #6132 及其 e2e 修正),与本 PR 的改动面再次相交,PR 被重新标记为有冲突。

2. 什么原因导致的
本 PR 分支在 37cbd311e5 时与 master(58a8796224)是干净的;随后 master 合入 #6132,改动了 src/server/prepare-openai.ts(删除 fold-point 逻辑)和 docs/website 配置面——与本 PR 在同一文件的不同区域有改动,触发 GitHub 重算出 dirty。

3. 影响范围与严重程度
无用户行为影响。纯基线同步;双方改动区域不相交。

4. 怎么修复的
把新 master 合并进 PR 分支(merge commit 4b4d024aa2)。本次三方合并零文本冲突(docs/configuration.md、docs/configuration.zh-CN.md、src/server/prepare-openai.ts 均自动合并),但按纪律做了语义并集核验:① prepare-openai.ts 中 master 的 fold-point 删除完整生效(全文件零残留引用),同时本 PR 的 msgid-pin 块(188 行起)与 v6 probe 调用(209 行)原样保留、顺序不变(pin 先于 probe,probe 看到 pin 后的视图);② docs 双语文件中本 PR 的 4 条 rollout 观测层条目与 master 移除 fold-point 条目两者并存,无互相覆盖。typecheck 干净,定向套件 79/79 通过(rollout、openai-msgid-pin、snapshot-render、fastpath、native-identity-obs、msgid-rule-parity)。推送前按纪律 fetch 确认远端头未变,plain push。

5. 是否有回归问题
无。本次合并未引入任何本 PR 之外的语义变化——fold-point 移除是 master 自己的已发布变更(其自身 CI 全绿),本 PR 侧代码路径经 typecheck + 定向测试验证完好。高危模式逐项过:无语义漂移、无默认值/阈值变化、无输出格式变化。

6. 当前实时状态(本轮实测)
head 4b4d024aa2:mergeable=True / mergeable_state=blocked(等人工 review/approval);新一轮 CI 已在跑(门禁类如 unused-exports/static-gates/ubuntu 侧已见绿,test shard 运行中)。注:镜像缓存轮询滞后于新 sha,以上为一次性 GitHub API 读取核实;若后续任一 test shard 红,本车道会自动再醒处置。

7. 收工判定
冲突消除,无需决策、未开新 issue。PR 回到「可合待审」状态,剩人工 review/merge——我不代合。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] (ai: vllm-qwen/qwen3.8-27b)

Progress report — re-review of new head 37cbd311e (session was restarted after a stall; resuming now)

Since my last review (head e1a3ef31f + my fix f6968ee9f), the branch moved:

  1. Master merge: a3e4aa6a27 merges origin/master @ 58a8796224 (verified ancestor of current master 33104b1f9) into m1-native-identity, followed by 37cbd311e5 "fix(ci): pass architecture-docs + unused-exports gates after master merge".
  2. Scope grew beyond M1 — the authoritative three-dot diff vs master is now 16 files, +1257/−34. Beyond the M1 observation layer (unchanged file set: native-identity-profiles.ts, native-identity-obs.ts, handle.ts call site, tests/issue5571-*, docs, rollout.yaml), the branch now also carries:
  3. Kernel check: the PR diff contains no kernel/ changes (merge-base..head verified) → the authorization hard rule does not fire.

Currently doing: reading all new hunks end-to-end, verifying M1 files are byte-identical to f6968ee9f, checking CI for the new head via mirror cache, then targeted local tests before posting the full structured review. Full verdict coming shortly.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI lane — stale trigger confirmed, nothing to fix this run.

This wake reported architecture-docs + unused-exports red at head a3e4aa6a27 (run 42932182954). That head is stale: it is an ancestor of the current head and both gates were already fixed in the earlier wake (commit 37cbd311e5 "fix(ci): pass architecture-docs + unused-exports gates after master merge").

Live verification performed in this run (no memory/earlier-comment assertions):

No code changes and no push were needed this run. The CI lane has nothing outstanding: the PR is green and awaits human review/merge (I do not merge).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant