Repository navigation
[PR] Native-Identity Fastpath: Observation Layer (M1) + Identity-Only Hit Gate (v5) | 原生身份快路径:观测层+身份制判据(一次做完) - #5576
Closed
ranxianglei wants to merge 23 commits into
Closed
ranxianglei wants to merge 23 commits into
ranxianglei wants to merge 23 commits into
Conversation
…t-native session/msgid evidence #5573 M1 (L1): pure-observation collection of host-native identity evidence per the owner design draft — per-client profile table, session-ext persistence, one log line per observed request. Rollout-gated, default off. - src/server/native-identity-profiles.ts: evidence table (wb / codex / cc / opencode / gemini / unknown) with match rules, header granularity, bounded body signals and compaction prefaces; BILI_NATIVE_IDENTITY_PROFILES_JSON extends additively (built-ins stay the floor). - src/server/native-identity.ts: bounded extraction (fixed-pattern scans, never a full-body JSON.parse), sha1-8 digest red line (raw id values never persist or log), stability tracking + event FIFO under ext key nidv:obs (~4KB budget), debug line per request with WARN escalation on compaction hits. Follows the fastpath pf:state house pattern for ext access (replicasAvailable guard; legacy-only degrades to log-line-only). - Collection sites: the relay chokepoint (all model-shaped POSTs, ahead of passthrough/bypass) and the incremental-lane entry (#3248 hosts — the small delta surface plus the #4491 msgIds envelope). - Config surface: rollout.features.nativeIdentityObservation (default 0/off), env BILI_ROLLOUT_NATIVE_IDENTITY_OBSERVATION; rollout.yaml seed + CONFIGURATION.md en/zh regenerated via tools/gen-config-docs.mjs. Feeds NO decision: request handling, folding, parsing and outbound bytes are untouched; off or unclassified traffic produces zero diff (hermetic A/B pinned, incl. known clients). Tests: 19 pins (profile fixtures + negatives, stability matrix, adversarial preface placement, PII red line, FIFO/budget caps, restart continuity, gate off/on, byte-stable, latency smoke p99~2ms on a 1MiB codex-shaped body).
Contributor
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.8083-pr.5576.4db0b26This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5576.tgz
npm install -g packageInstall instructions are refreshed on each push. |
ranxianglei
changed the title
…nternal types — CI gate follow-up for the M1 commit docs-coverage + lifecycle-ledger-guard: the operator extension env is now seeded in website/config-reference/env-only.yaml (CONFIGURATION en/zh regenerated) and registered in LIFECYCLE.md §1 with its release condition. unused-exports: five observation types and three profile-table internals had no cross-file consumer — dropped the leading export (tests consume them structurally).
…le priority comment Review follow-up for #5576 (M1 observation layer), zero-behavior: - remove write-only envOverrideWarned (its guard block never warned); - resolveNativeIdProfile doc now states env-added rows trail built-ins.
Owner
Author
(ai: vllm-qwen/qwen3.8-27b) 查重：重复 —— 本 PR（GitHub #5576）与在途 PR #5573 是同一交付物的两个并行实现。 两者都实现设计 issue #5571 的 M1（客户端原生身份观测层），配置面完全相同（ 独立审核已完成（对 origin/master d244fff 逐文件审了 diff 与代码，不依赖 PR 自述）：
顺手修掉的两个小问题（已推 PR 分支，commit 结构化八问：
收工判定：PR 本身可合（架构正确、四条红线全部有测试钉、小问题已顺手修掉、CI 跑动中）；剩余唯一事项是 owner 拍板保留哪个 PR——决策项见下一条评论（本线程已打 |
Owner
Author
ranxianglei
changed the title
ranxianglei
commented
Nov 3, 2026
ranxianglei
left a comment
ranxianglei
left a comment
Owner
Author
There was a problem hiding this comment.
自审(合流版覆盖范围:Part A M1 观测层 + Part B v5 判据,评审人=作者本人):
结论:可合并。 核对点:
- 两部分文件集不相交(A: profiles/obs/handle.ts/rollout.yaml/docs;B: fastpath.ts + fastpath 测试),在含 owner 打磨(dual_mismatch 摘要化/CI gate/dead extraHeaders)的最终谱系上叠加零冲突。
- v5 闸门顺序正确:sess-change 置于 shape/prevId 之后、multimodal/fragment 之前(fail-fast),仅 id 模式生效;
readSaved白名单cs恰 8 位 hex 校验(初版遗漏→闸门静默失效,已修+回归钉死)。 - digest-only 两侧钉死(blob 与日志);off⇒零存储流量;unknown 客户端不写——观察层对现网行为零影响,golden diff 预期为零。
- 两处 owner 需知悉语义(body 已标):① system 无 msgid → head-drift 两侧仍挡命中(flagged deviation,可否决);② head 就地改写骑乘 stash 前缀核,warn 流供 M3;sess 切换"跟随流量走",每次切换一次冷解析。
- count 模式逐字节行为不变;131/131 + tsc 0(最终谱系复跑)。
…1-03 lineage) Owner ruling 2026-11-03 (m02609 consolidation): head id inequality demoted to warn stream for shape-valid host-msgid clients; byte equality stays the gate in count mode; NEW cs sess-change gate (client session header digest); readSaved whitelist fix for cs. Content identical to the earlier 0e83b21/dd20ffb4 attempt, re-based onto the true PR head lineage (878aad4 M1 + guards + polish).
ranxianglei
commented
Nov 3, 2026
ranxianglei
left a comment
ranxianglei
left a comment
Owner
Author
There was a problem hiding this comment.
Independent review at 7a645c3b (stack: your M1 + v5 + v6)
Verified on a clean re-run, not just trusting the implementer pass:
tsc --noEmitclean; target suites 146/146 (fastpath27 incl. 5 new v6 pins,issue5403-watermark-fastpath,lane-decisions-matrix,native-identity-observe— your L1 suite stays green under the new stack); affected-module set (14 suites) 121/122 with the one skip being pre-existing and conditional (SMT solver not on PATH).- L1 commits untouched by the port; the only cross-lineage merge decisions were doc-table additions (both knob rows kept) and taking
tests/fastpath.test.tswhole (the file-level content = old-base tests + v5 tests + v6 tests, which is exactly this lineage's base plus both layers). - No small fixes needed in review — nothing to push back as fix commits.
Three points I scrutinized hardest (all hold, flagged for your sign-off anyway):
- Mid-insertion criterion (
src/server/fastpath.ts, v6 classification block): "unknown real id before a still-known slot" is the only mutation shape provable from ids alone — vanished ids are legal (lists span[K-1..T-1]), index 0 is settled by the origin pin first, pure-trailing fresh ids ride by R4/O3 design. The in-code proof requirement is pinned at the scan site (2267337c). Residual trust surface = pre-v6's exactly (ids were always the anchor); strictly narrower than before. - Ledger growth policy (
growPfLedger+finalizeWatermarkTurn): grows from any shape-valid list (hit ORtail-ids-prevmiss — shape validity IS id authenticity), holds on sess-change swap (ledgerHold) so a swapped conversation can't poison the baseline, off for knob-off turns (no half-migrated sessions). Also fixed a latent breaker bug found while stacking:noteCreditedMiss's arm-path was dropping evenlwhen disabling the lane — ledger + last-list now survive a 5-strike disable. - Rewrite→recovery convergence: one credited
rewrite-passthroughcosts a full legacy pass BY DESIGN; credited parity keeps pathological churn bounded by the existing 5-miss/60-min breaker per session; next turn rides cleanly because the mutation's fresh ids are already appended. Test 1 pins the full end-to-end geometry.
Interpretation calls for sign-off (none deviate from contract semantics): I1 set-diff-as-alignment-front-load (equivalence argument in module header), I2 mid-insertion as sole signal, I3 knob-off includes state migration. All three are documented in-code at their sites.
Branch hygiene note (not touched, your call): remote branch m1-native-identity still exists from an earlier iteration — it carries a SUPERSEDED copy of the observation layer (old native-identity-obs.ts/profiles.ts split) plus v5/v6 on the stale base. This PR's head 2026-11-03_native-identity-observation is the sole line of record; suggest deleting the old branch after merge.
ranxianglei
force-pushed
the
2026-11-03_native-identity-observation
branch
from
November 3, 2026 06:22
7a645c3 to
6cda69a
Compare
…hit authority (#5576) Owner contract (frozen 2026-11-03, pf-v6-contract.md): bind the ledger to the CANONICAL structure, not the wire protocol — byte-compare machinery retires. What changed (new rollout gate watermarkCanonicalV6, default 0%, nested grey inside pluginFullResendWatermark; OFF = v4/v5 lane byte-for-byte untouched): - Hit verdict comes SOLELY from alignClientAgainstTranscript: snapshot-render's buildView (now exported) row-chain walk of the freshly converted cores over stored unified-transcript rows (kernel id fast path → content-sig fallback, min seam run 2). Surviving pre-filters are pure server-side arithmetic: ahead / tail-ids-shape / no-progress / sess-change (R7 verbatim); id VALUES are still never consulted (M1 observation decides later consumers). - View on a hit = stored rows up to the seam (covered regions render behind their synthetic acp_summary_* anchors) + this turn's fresh tail cores; windowMessages slice element-space from the wireIndex seam origin with the persisted-anchor fallback. System text re-decodes from the CURRENT head every turn — passthrough by construction, retiring the v5 head-drift gate. - R6 logical ledger: one model-independent ledger shared across models — model switches are non-events; per-model projections are derived rendering only. Host-side content rewrites break the chain (chain-not-started / seam-too-short) and self-heal through UNcredited legacy turns by default (BILI_PF_V6_REWRITE_STRICT=1 credits them toward the breaker for exotic thrash-rewriting hosts). - Zero-retention doctrine: canonical outcomes pin NO seed stash — no multi-MiB prefix cores resident between turns, memory flat across model switches. Full-length joins (fresh trailing element duplicating old content false-matching through the sig fallback) degrade to uncredited v6-no-seam-origin instead of silently dropping the new tail (tested). Docs: rollout.yaml + CONFIGURATION(.zh-CN).md rows, BILI_PF_V6_REWRITE_STRICT entry, TECHNICAL-NOTES(.zh-CN).md §v6 under the watermark section. Tests: tests/fastpath-v6-canonical.test.ts (12 cases incl. anchor/suppression order, system passthrough vs v5 head-drift contrast, R6/R7, strict-credit both modes, width-shrink before the expensive stage, dup-numbering agreement, guard residual, and a v6-OFF v5 regression anchor); tsc 0 + 283/283 across 20 related suites locally.
ranxianglei
commented
Nov 3, 2026
ranxianglei
left a comment
ranxianglei
left a comment
Owner
Author
There was a problem hiding this comment.
v6 重写完成(9cd5f5c)——按冻结契约从标准结构重实现,worker 的 host-id 账本版本已从分支移除(本地留存 wip/v6-worker-superseded 备查)。要点:命中判定唯一权威 = 转录稿行链对齐(snapshot-render buildView 导出复用);字节比对机制降级为纯负向过滤(戳超前/列表宽度/零增量/会话换绑);system 每轮从当前头部解码(透传构造,head-drift 门退役);换模型为非事件(R6 逻辑账本);宿主改写断链默认不计名自愈 legacy(BILI_PF_V6_REWRITE_STRICT=1 计名进熔断);规范车道零钉存(不驻留前缀核)。默认 0% 灰度,关时 v4/v5 逐字节不动。tsc 0、config-surface guard OK、20 个相关套件 283/283(含新增 12 例 v6 测试 + v6 关闭时的 v5 回归锚点)。已知残余(sig 回退下全新尾部与旧内容雷同会误配 → 守卫转为不计名降级而非静默丢尾)已在代码注释与测试中钉死。PR 描述已同步 L3/验证/签核点。
…apshot partitions (#5600) Design authority: issue #5600 (owner rulings 2026-11-03). Default OFF (BILI_FASTPATH_PF_MSGID); zero impact on existing lanes while off/soft-degraded. Identity (§3.1): server mints pf_<hex12> per (session, body-shape); client echoes via x-bili-pf-session. Unverified echo → probe by body geometry; miss → mint fresh; fabricated echoes self-heal through probe/mint, never trusted. Storage (§4): pf_rows is a strict append-only L0 ledger keyed on absolute body element ordinals — multimodal pairs share one ledger id + one elem, so row count ≠ element count and id matching is an EXISTS over (sid, elem), never an ordinal arithmetic. Compaction is archive-and-reset: rows are NEVER deleted (§5, #3880 class); the watermark moves to the post-archive frontier. Meta plane is LSM (pf_log WAL + pf_ckpt snapshots with full-state checksums, folds every 32 entries) so a lost latest ckpt never corrupts state; hydration replays ckpt+log at boot before serving (§7). Session delete cascades with no retention period. Hit authority (§6): width arithmetic against the resident's own frontier plus origin pin (elem-scoped ledger EXISTS). Soft plane: lane requires pfStore(); any degradation falls back to legacy lanes byte-for-byte.
ranxianglei
added
the
ci:full
added 4 commits
November 3, 2026 19:39
) Closes the four gap rows of the #5600 design-contract test matrix on the server-minted pf_sid lane (owner ruling 2026-11-03, option A): - sess-change: native session-carrier swap under an unchanged sid and a shape-valid list is ONE uncredited legal full (R7 parity); the sid echoes through and the pf_rows frontier is untouched (archive-and-reset doctrine). - intra-family model switch: same geometry + origin pin under a different model string still HITS (no model gate on this lane; the ledger is model-independent per contract R6). - breaker naming: five consecutive credited v6m-window-system strikes arm the shared five-miss/60-min breaker; the held turn restamps WITHOUT resolving a sid; after the TTL backdates, the next credited miss counts fresh (staleHold reset) and clears disabledAt. - id-stable in-place content edit: geometry and ids unchanged while one element's text changes — the hit rides the frozen core verbatim (server snapshot stays absolute authority, §6.2: content bytes never judge) and exactly the one new element row binds its ledger id.
…, #5600) Follow-up 2 of the owner's ruling on #5600 (option A): - rollout.yaml: new rollout.features.watermarkV6MsgId entry (nested grey inside pluginFullResendWatermark, default 0%, env override BILI_ROLLOUT_WATERMARK_V6_MSG_ID) with full semantics en/zh; the canonical-lane entry stays. - TECHNICAL-NOTES (.zh-CN): new 'v6 msgid lane' section — echo->probe->mint identity resolution over the durable pf_rows partition store (LSM meta plane, WAL + checksummed checkpoints, boot hydration, archive-and-reset), hit authority (ledger-width arithmetic vs our own frontier + window-origin pin; content bytes never judged), event matrix (compaction-class shrink, always-credited v6m-window-system, uncredited sess-change, model switches non-events, shared breaker), soft-plane degradation, zero retention. Dispatch priority noted inside the canonical-lane section (msgid first, canonical its else-if). - CONFIGURATION (.zh-CN): generated feature/env rows via tools/gen-config-docs.mjs; BILI_PF_V6_REWRITE_STRICT appendix rows now describe BOTH lanes (msgid: compaction-class credit; canonical: chain breaks; v6m-window-system independent of the switch). - env-only.yaml: seed entry for BILI_PF_V6_REWRITE_STRICT — fixes the docs-coverage gate (check-coverage failed on the L4 commit: the variable was read in src but undocumented).
…5600) - lifecycle-ledger-guard: BILI_PF_V6_REWRITE_STRICT is read from src but was never registered in LIFECYCLE.md — added to the §2.9 compression-lane table (same gap class this PR's d908da4 already closed for BILI_NATIVE_IDENTITY_PROFILES_JSON). Guard now green locally. - mysql-contract: the v11 fastpath snapshot store landed SQLite-only (schema.ts pf_rows/pf_log/pf_ckpt) while store.ts issues its queries against whichever backend is configured — the MySQL DDL had none of the three tables. Added them to the always-run fresh-install DDL in the file's dialect (LONGTEXT JSON cols, BIGINT seqs, inline KEYs; ledger_id VARCHAR(512) fits InnoDB's utf8mb4 index limit) and bumped MYSQL_SCHEMA_VERSION 10->11 to keep both backends' shape bookkeeping in lockstep with STORAGE_SCHEMA_VERSION. No local MySQL in this environment — verdict comes from the CI lane.
ranxianglei
added a commit
that referenced
this pull request
Nov 3, 2026
…t FULL-ON for remote CI coverage
Owner ruling 2026-11-03: push with the feature fully open so remote CI —
including windows scenarios not reproducible locally (npm ci / install
lanes) — exercises the watermark path end-to-end; REVERT before merge.
- registry defaults 5%/0%/0% -> 100% (pluginFullResendWatermark,
watermarkCanonicalV6, watermarkV6MsgId), each flagged TEMP-ON inline
- golden plan assertion (wm-fastpath-intro-notice) pinned to 100 for this
push only (shipped default stays { percent: 5 }, owner ruling 2026-11-02)
- fastpath-v6-canonical suite pins msgid OFF explicitly (while live the
msgid lane intentionally shadows canonical — isolation needs the pin)
- fastpath-pf-msgid 'lane off' test pins the env kill-switch explicitly
REVERT COMMIT (before merge): restore { percent: 5 } / { percent: 0, base }
x2 in src/rollout.ts, drop the three TEMP markers and both test pins.
added 4 commits
November 3, 2026 23:26
…conflict-fix # Conflicts: # src/server/handle.ts
…s it missed The v6 msgid lane (#5600, commit 8cce4df) raised STORAGE_SCHEMA_VERSION to 11 and updated tests/storage-ext.test.ts + tests/storage-unified.test.ts, but left stale v10 pins in four other storage suites — every migration-to-head assertion now fails on PR heads. Replace the hardcoded 10s with the STORAGE_SCHEMA_VERSION constant (the pattern #5600 already used), so future head bumps cannot re-break these pins.
The #5576 canonical lane added watermarkCanonicalV6 {percent:0, base:pluginFullResendWatermark} — the first nested-base feature. Nested features resolve inside their base's cohort (base off ⇒ source 'off', by design per NESTING in src/rollout.ts), which breaks the universal 'every percent feature is bucket' assertion. Skip based features in that loop; their semantics are covered by the nesting resolution itself.
…r slices src/server/native-identity.ts (M1 observation layer) trims its NidvEvent[] ring with .slice(-8)/.slice(-4) — array receivers, code-unit-safe per the gate's own rule. Add the allowlist entry following the existing array-receiver convention.
added 2 commits
November 4, 2026 00:30
… node22 flake) p99 over 100 samples measured 27.348ms on one windows-latest/node22 shard vs 2.6–4.2ms on the other three shards of the same run — the 20ms bound was not CI-portable. 50ms absorbs the observed runner variance while staying far below any parse-class regression cost.
MysqlBackend.init() executes MYSQL_DDL statement-by-statement via split(";"). The #5600 pf_rows DDL comment contains a semicolon ('pf_sid; ledger_id holds...'), so the naive split cut the comment in half and executed its orphaned tail as SQL — ER_PARSE_ERROR on MySQL 8.0 in the mysql-contract lane, and on every real MySQL-backend boot after upgrade. Strip -- line comments before splitting (the blob has no string literals), keeping the documentation intact.
Owner
Author
(ai: vllm-qwen/qwen3.8-27b) PR #5576 的两个阻塞(与 master 冲突 + 红 CI)已全部清除,新 head 1. 是什么问题 2. 什么原因导致的
3. 影响范围与严重程度 4. 怎么修复的(5 个独立标注 commit,均在 PR 分支上)
5. 是否有回归问题 6. 是否破坏缓存 7. 是否改了设计方向 8. 是否引入了新的配置 结论:冲突已解、红 CI 已修且全量绿,无需决策项、无授权卡点,可以进入正常 review/合并(合并由人操作)。 |
…+ deterministic ledger-tie ordering (#5576 review) - mysql.ts: 'Blob has no literals' was factually false ('{}', 'sha256', 'active' defaults exist); state the real invariant the ';' split relies on. - pfLookupLedger: ORDER BY pos -> pos, sid (pos is unique per sid only; cross-sid ties were engine-order-dependent between SQLite and MySQL).
Owner
Author
|
(ai: sglang-gpu0/qwen3.8-27b) 总体结论：当前形态 NO-GO。 架构面整体扎实（clean 项见下），但有 1 个 BLOCKER（msgid 车道重启后静默送截断历史，已实证复现）+ 3 个 MAJOR。两个审查面（存储平面 / 判决平面）各自独立走完全部 hunt 区，BLOCKER 已在本地复核属实。 1. 这是什么水位快路径（#5403/#5418）的身份加固系列，四层各带开关：M1 原生身份观测（ 2. BLOCKERB1 — msgid 车道进程重启后，hydrated resident 用截断的核心集重建，随后以可信 3. MAJORV1 — v5 ride 行为未受任何新开关门控，随父开关 5% 灰度直接进现役 cohort；无退出路径；shipping 方向零测试行。 base 上头漂移一律 S2 — gc S3 — 请求中途 pf 读故障杀死活请求而不是降级 legacy。 4. MINOR / NIT（一句话可修的列在前）
5. 已直接修的小瑕疵（additive commit
|
…t FULL-ON for remote CI coverage
Owner ruling 2026-11-03: push with the feature fully open so remote CI —
including windows scenarios not reproducible locally (npm ci / install
lanes) — exercises the watermark path end-to-end; REVERT before merge.
- registry defaults 5%/0%/0% -> 100% (pluginFullResendWatermark,
watermarkCanonicalV6, watermarkV6MsgId), each flagged TEMP-ON inline
- golden plan assertion (wm-fastpath-intro-notice) pinned to 100 for this
push only (shipped default stays { percent: 5 }, owner ruling 2026-11-02)
- fastpath-v6-canonical suite pins msgid OFF explicitly (while live the
msgid lane intentionally shadows canonical — isolation needs the pin)
- fastpath-pf-msgid 'lane off' test pins the env kill-switch explicitly
REVERT COMMIT (before merge): restore { percent: 5 } / { percent: 0, base }
x2 in src/rollout.ts, drop the three TEMP markers and both test pins.
…on BLOCKER + S2-S5 Review: #5576 (comment) B1 (BLOCKER): post-restart hydration rebuilt the resident from a TRUNCATED core set. Every geom entry stores f = its window's pre-append start; the old applyGeom consumed that as the row-read frontier for EVERY entry, so after N windows the restart read began at the LAST window start while elems still claimed full width -> next legit hit served 'last window + new tail', u1/a1 gone, compounding every boot. CI missed it because hydration tests asserted decision/measuredT/headers, never view CONTENT. Fix (review option b): only arm/reset entries may move the frontier; geom's f is no longer read back. Folding deletes the arm/reset entry, so the checkpoint now carries the active world ORIGIN (field o) and hydrate seeds basePos from it; PFResident gains worldOrigin. Unresolvable shape degrades to legacy. B1 regressions (content-level, as the review demanded): repeated-boot test asserts hydrated view texts == FULL transcript across 3 simulated boots crossing L1 folds; bad-checkpoint test covers pre-fix shapes and the one-turn self-heal. S2: archiveSession swept session_ext (killing the pf:sids registry) but not the pf_* tables => archived-without-restore sessions kept a permanent full-transcript copy. The archive transaction now sweeps the partitions via pfDeleteSids(sids, true), deleteSession's pattern. S3: read-side faults (SQLITE_BUSY under shared index.db, disk full) killed live requests instead of degrading. evaluateMsgIdLane is now a thin try/catch wrapper around the core: any read-side throw falls through to the existing lanes (same null contract as plane absence). S4: the ckpt checksum hashed a {t} subset and had zero verify points. It now covers the meta_json bytes themselves and hydrate VERIFIES it; an untrusted ckpt is skipped (not fatal) so a live tail can still name the frontier. S5: MySQL/replica shadows (sync===null) exposed NO-OP pf closures behind replicasAvailable()==true -> per-turn fabrication warn + unbounded sid minting. pfStore() gates on the new UnifiedShadow.pfPlaneUsable() capability probe and reports null (soft plane). Self-heal tightening: finalize re-ingests on v6m-hydrate as well (append-only archive-and-reset), bounding an unrecoverable-frontier partition to ONE legacy turn — the appended reset entry carries fromPos for the next hydration.
…atrix, loose compaction, infra-fault degrade, forged-burst contract Closes the remaining test-side lines of #5576 (comment) : - V1(a)+V3 (tests/fastpath.test.ts): the v5 lane is the only LIVE identity lane today, yet its #3617 matrix had zero positive-direction lines HERE (only inside canonical/msgid lanes). Now pinned side-by-side: id-mode stable-id in-place edit => HIT + diagnostic ride warn + STALE byte outbound (owner ruling 2026-11-03 semantics, frozen-core behavior); count-only same bytes => head-prefix-drift uncredited gate. Plus the R7 sess-change trio on the default lane: swapped carrier = event (uncredited, reseed), same carrier silent, carrier appearing where none was stored neutral. - V4 (tests/fastpath-pf-msgid.test.ts): compaction-class misses stay UNcredited with BILI_PF_V6_REWRITE_STRICT off — breaker untouched across repeated host compactions while the frontier still self-heals to a ride. - S3+V5: mid-flight read-side store fault (SQLITE_BUSY shape) degrades to the legacy lanes WITHOUT throwing and WITHOUT breaker credit, even under strict — infra failures are not misses of this partition. - V6 (documented contract, behavior frozen not changed): forged-sid echoes probe, fail, mint fresh — one sid per turn, registry grows linearly, nothing ever credits toward the park. Pinned deliberately so any future cap/abuse policy lands as a conscious decision (policy = owner ruling, reported separately).
…e cap, M1 bucket digest, V2 seam note Completes the review lines the parallel round-1 commits (e9fa08b/4169c6948) left open, on top of their base: - V1(b): the id-mode in-place edit ride had no lifetime bound — the ride re-stashes its own view, so a persistent client edit kept serving stale stashed bytes until model switch/TTL/restart. Consecutive drift turns now count in persisted state d (reset by a clean head or any reseeding miss); after PF_V5_DRIFT_RIDE_CAP=5 one UNcredited legacy turn reseeds from real bytes (self-heal, never breaker fuel; warn stream bounded to one line per cap-many turns). Regression pin drives the full 5-ride episode through the cap into the healed seed; negative control verified (cap disabled => test fails). The #3617 matrix row from 4169c69 stays green alongside. - M1: plugin-lane nidv record buckets are stored DIGESTED (n:<sha1-8>) like every other bucket in the module — the raw host-native conversation id no longer reaches the ext store as a key (module privacy redline). Test read-sites follow the new bucket form; record CONTENT assertions unchanged. - V2: known seam residual documented at the snapshot-render gate, pointing at issue #5632 (filed earlier from this same review; no duplicate filed). Behavior change to disclose: id-mode sessions with a persistent in-place head edit previously rode stale bytes for the whole episode; they now get ONE forced uncredited legacy turn after 5 consecutive drift turns, after which the edited bytes are seeded. Count-only clients and all other lanes are untouched.
Owner
Author
(ai: sglang-gpu0/qwen3.8-27b) Review 全部条目已闭环。本轮有一个并发情况需要说明:两条车道并行修同一份 review——另一条先落地( 结构化八问:
需要 owner 拍板/注意的事项:
当前状态:typecheck 干净;本地定向 185 pass / 0 fail(fastpath 族 ×3 + lane-decisions-matrix + native-identity-observe + storage-archive-gc + issue4665 + compaction-archive);B1 与 V1 均有负控制验证。远程 CI 正在 |
Owner
Author
|
(ai: sglang-gpu0/qwen3.8-27b) 第二轮评审（re-review，@cf6300ed4 → 4169c69）结论：第一轮全部问题已确认修复，逻辑面复审通过。距合并还差三件事：①TEMP-ON 回退（commit message 里自己写的 REVERT BEFORE MERGE）；②rebase 到最新 master；③3 个 CI 红转绿。 一、Round-1 修复逐条核验（全部通过）
二、合并前三道硬门
三、新发现（MEDIUM，不阻塞合并，但要处理）——本轮 3 个红的根因分析3 个红全是同一族：两条 pi e2e 车道（package-native lane / codex-responses WS lane）+ 包含它们的 windows-24 shard，失败签名一致：child 请求到达 fake-upstream 时不带 plugin stamp（oracle convs=[parentConv, null]）。 我的判断：不是 #5576 判定逻辑回归，是 TEMP-ON(100%) 放大了一个既有的 e2e attach 竞态，在慢速 windows runner 上触发。证据链：
建议（test-infra 决策，给 owner/作者选）：a) e2e 环境拉长 attach 健康探测 deadline；b) 该类重量 e2e job 与其所在 shard 降并发；c) bootstrap spawn 兜底改用 listen(0) 随机口避免 18787 固定碰撞（长期更干净）。若 TEMP-ON 回退 + rebase 之后重跑仍现同形失败，再按代码问题立项。 四、遗留 minor（不阻塞）
总体：条件 GO——完成 ①②③ 且 CI 全绿即可合。 |
added 2 commits
November 4, 2026 11:16
…rey defaults Reverts the owner-ruling 2026-11-03 TEMP pre-merge verification push (2bdb33f) now that remote CI has exercised the watermark stack at full cohort. Restores the shipped defaults in src/rollout.ts: pluginFullResendWatermark { percent: 100 } -> { percent: 5 } watermarkCanonicalV6 { percent: 100 } -> { percent: 0 } watermarkV6MsgId { percent: 100 } -> { percent: 0 } Drops the three inline TEMP-ON markers and the two TEMP-only test pins: wm-fastpath-intro-notice asserts the 5% baked plan again; fastpath-v6-canonical and fastpath-pf-msgid no longer force the kill-switch envs (the registry default 0 keeps those lanes off exactly as before the push). Code now matches the documented "5 (experimental grey)" in website/config-reference/rollout.yaml and CONFIGURATION.md again.
This was referenced Nov 4, 2026
ranxianglei
added a commit
that referenced
this pull request
Nov 7, 2026
[PR] feat(fastpath): v6 composite-identity incremental fastpath lane (full rework, #5576 line voided)
Louie0806
pushed a commit
to Louie0806/billion-context
that referenced
this pull request
Nov 7, 2026
) msgid-identity-driven fastpath rework per contract issue ranxianglei#5643 (old ranxianglei#5576 line voided): server-minted pf_sid worlds keyed by (sessionid,msgid) pairs, pure set-difference judgment over a FULL element id list, scattered NEW-span parse with _N/wireIndex absolute rekeying against the whole ledger, client compression ignored (NEW may sit anywhere), and a hard identity pre-gate that REJECTS with zero side effects on channel/malformed/width/consistency faults (no silent legacy fallback, no count guessing, no signature fallback). - fastpath-v6.ts: evaluateWatermarkV6/finalizeWatermarkV6, matching rules (echo O(1)+spot-check / composite probe / mint+arm), world mirrors (LRU-cached fold of ckpt+log, never ahead of disk), pending single-transaction commits (pf_rows + pf_log together), breaker 5-strike / 60min parking with memory-first guard state, opportunistic L1 compaction in finalize, hydration path with S4 checksum aborts. - rollout: new feature watermarkV6MsgId percent:0 (default OFF), nested under pluginFullResendWatermark bucketing. - headers: x-bili-pf-session added to the upstream hop filter. - prepare-openai: v6 evaluated before the v5 watermark lane (silently falls back when not engaged); V6Hit consumes the same field-rebuild path; finalize branches by evaluated source; snapshot pin skips the empty window of a no-progress hit. - handle: FastpathV6IdentityFault mapped to a 400 JSON refusal at the prepared-pipeline call site (supresses raw forward); overflow refold fails open. - storage facade completion: pfNeedsCompaction delegation through UnifiedShadow/SessionStore (soft degrade when sidecar unavailable).
ranxianglei
added a commit
that referenced
this pull request
Nov 7, 2026
…path-v6-msgid Revert "[PR] feat(fastpath): v6 composite-identity incremental fastpath lane (full rework, #5576 line voided)"
Louie0806
pushed a commit
to Louie0806/billion-context
that referenced
this pull request
Nov 7, 2026
…th lane (full rework, ranxianglei#5576 line voided)"
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[PR] Native-Identity Fastpath: Observation Layer (M1) + Identity-Only Gate (v5) + Canonical-Ledger Unification (v6) | 原生身份快路径:观测层 + 身份制判据 + 标准账本统一 + 持久化快照 msgid 平面
One PR per owner ruling (2026-11-03): the native-identity fastpath work lands as a single stack on this branch, in four semantic layers (top-to-bottom = oldest→newest: your M1 commits at the bottom are untouched).
L1 — Observation layer (M1, owner commits, unchanged)
nativeIdentityObservationknob (default 0% / off): pure observation of client-native session/message identity (WorkBuddy x-conversation-id family + X-Compaction headers; codex installation/item id families + previous_response_id rotation + compaction preface; Claude Code dual-channel cross-check; opencode ses_ affinity; Gemini state-snapshot). Digests only (sha1-8), never raw ids, zero request-path effect. Feeds #5573 field-data collection.L2 — Identity-only hit gate (v5, commit
6cda69a7)Owner ruling 2026-11-03 (m02609 thread): with a shape-valid host-msgid list the ID CHAIN is the proof — width arithmetic re-derives counts server-side, prevId chain pins the window origin. Both byte-compare loops demote to diagnostic warn in id mode (
watermark v5: N/M head element(s) differ … riding the stashed prefix); count-only clients stay byte-gated exactly as pre-v5. Plus the cs sess-change gate: state persists an h8 digest of the client's native session header (x-conversation-id/x-claude-code-session-id/x-session-affinity— the first of these carrying a value claims; the stored digest itself refreshes to the newest claim each evaluated turn, last-evaluated-wins); when both turns claim an identity and it changes → one legacy pass + re-seed from current traffic (identity follows traffic; conversation swap must never ride stashed cores). Flagged deviation pinned in code+tests: system text carries no msgid → head-drift gate stays gating in BOTH modes on the v5 lane; v6 retires it structurally (see below).L3 — Canonical-ledger unification (v6, commit
9cd5f5cd)Contract R1–R8 (
pf-v6-contract.md, owner rulings 2026-11-03): bind the bookkeeping to the CANONICAL structure, not the wire protocol. New rollout gatewatermarkCanonicalV6(default 0%, nested grey insidepluginFullResendWatermark; OFF ⇒ v4/v5 lane byte-for-byte untouched):openaiToCoreover exactly the bytes legacy would parse, whole-body while affordable so intra-call cluster numbering can only AGREE with stored rows) andsnapshot-render.ts::buildView(now exported for this consumer) walks the stored unified-transcript rows backwards (kernel id fast path → content-sig fallback, min seam run 2, one tolerated gap). All v4/v5 byte-compare machinery survives ONLY as pure negative filters on the v6 lane: echoed-stamp-ahead (credited), ledger-list width ≠ T−K+1 (where compaction shrinks die, pre-expensive-stage), count-only zero delta, and the R7 sess-change gate (verbatim). Id VALUES are still never consulted.acp_summary_*anchors — folded history rides as ingested) + fresh tail cores;windowMessagesslice element-space from thewireIndexseam origin (persisted-anchor fallback).chain-not-started/seam-too-short) = rewrite class: default SELF-HEALING UNcredited legacy turns (host compaction is normal traffic; legacy rewrite detection then marks the boundary so later turns align against the rebased view), credited toward the breaker underBILI_PF_V6_REWRITE_STRICT=1for exotic thrash hosts. Count-only zero-growth and unresolvable shapes degrade uncredited.sessions.hardMaxBytes(Windows, 0.1.210) #5403 retention win with none of the stash's byte-reproduction debt). Known residual GUARDED + TESTED: a fresh trailing element duplicating older content can false-match through the sig fallback into a full-length join whose served view would silently drop the tail — the seam-origin guard turns that shape into uncreditedv6-no-seam-origininstead.d4f5e030..7a645c3b) was removed after review found its ledger bound to SHIM host msgids — exactly what the contract eliminates; this commit re-implements from scratch.L4 — Persistent snapshot msgid lane (v7-m, commit
8cce4df2, design authority issue #5600)New plane, fully independent of L1–L3 (default OFF; rollout gate
rollout.features.watermarkV6MsgId, nested grey insidepluginFullResendWatermark, env overrideBILI_ROLLOUT_WATERMARK_V6_MSG_ID): server mints opaquepf_<hex12>session ids per (session, body shape), client echoes viax-bili-pf-session; unverified echoes are probed by body geometry and fabricated ones self-heal through probe/mint. State lives in the durable sidecar (§3.1/§4/§5 of #5600): strict append-onlypf_rowsledger keyed on absolute body element ordinals (multimodal pairs share one id + one elem ⇒ hit matching is an EXISTS over(sid, elem)origin pin, never ordinal arithmetic), compaction as archive-and-reset with rows NEVER deleted, LSM meta plane (pf_logWAL + checksummedpf_ckpt, folds every 32 entries; boot hydration replays ckpt+log before serving), cascade on session delete, no retention period. Hit authority = width arithmetic against the resident's own frontier + origin pin; soft-plane degradation falls back to legacy lanes byte-for-byte whenever the store is unavailable.Env-wording erratum (2026-11-03): the L4 commit message references
BILI_FASTPATH_PF_MSGID— no such variable exists anywhere in src. The real gate isrollout.features.watermarkV6MsgId(config-file tier) with env overrideBILI_ROLLOUT_WATERMARK_V6_MSG_ID; the commit text stands corrected by this note.Contract (owner rulings, 2026-11-03 — governing spec for L3)
Owner's contract language (verbatim, highest source; supersedes all earlier drafts and interpretations):
Structural fact (to prevent misjudging during regression): model switches ACROSS protocol families (chat ↔ responses ↔ google proto) physically cannot share a snapshot — byte formats are incompatible, outbound is always rebuilt per family. Cross-family switch = NOT shareable (physics, not design choice). Within-family model switch = ignored (ledger is model-independent;
mstays informational).Verification (at tip
8cce4df2)tsc --noEmitclean;scripts/config-surface-guard.mjsOK (new knobs registered).tests/fastpath-pf-msgid.test.ts(7 cases: off-by-default anchor, soft-plane byte-identity, mint→arm→hard-pin hit, wrong-origin self-heal without touching the resident, fabricated echo re-mint, archive-and-reset compaction, boot hydration across an L1 fold); existing 19-casefastpath.test.tsuntouched and green alongside. Storage suitesstorage-ext/storage-unifiedpass after bumping three staleuser_versionliterals to the liveSTORAGE_SCHEMA_VERSIONconstant (v10 → v11).9cd5f5cd: 283/283 across 20 related suites incl.tests/fastpath-v6-canonical.test.ts(12 cases: anchor/suppression order exact, system-passthrough vs v5 head-drift contrast, R6 model non-event, R7 identity event, strict-credit both modes, width-shrink pre-stage, slot tolerance, dup-numbering agreement end-to-end, guarded residual, degrade reasons, and a v6-OFF v5 regression anchor proving byte-for-byte fallback).Sign-off checkpoints (interpretation calls, no semantic deviation taken)
wireIndexsidecar with persisted-anchor fallback when the codec omits it (steady-state append makes the two coincide; divergence ⇒ legacy turn, never a guess).Config surface (added per 2026-11-04 review process note)
Three rollout features, each nested inside
pluginFullResendWatermark(a child gate can only fire where the parent cohort is live):nativeIdentityObservationBILI_ROLLOUT_NATIVE_IDENTITY_OBSERVATIONwatermarkCanonicalV6BILI_ROLLOUT_WATERMARK_CANONICAL_V6watermarkV6MsgIdBILI_ROLLOUT_WATERMARK_V6_MSG_IDPlus one env-only switch (deliberately NOT a rollout knob):
BILI_PF_V6_REWRITE_STRICT=1credits rewrite-class misses toward the shared breaker instead of the default uncredited self-heal.Why the existing surface couldn't express this: three independently greyscaleable layers must nest under ONE parent cohort — each layer has to be shippable and revertible without touching the other two, which no pre-existing single-feature knob provides. Compatibility story: every key defaults OFF ⇒ byte-for-byte pre-PR behavior until a parent cohort plus a child percentage both come up; the only delta reaching current users today rides the parent's own 5% baseline (v5 ride, V1).
Review round 1 (2026-11-04, comment #6620012603)
e9fa08bc: B1 world-origin hydration BLOCKER (+ two CONTENT-level repeated-restart regressions), S2 archive cascade sweep, S3 read-side fault degrade-to-legacy, S4 checkpoint checksum now verified, S5 MySQL/replica soft-plane capability gate.1787db4c(round-1 remainder, on top of4169c694): the V1 ride lifetime cap (above); M1 privacy redline — plugin-lane nidv record buckets are now stored DIGESTED (n:<sha1-8>) like every other bucket in the module, so the raw host-native conversation id no longer reaches the ext store as a key (test read-sites follow the new bucket form; record-content assertions unchanged); the V2 seam-gate code comment pointing at Snapshot-render seam: signature-only deep-duplicate pair passes the seam gate and silently truncates recent history #5632.4169c694: v5-lane R7 sess-change trio + Test Blind Spot Retrospective: Reinforcing Negative Assertion Discipline, Tightening Fake Upstream Strictness, Thin Inst | 测试盲区复盘:补反向断言纪律、假上游严格化、薄安装冒烟、形态矩阵与生命周期覆盖 #3617 bidirectionality matrix (V1a/V3 — the live lane previously had no positive-direction lines of its own), loose-mode compaction stays uncredited (V4), infra-fault degrade with zero breaker credit even under strict (S3/V5), forged-sid burst contract frozen as documented behavior (V6).1787db4c(reviewer-named fix-path option — owner may veto/tune at merge): the ride lifetime cap is implemented — consecutive id-mode drift turns count in persisted stated(reset by a clean head or any reseeding miss); afterPF_V5_DRIFT_RIDE_CAP=5one UNcredited legacy turn reseeds from real bytes (self-heal, never breaker fuel; the diagnostic warn stream is bounded to one line per cap-many turns). This is the reviewer's own fix-path option (cap OR M3-first OR explicit sign-off); M3 can still land later as the smarter long-term mechanism — the cap stays as the safety bound either way. The cap value is a single constant; tuning or reverting is a one-line + test adjustment.snapshot-render.ts(1787db4c) so the guard is discoverable in code.Behavior-change ledger (old → new, for the merge decision)
1787db4c)1787db4c)n:<sha1-8>(module redline: id values never persist). Feature default OFF ⇒ no live records to migratee9fa08bc)e9fa08bc)e9fa08bc){t}only and was never verified anywheree9fa08bc)e9fa08bc)TEMP-ON status: REVERTED (
6862785d). The temporary 100% verification push (2bdb33fa) has been rolled back — all three rollout keys restored to their shipped grey defaults (pluginFullResendWatermark {percent:5},watermarkCanonicalV6 {percent:0, base},watermarkV6MsgId {percent:0, base}), the inline TEMP-ON markers insrc/rollout.tsremoved, the two forced test pins restored, and doc/code re-aligned to "5 (experimental grey)". The end-to-end full-stack exercise it provided on remote CI (incl. windows lanes) is recorded in the round-1 verification below.Verification updates (post review round 1)
0a9f14e8(current head): TEMP-ON reverted (6862785d) +origin/master(23a826d3, incl. bun:sqlite storage work and kernel 0.0.180 / [Bug] Config save rejects valid mitm:// provider lanes as "invalid recipe on named provider" | 整文件保存把 mitm:// 路由误判为命名配方 #5620) merged in as a single merge commit. On the merged head:tsc --noEmitclean, and the targeted fastpath family +native-identity-observe+storage-unifiedsuites green together (190 run / 189 pass / 1 skip) — including the revertedwm-fastpath-intro-noticebaked-plan-at-5% pin and thestorage-unifiedbun:sqlite/pf coexistence hotspot.4169c694:tsc --noEmitclean; fastpath familytests/fastpath-pf-msgid.test.ts(16 cases incl. B1 repeated-boot hydration across L1 folds and untrusted-checkpoint self-heal),tests/fastpath.test.ts(21 cases incl. the new v5-lane R7/matrix lines),fastpath-v6-canonical(12),issue5403+wm-fastpath-intro-notice— all green together (54/54 in one run).1787db4c:tsc --noEmitclean; fastpath family +native-identity-observe+ archive suites green together (185 pass / 0 fail acrossfastpath,fastpath-pf-msgid,fastpath-v6-canonical,lane-decisions-matrix,native-identity-observe,storage-archive-gc,issue4665-archive-restore-identity,compaction-archive). Negative control verified for the V1 cap: disabling the cap makes its regression pin fail (and the4169c694matrix row stays green WITH the cap).e9fa08bc: e2e-local-registry / e2e-native-dsh / e2e-codex-persona / e2e-fake-codex success; remaining lanes completing at time of writing.Base-staleness note (RESOLVED):
origin/master(23a826d3, incl. bun:sqlite storage work) has been merged into this branch at0a9f14e8. The overlapping hot files (src/persist.ts,src/server/handle.ts,tests/storage-unified.test.ts,src/storage/shadow.ts, CONFIGURATION.md / .zh-CN, LIFECYCLE.md) were resolved semantically (clean auto-merge, no textual conflicts) and verified on the merged head:tsc --noEmitclean + targeted fastpath/storage suites green. The pf / native-identity additions coexist with master's bun:sqlite plane without conflict.