Skip to content

clog P1: pure core + naive engine + WAL (M0+M1) - #3

Open
hhff wants to merge 28 commits into
mainfrom
clog-p1
Open

clog P1: pure core + naive engine + WAL (M0+M1)#3
hhff wants to merge 28 commits into
mainfrom
clog-p1

Conversation

@hhff

@hhff hhff commented Aug 16, 2026

Copy link
Copy Markdown
Member

What this is

The first milestone pair (M0+M1) of clog — an orientation engine for agentic systems, per docs/clog-spec-v1.md as amended by docs/superpowers/specs/2026-08-15-clog-build-design.md. Hosts write structured claims; clog maintains materialized views incrementally and renders a token-budgeted situation document per scope. Reads are snapshot clones — zero retrieval work at turn time. Clog never calls a model and carries no host semantics.

Integration shape (the point of this milestone): install the crate, wire connectors to observe(), have the agent pull situation(scope) each turn. See examples/clog/README.md and cargo run -p clog --example quickstart.

Contents

  • examples/clog library crate (~8k lines): public API (open / observe / retract / revoke_observer / situation / select / merge_entities / advance), pure semantic core (validation, scoring, belief resolution, alias map, rules-tier classifier, deterministic renderer + budgeter), naive oracle engine over imbl snapshot maps, single-writer actor with ArcSwap publication, postcard+CRC32 WAL with torn-tail quarantine and write-ahead ordering.
  • Docs: spec, build design (all decisions + amendments: postcard over bincode, imbl, WAL-records-effects, Batch.as_of, masked change detection, alias-map-as-view, canonicalized boosts), resolved fold F-checklist (docs/fold-answers.md), P1 implementation plan.

Test evidence

cargo test -p clog: 61 unit + 27 integration + 1 golden + 6 property + recovery + 4 doctests, all green; --features test-crash adds the R1 subprocess crash-point suite (abort after WAL fsync at every batch, reopen equals fresh-fed prefix byte-for-byte). clippy --all-targets -D warnings and cargo fmt --check clean. Spec invariants are enforced by named tests (INV-1/3/4/5/6/8/9/10/11 in P1 scope); the G1 agency goldens freeze the §5.8 rendering contract (independently hand-recomputed scores).

Review process

Built task-by-task with a fresh implementer + independent reviewer per task (5 in-loop fix rounds), then an adversarial whole-branch review: 6 finder lenses, each Critical/Important finding judged by 2 independent refuters. 10 findings survived (3 Critical: WAL partial-frame stranding, scoring_clamp overflow panic, entity-name render injection) and were fixed + re-reviewed in a single verified wave; 10 were refuted; 15 minors triaged (fixed or recorded).

Deferred (by design, P2+)

set_focus at runtime, ticks/decay-bucket recomputation, watches/wakes, recall/kNN + eval harness (garden3d corpus), fold engine adapter, WAL snapshot files + format versioning, single-writer directory locking. Known caveats are documented in the crate README.

🤖 Generated with Claude Code

hhff and others added 28 commits August 15, 2026 17:32
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Implements the §5.3 total order over a subject's claims: later
occurred_at wins, tie-broken by reliability, then credibility, then
recorded_at, then claim_key lexicographically. resolve() applies the
credibility floor (only claims at or above it are eligible to win)
with an only-claim exception when a lone member is floored.
Implements the free, deterministic rules tier of the cascade classifier
(spec §5.6): compile() pre-compiles a KindTaxonomy's regexes into a
RuleSet, classify() returns the first matching kind in config order
with confidence 1.0 and JudgeSource::Rule. A rule matches when any of
its matchers match (any_of). Invalid rule regexes are rejected at
compile() with ClogError::Corrupt, documented as a deliberate v1 fit
(config-as-state) pending a possible dedicated InvalidConfig variant.
Adds the render module: a §5.8 grammar parser (SlotName, Segment,
Template, parse) that scans for %{...} slots and rejects unknown
names/keys, unterminated slots, empty slots, and non-usize limit
values as ClogError::TemplateError without ever panicking; the
byte-for-byte DEFAULT_TEMPLATE; and rfc3339_utc via Howard Hinnant's
civil-from-days algorithm. Renderer body itself lands in the next
task.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Implements the render/mod.rs slot renderer + budgeter (spec §5.8):
headline() normalization, the UrgentItem/LoopItem/EntityItem/ChangeItem/
SlotInputs engine-to-renderer contract, and render() itself, which
substitutes each %{...} slot's formatted item lines into the template and
budgets down to a char count by dropping whole items from the end of
slots in reverse priority order (changes, entities, open_loops, urgent),
appending/updating a "… (n more)" marker per truncated slot. Per-slot
limit= caps items before budgeting.

Corrected the u_tmpl_2 test's tight-budget derivation from full.len() - 1
(bytes) to full.chars().count() - 1: the render contains three U+00B7
middle dots, each 2 bytes but 1 char, so a byte-derived budget was never
tight enough to trigger truncation against the spec-correct char-counting
budgeter, which would have made the test's own assertions vacuous. The
assertions themselves are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… views

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ed allow

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…AS-3, INV-6)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds wal.rs: an append-only, length-prefixed, CRC32-checked frame log
that is clog's source of truth (spec §6.3, INV-11). open_dir replays
every existing frame in order; a torn or corrupt tail (short read,
absurd length, CRC mismatch, or a CRC-valid record that still fails to
postcard-decode) is never applied, quarantined to wal.corrupt, and
truncated from the log so reopening afterwards is clean (R2). append
writes one frame and fsyncs per FsyncPolicy.
…1/3/4/5/9)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… masked change detection

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…at consolidation

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds tests/props.rs per Task 16 brief, with two rulings applied:
- norm() strips the whole header line + changes-since-last-brief
  section (not just the rev marker), matching tests/api.rs's
  established INV-3 normalization.
- P6's generated claims each carry a fixed entity (p:shared-ent) so
  believed winners are readable via select(View::EntityState), which
  is keyed by entity and would otherwise see none of these claims.

All 6 properties pass at cases=64 and at PROPTEST_CASES=256 with no
counterexamples.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ten-claim, three-client, two-scope fixture (spec §11.4) exercising belief
resolution's freshness-over-trust limitation and its healing on retraction,
the entities slot dropping subject-less entities, and merge_entities
re-keying entity grouping. Adds one auxiliary `fyi` classification rule
beyond the brief's four so cash-and-collections' fyi=0.5 weight is actually
exercised by a live claim; documented in task-17-report.md along with the
hand/script-verified score tables for every checkpoint.
R1 re-executes the recovery test binary as a child that aborts in the
test-crash hook immediately after each of the script's four WAL appends
(post-fsync, pre-apply), then reopens that directory and asserts the
recovered world is byte-identical to a fresh instance fed exactly the
durable prefix -- same text, same scope rev, same as_of -- and that the
survivor still writes, taking rev N+1.

R3 pins the rebuild_on_open contract: a rebuild equals a normal open.
Trivially true in P1 (every open is already a full WAL replay), and
load-bearing the moment an engine-state cache lands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…exit checklist green

Rewrites examples/clog/README.md for the host-integration reader (what clog
is, the observe/situation loop, P1 API table, quickstart, config knobs,
documented caveats, status/roadmap), adds a runnable
examples/clog/examples/quickstart.rs mirrored in the README, and promotes
Clog's top-level doctest from no_run to an executing example. Formats the
clog crate for the first time (cargo fmt -p clog, no other workspace crate
touched) and re-verifies clippy/tests pass after reformatting.

Exit checklist green: cargo test -p clog; cargo test -p clog --features
test-crash (R1 included); cargo doc -p clog --no-deps (warning-free);
cargo clippy -p clog --all-targets [--features test-crash] -- -D warnings;
cargo fmt -p clog --check.
Critical:
- wal: a failed append could strand a partial frame mid-log; the next
  successful append landed after it and reopen's CRC scan then quarantined
  and truncated everything from the tear onward, silently discarding acked
  batches. `append` now records the pre-write file length and truncates back
  to it on ANY error; if the truncation itself fails the Wal is poisoned and
  refuses further appends rather than compounding the damage.
- validate: `scoring_clamp` overflowed `now + 24h` when `now` was within a
  day of u64::MAX (reachable - the manual clock saturates there), panicking
  the writer under overflow checks. Now saturating.
- actor: an entity's registry display name reached the rendered document
  verbatim, so a newline in `EntityRef.name` fabricated document lines. It
  now goes through `render::headline` like every other host string.

Important:
- actor/types: INV-5's duplicate skip used `Claim: PartialEq`, whose
  `EntityRef` equality ignores `name`, so a re-observe changing only a
  display name was swallowed (no commit, registry never updated). New
  `claims_identical` compares the name lists alongside, leaving EntityRef's
  identity semantics (load-bearing for every index) untouched.
- actor: merge claims stamped occurred_at/observed_at from a clock that
  reads 0 on a fresh Manual instance, failing spec 10 validation. Now stamps
  `now.max(1)`.
- types/kinds: `Matcher::BodyContains` now carries its own case-sensitivity
  flag (spec 5.6) instead of hard-coding case-insensitivity.
- types: Focus boost docs said "additive"; they are multiplicative (5.4).
  View::Live and Config.rebuild_on_open docs corrected.
- validate: reject `subject_key: Some("")`/whitespace-only, and
  `decay_buckets_per_half_life == 0` (NaN scores) at open.
- validate: focus *value* errors are now Corrupt { "config: ..." } rather
  than InvalidFilter, matching the bad-regex ruling; UnknownKind unchanged.
- actor: `mask_header` string-replace replaced by a structural body/scope
  comparison, removing the adversarial-scope-name fragility.

Tests: WAL rollback + poison, u64::MAX clamp, entity-name injection,
display-name re-observe, fresh-clock merge, case-sensitive BodyContains,
belief tiers 2/3 in isolation, a real budget-cap + drop-order render test,
min_score on Urgent, and config rejection at open. Goldens unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Boots the agency fixture frozen mid-story (belief competition live,
merge pending), re-prints the situation after every command; crash
command demos WAL recovery. Store persists at ./clog-repl-data,
idempotent reseeding via INV-5.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant