feat(engine): replace the ablation scoring mode with a checked, digest-bound one - #1356
monotophic wants to merge 5 commits into
Conversation
|
Authored by Claude Fable 5.1 in Claude Code, analysis in partnership with @monotophic Merged current @JustVugg similar steering request to #1102, #1353, and #1355, this PR is part of the set to make the full checkpoint-quality GLM 5.2/5.3 containers work and to establish the OpenAI API instrumentation needed for container quality studies. Let me know if this PR is off target or otherwise not likely to merge. Thanks for all you do, this project is awesome :) |
|
Reviewed, and I want to answer your own question rather than pretend the review is about the code. The code is careful and the defect it replaces is real. I read the old You ask whether I want the old mode back. The answer is that this is the one place in the set that breaks the project's convention, and you say so plainly rather than hiding it: no new environment variable, but So the question back to you is narrow: is Two smaller things either way. 801 lines of And the new loader materialises the whole manifest in RAM where the old one streamed a line at a time. |
Add tools/check_ablate_evidence.py, a standalone validator for the ABLATE_OUT evidence stream the engine's ablation-sweep mode writes: it accepts only a complete, self-binding artifact and refuses everything else -- a truncated or replayed record stream, a header that does not bind the manifest or the external config.json it names, any field with the wrong type, range or key set at any of the four record kinds (header, item header, target row, terminal completion), and a target row whose fields contradict one another in a way the producer can never emit. Framing is checked before content: NaN/Infinity JSON constants, duplicate object keys, non-ASCII text and malformed JSON are all rejected up front. The manifest proof and the header's config identity are cross-checked against the actual config.json and manifest named in the header, so evidence cannot be validated against the wrong run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…hecked ablation scoring mode Add evidence_digest.h (SHA-256 identity of the loaded config.json and of a manifest, bound into the ABLATE_OUT header so evidence can never be checked against the wrong run) and a logit-row status layer in sample.h (LogprobStatus / LogprobRow / logprob_row_checked / logprob_from_row_checked) that classifies a row's reduction -- FINITE, NAN, POS_INF, NEG_INF, ALL_NONFINITE, FINITE_OVERFLOW, or INVALID -- before any log-probability arithmetic is trusted, so a malformed row fails closed with a named status instead of silently producing a NaN a reader cannot distinguish from a real value. The per-target subtraction promotes the logit to double before subtracting the row's own double logZ, so its rounding no longer depends on where in the pipeline the widening happens. Replace ABLATE_SCORE's ablation-sweep mode with a checked version built on that status layer: every manifest field is validated against the loaded config before anything runs, a manifest that does not fit the documented format is refused whole with the offending line named on stderr, and the per-item evidence stream records each target's status alongside its negative log-likelihood, logits, margin, argmax and top-k. The evidence writer takes a plain FILE* (opened O_EXCL, no existing path overwritten) -- it only ever had one backend, so the prior AblateWriter vtable and the AblateModeRunFn dispatch indirection around it are both gone in favor of calling the writer functions directly. Coverage: test_ablate_mode.c drives the mode end-to-end against a small fixture config (vocab=64), including the manifest parser's boundary cases (token id exactly at the vocabulary bound, an ERANGE-triggering field) found by a mutation sweep; test_ablate_mode_gate.py checks the mode is reachable only through its documented environment gate; test_logprob_status.c pins logprob_row_checked's and logprob_from_row_checked's classification of every row shape in isolation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
logit_topk_select measured 2.01x slower than dev's equivalent top-k selection at V=151552 (real GLM-5.2 vocab), topk=32. Cause: after the first k iterations fill all k slots, every one of the remaining V-k iterations still ran an O(k) linear scan of tk_id[] asking whether an empty slot exists, an answer that can only ever be 'no' from that point on -- loop-invariant work paid on almost every iteration for a result invariant after the first k. Fixed with a fill counter: the first k iterations fill slots 0..k-1 in that exact order (as the old empty-slot scan also did, since it always returns the lowest-indexed empty slot), so 'is there an empty slot' is 'have fewer than k been filled', O(1) instead of O(k). The minimum-slot scan for i>=k is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… imports check_ablate_evidence.py is reached by nobody in coli's own import graph -- it is a person-invoked verifier, the same position k3_tokenizer.py has always been in -- so pack_python's import walk cannot carry it into the release archive on its own. Bring in this branch's release.yml hunks: copy the tool into dist/tools, assert it shipped, assert it parses. test -f and ast.parse both only look at the one file, and only at its syntax -- neither executes it, so neither catches a real import-time failure. Add a fourth gate that actually imports the packaged module from the archive root, never executes it, so a real import failure fails the job instead of shipping silently broken. test_pack_python.py's HumanOnlyToolsShipExplicitly pins both halves of this on the real tree: the import walk does not reach check_ablate_evidence.py (if it ever does, the explicit copy became redundant), and release.yml both copies and gates it, the same way it already does for k3_tokenizer.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…allocations
Re-review found that documenting the prefill buffer this cap did not
bound undersold the problem twice. First, docs/ENVIRONMENT.md asserted
the hazard closed ('so that a corrupt manifest cannot ask the loader
for an arbitrary allocation') while this file's own comment said the
opposite; a doc our own comment refutes is worse than the narrower true
statement it replaced. Second, measurement found THREE unbounded
allocations behind this one constant, not one: kv_alloc's per-layer KV
buffers (177.75 GiB at the old 2^20 cap, GLM-5.2 shape), the prefill
buffer x (24.00 GiB, the one originally named), and this loop's own
resident manifest bookkeeping (240 B/item, unbounded in item count).
kv_alloc is 7.41x the prefill buffer, not the smaller of the two.
Given the doc already claims the hazard is closed, the honest fix is
to make the claim true rather than narrow it further. The bound: a
manifest item's T is now checked against min(CTX, ABLATE_MAX_ITEM_TOKENS)
via ablate_item_token_limit(), not the bare 2^20 constant. CTX
('Maximum context length (tokens) the KV cache is sized for',
docs/ENVIRONMENT.md) is an existing engine limit already read by
kv_pool_bytes and expert_avail for exactly this kind of sizing -- no
new constant, no new environment variable. An item longer than the
context this engine is configured to hold cannot be processed
regardless of what a manifest asks for. The check still runs before
the item can contribute to any of the three allocations; the backstop
constant remains in case CTX itself is set to something absurd.
Two tests, both required so the bound is proven exact, not merely
strict: a manifest at cap+1 (4097 under the CTX=4096 default) is
refused by name; a manifest at the real maximum (4096) loads and runs
to completion -- without this second case, a bound that refused
everything would also pass the first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Split reused verbatim from upstream/p0-engine-preamble (b6070718), verified there by 18,093 old-vs-new behavioural comparisons with zero divergences. Keeps PreambleError, parse_engine_banner, parse_engine_loaded, parse_engine_preamble and their module constants. Removes ManifestFormError and canonical_manifest_bytes, which travel to JustVugg#1356 with their only consumer (check_ablate_evidence.py, removed from this PR next). The module docstring no longer describes the absent half. This transiently breaks check_ablate_evidence.py's import of the removed names; the next commit removes that module from this PR. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
check_ablate_evidence.py, its test module, and the three release.yml hunks that copy and gate it (the cp into dist/tools, the test -f presence gate, the ast.parse packaged-file check) move to JustVugg#1356, which owns their writer (canonical_manifest_bytes/ManifestFormError, removed from engine_evidence.py the previous commit). test_eval_glm.py's shared byte-limit test drops its ABLATE-side assertions (module gone) and keeps the EVAL-side ones unchanged; test_pack_python.py's HumanOnlyToolsShipExplicitly class is removed verbatim rather than edited, since check_ablate_evidence.py was one of its two pinned entries -- it is carried to colibri_lab/dispatch/2026-09-17-program-d-r2/CARRIED_TO_P2_test_pack_python_class.txt for JustVugg#1356 to restore alongside its own copy of the checker. test_engine_evidence_is_needed_by_the_real_tree, this branch's own addition to that file, is untouched. This is a forward commit that deletes -- intended, not a mistake. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
cc80928 to
14f299c
Compare
|
Set-level explanation and the full traceability matrix: #1355 (comment) The diff is smaller: 15 files and +5,449 → 12 files and +3,706, a 1,743-line reduction.
|
| category | lines | share of 838 |
|---|---|---|
manifest parse / validation / digest (ablate_manifest_load is 171 of these, c/colibri.c:8178-8348) |
294 | 35.1% |
| writer / output path (gross added) | 171 | 20.4% |
| cap / refusal path — new in this revision, did not exist when you reviewed | 56 | 6.7% |
test-only, #ifdef COLI_TEST_ABLATE_ADAPTERS, absent from a normal build |
35 | 4.2% |
| remainder — scoring helpers, error paths, dispatch | 282 | 33.7% |
Categories are exact integer counts and sum to 838. Comments are 151 of the 838 (18%), spread across all
categories rather than concentrated in one.
The bulk is validation the old mode did not do — which we see by reading dev: strtol failure
meant continue, an out-of-range cell count was silently clamped, a bad item printed to stderr and continued, and
the whole thing returned 0 regardless.
The writer collapse — done, and one part of your ask that is NOT done
AblateWriter is gone. The four-function vtable you objected to — vprintf_fn, puts_fn, flush_fn,
close_fn — no longer exists: 21 references at the head you reviewed, 0 now. The writer is a plain FILE *
and the C tests drive it through temporary files they create and clean up themselves (PID-suffixed, opened with fopen) — the substance of what you suggested, though not literally tmpfile(3).
But you named two things, and the second one survives. AblateOutputRun still carries two function pointers,
open_fn and body_fn, and they are not compile-gated — they are on the production path
(c/colibri.c:8443-8466). They are the seam that lets the C tests run the parser with no model at all. We are
flagging it rather than letting you find it: the indirection is reduced, not eliminated. If you want that
collapsed too, say so and it is a small follow-up.
We also removed something you did not name: AblateModeRunFn and its one-entry ablate_mode_dispatch table —
a function-pointer typedef with exactly one implementation (2 references each, now 0). What remains is
ablate_model_mode_run, an 8-line wrapper that null-checks the path and calls run_ablate_score directly.
Your asks
| you asked | disposition | where it lives now |
|---|---|---|
"is ABLATE_SCORE used by anyone outside your own tree?" |
done — searched, nothing found; replacement stands per your own condition | above |
| "Could the writer collapse to a plain FILE pointer with the tests driving a tmpfile?" | done for AblateWriter, NOT for AblateOutputRun — the AblateWriter vtable is gone (21 references → 0) and the writer is a plain FILE *, driven by temp files the tests create and remove (not literally tmpfile(3)). AblateOutputRun, which you also named, still carries two function pointers on the production path; see below. Bite re-checked: 4 distinct mutations against test_ablate_mode.c, each a real failure |
c/colibri.c |
| "I would make the prose match what the cap actually bounds" | done, and then some — see the section below | c/colibri.c, docs/ENVIRONMENT.md |
The cap: we went past the prose fix, and here is why
You asked for prose only. We started there, then measured, and found the prose was understating it: there are
three unbounded allocations reachable from a manifest, not one, and the largest is kv_alloc at 177.75 GiB
at the cap — 7.4× the prefill buffer you named.
At that point "make the comment honest" had two possible forms: admit our own doc is false, or make it true. Our doc
in this PR says the cap exists "so that a corrupt manifest cannot ask the loader for an arbitrary allocation", and
ABLATE_MAX_ITEM_TOKENS is new in this PR — so that one is ours.
The cap now bounds what its name claims, derived from a limit the engine already has rather than a new constant:
min(CTX, ABLATE_MAX_ITEM_TOKENS), checked ahead of all three allocations. A manifest over it is refused by name
before anything is allocated; a manifest at the real maximum loads. No new environment variable.
Disclosed, not closed: this bounds item length, not item count. A long manifest still costs ~240 B/item,
which is ordinary loader behaviour proportional to input. The hazard was a short line demanding 178 GiB, and that
axis is shut.
Your 21 GB figure reproduces at hidden 5,120; at GLM-5.2's actual hidden of 6,144 the prefill buffer is
24.00 GiB (2^20 * 6144 * 4 B), against 177.75 GiB for the KV allocation. Both are worked out in full in
the comment beside the cap rather than left as bare constants, and docs/ENVIRONMENT.md now states the derived
bound and names all three allocations it covers.
Details — origin accounting
maintainer-requested: ABLATE_SCORE search; writer collapse; cap prose.
re-review-found, disclosed: the three-allocation measurement and the derived bound; removal of the
AblateModeRunFn typedef and its one-entry ablate_mode_dispatch table — a function pointer with exactly one
implementation, wrapping what is now a direct call — which you did not name and we removed anyway
(ablate_model_mode_run remains, as a plain static that null-checks the path and calls run_ablate_score); a 2.01× logit_topk_select regression fixed with
byte-identical output; and a real import check (python3 -c "import tools.check_ablate_evidence") added to the
release-archive gate alongside the existing test -f and ast.parse, not replacing them. That gate is ours,
added in this PR, and neither of the original two executes anything, so neither catches an import-time failure. The
archive is verified correct today; the gate now asserts that rather than implying it.
#1356 and #1102 both add entries to c/Makefile and will conflict there — only there; the engine sources merge
cleanly. Whichever lands second, we re-merge dev and push within a day.
Authored by Opus 5 in Claude Code, analysis in partnership with @Monotophic
2 participants
Authored by Opus 5 in Claude Code, analysis in partnership with @monotophic
Revised 2026-09-17 in response to review: 15 files and +5,449 down to 12 files and +3,706, a 1,743-line
reduction. The
AblateWritervtable is collapsed to a plainFILE *as asked, the cap now bounds what its name claims, andtools/check_ablate_evidence.pywith its release-archive gate have moved here from #1355, where they had no writer.ABLATE_SCORE=<manifest>replaces the decode with a teacher-forced sweep: for each item it configures the ablation cells, runs one prefill, and reads the final logits at every target position. That mode already exists ondev.What it cannot do is produce evidence anyone can check. It accepts whatever a permissive whitespace scan can make of a line, skips lines it cannot parse and carries on, silently truncates an over-long cell list, and writes
records naming neither the config nor the manifest they came from. A partial run looks exactly like a complete one
— it exits 0 after writing nothing but a header.
This PR replaces that mode's implementation, and the change is visible to anyone parsing its output. The
artifact's schema goes from
coli-ablate/1tocoli-ablate/2, the mode's exit status is now its own resultinstead of a constant 0, and inputs the old mode would have partly executed are now refused as a whole.
docs/ENVIRONMENT.mddescribes the mode as it now behaves.On replacing the mode rather than adding an alternative — the question is answered. The review asked whether
ABLATE_SCOREis used by anyone outside our tree, and pre-committed: if not, replacing it is right. We conducteda search of the project's public GitHub and found nothing — all upstream references, 53 issues and discussions,
and public code search. That is the strongest claim the evidence supports; it is not a claim of no use anywhere.
One fact offered without a recommendation: the mode was written by @jeswr, still an active contributor here.
We have not contacted them — that is the maintainer's call.
What the new mode gives you: the manifest is a defined format with an exact field count and single-space
separators, so the same content always produces the same digest; every field is bounded against the loaded config
before anything runs; anything outside that is refused as a whole, naming the record at fault, including a
duplicate item id. A CRLF terminator and a missing final newline are both accepted and normalised — a host editor
produces them without meaning to — and all three framings of the same content bind the same digest. The artifact
gains the digests of the config and the manifest, the item and target counts it expects, and a terminal record
carrying the counts actually completed, so a truncated file is detectable. Positions whose logits are not finite
are refused by name instead of being written out as unusable numbers, the top-k list holds real vocabulary entries
instead of padding a fixed width with sentinels, and the output path is created exclusively, never truncated.
Every refusal says why on stderr before the mode returns non-zero.
The first commit adds the two small foundations that make the above checkable: a classified reduction of a logit
row (
logprob_row_checked), which reports why a row was unusable rather than only that it was; and a shortself-contained SHA-256 (
evidence_digest.h) so a mode can name the exact bytes it consumed without the enginetaking on a dependency it otherwise never needs. The plain sampling path is untouched, and the new path is pinned
to agree with it on every well-behaved row.
The reported numbers are unchanged. The negative log-likelihood and the partition function are computed exactly
as before, in double, and are only printed at full precision now — a reader comparing old and new files sees more
digits of the same value.
The cap: we went past the prose fix you asked for
The review asked us to make the prose match what
ABLATE_MAX_ITEM_TOKENSactually bounds. We started there, thenmeasured, and found the prose was understating it: three unbounded allocations are reachable from a manifest,
not one, and the largest is
kv_allocat 177.75 GiB at the cap — 7.4× the prefill buffer the review named.At that point "make the comment honest" had two forms: admit our own doc is false, or make it true. The doc in this
PR says the cap exists "so that a corrupt manifest cannot ask the loader for an arbitrary allocation", and
ABLATE_MAX_ITEM_TOKENSis new in this PR — so the variable whose comment lied is ours.The cap now bounds what its name claims, derived from a limit the engine already has rather than a new
constant:
ablate_item_token_limit() = min(CTX, ABLATE_MAX_ITEM_TOKENS), checked ahead of all three allocations.A manifest over it is refused by name before anything is allocated; a manifest at the real maximum loads. No new
constant, no new environment variable.
Disclosed, not closed: this bounds item length, not item count. A long manifest still costs ~240 B/item,
which is ordinary loader behaviour proportional to input. The hazard was a short line demanding 178 GiB, and that
axis is shut.
The 21 GB figure in the review reproduces at
hidden5,120; at GLM-5.2's actualhiddenof 6,144 the prefillbuffer is 24.00 GiB (
2^20 * 6144 * 4 B). Both allocations are worked out in full in the comment beside thecap rather than left as bare constants.
Context. This PR is one of five independent contributions derived from a single locally-verified working tree
(the fp8 container line in #1102, the OpenAI-compatible server hardening in #1353, the evaluation harness in #1355,
this engine block, and a transcript check tool in #1357). This one carries the engine block: the ablation scoring
mode, the row-status and digest headers, the offline checker that reads the mode's output, and their tests.
The round-trip case now runs entirely inside this PR — the producer and
tools/check_ablate_evidence.pyareboth here, so nothing about it waits on another PR. It no longer carries
tools/engine_evidence.pyat all. This revision moved the manifest helpers intocheck_ablate_evidence.py, which left nothing here importing that module — so shipping it would have meant 524lines of code nothing in this PR uses. It lives in #1355, which has a real consumer, and in #1357, which imports it. The others are proposed separately, each with its own
evidence. The set-level explanation and a traceability matrix covering every request across the three PRs are in the #1355 comment: #1355 (comment).
Behavioral contract
ABLATE_SCOREis set, and no other path reaches it.min(CTX, ABLATE_MAX_ITEM_TOKENS)is refused before any of the three allocations it would otherwise size.nlland partition function are bit-for-bit whatdevcomputes.engine accepts, and all three bind one digest.
set absent.
Structural changes on the default load path. The mode's digest is computed only when
ABLATE_SCOREis set, so the load path does no new work on an ordinary run. Eight things about it are nonetheless different fromdev, andyou should not have to find them yourself:
<stdarg.h>and<inttypes.h>;#include "evidence_digest.h"— a new file whose functions are plainstatic;Cfggainedchar config_sha256[65], so everyCfgin the process grows;cfg_rootgained a third parameter;cfg_rootruns only when a caller passes a buffer;load_cfgreads one environment variable on every call to decide whether to pass that buffer — onegetenvper model load;run_score's prefix probe passesNULLfor the new parameter;sample.hgained threestatic inlinefunctions and two types; that header is compiled intoolmoe.cas well, where they are unused.Capstone matrix — one decisive artifact per claim.
devand one from the integrated tree, run against the same real GLM-5.2 container (78 layers, 256 routed experts, hidden 6,144, int3/g64), same prompt, temperature 0, KV cache deleted before each arm so neither could inherit the other's prefill: the same 24 generated tokens, id for id, on CPU and again on CUDA. The CUDA arms are asserted to have actually used the GPU —resident set: 625 tensors, 10.17 GB VRAM— because an earlier pass withCUDA_EXPERT_GBat its default of 0 placed nothing in VRAM and silently ran on the CPUtests/test_ablate_mode_gate.pyreads the engine source and asserts exactly one product entry into the mode plus one compile-gated adapter entry, and thatrun_ablate_scorehas exactly one caller inside the mode; it fails if a second call site appearsnllis stilldev's double-precision reductiontests/test_ablate_mode.cdrives a row through the real emitter whose logits are chosen so a single-precision intermediate differs, and compares the emitted text with==against an expectation computed in the test fromdev's formula; substituting the float intermediate fails itconfig_sha256really comes from the load pathconfig.json, callsload_cfg, and compares the field against an independently computed digest and a pinned literal; it also asserts the field is left unset when the variable is not settests/test_ablate_mode.cand in this PR's Python suite, and reproduces outside both:printf 'coli-ablate-manifest/2\n0 3 2 0 0 1 2 3\n' | shasum -a 256→c63a48c375b14ca60f26c7e3c5dd36b5929ffaf669a45511c93deee6e8bbd5edABLATE_OUTagainst the manifest and the config, and passedtests/test_ablate_mode.creports 79 named checks at runtime (ok:lines from a real run — a static count ofCHECK(call sites gives 54, because several sit in loops) with no model and no weights — the loader, the writer, the three accepted framings, each named refusal, and the dispatch contract;tests/test_logprob_status.creports 138 the same way, including published SHA-256 vectors at the block boundary and both padding branches. BothPASSED (0 failures)at the head being pushedtests/test_ablate_mode.csection "the per-item token limit is derived from CTX, not a bare constant": cap+1 (4097 tokens) is refused, naming the CTX=4096-derived limit, and a manifest at the real CTX-derived maximum (4096) loads and runs. The refusal is asserted to name the limit, not merely to occurtest -f,ast.parseand the newpython3 -c "import tools.check_ablate_evidence"all pass. Stated precisely, because the distinction matters: at this headcheck_ablate_evidence.pyhas no local imports at all — the manifest helpers moved into it during this revision — so the import check currently has no dependency to catch. It is kept because it is one line and asserts importability for any dependency the tool gains later. It is hardening a gate that could not detect importability, not a fix for a live breakWhat changed since the reviewed head, how the fleet comparison was set up, and origin accounting
What changed since
cc80928e, the head reviewed on 2026-09-17AblateWritervtable collapsed to a plainFILE *, tests driving temp files they create and remove (not literallytmpfile(3)).AblateOutputRun, also named in the review, is NOT collapsed — it keeps two production-path function pointersAblateModeRunFntypedef and its one-entryablate_mode_dispatchtable removed — a function pointer with exactly one implementation, wrapping what is now a direct callablate_model_mode_runremains as a plain static that null-checks the path and callsrun_ablate_scoretools/check_ablate_evidence.py, its test, and therelease.ymlgate moved here from #1355coli-ablate/2mode, so they moved insteadtest -fandast.parse, not replacing themlogit_topk_selectregression fixed, output byte-identicalENVIRONMENT.md/comment contradiction correctedWhere the added
c/colibri.clines go, measured at this head (+838 / −74). Categories are exact integercounts and sum to 838: manifest parse / validation / digest 294 (35.1%), of which
ablate_manifest_loadis171 (
c/colibri.c:8178-8348); writer / output path 171 gross (20.4%); the cap / refusal path 56 (6.7%),which is new in this revision; test-only
#ifdef COLI_TEST_ABLATE_ADAPTERSlines 35 (4.2%), absent from anormal build; remainder — scoring helpers, error paths, dispatch — 282 (33.7%). Comments are 151 of the 838
(18%), spread across categories. The old
run_ablate_scoreondevis 79 lines(
origin/dev c/colibri.c:7929-8007); "about 88" is the diff hunk's old-side span.The writer collapse, stated precisely — including the part that is not done.
AblateWriterand itsfour-function vtable (
vprintf_fn,puts_fn,flush_fn,close_fn) are gone: 21 references at the reviewedhead, 0 here. The writer is a plain
FILE *, driven in the tests by temporary files they create and clean up themselves (PID-suffixed, opened withfopen) rather than bytmpfile(3)itself.AblateOutputRunsurvives withtwo function pointers (
open_fn,body_fn) which are not compile-gated and sit on the production path(
c/colibri.c:8443-8466) — they are the seam that lets the C tests run the parser with no model. The review namedboth; only one is eliminated, and collapsing the second is offered as a follow-up rather than claimed as done.
c/Makefileand #1102 — the precise statement. #1356 and #1102 both add entries toc/Makefileand willconflict there — only there; the engine sources merge cleanly, which we verified by merging them in both orders.
Whichever lands second, we re-merge
devand push within a day. Separately, if #1102 lands first,quant.hgainsfp8_format.hand these two rules should list it too, exactly as their sibling rules do.No shared file remains in this PR.
tools/engine_evidence.pyandtests/test_engine_evidence.pywere droppedfrom it entirely once this revision left it with no importer of that module; they stay in #1355 and #1357, which do
import it. Any merge order still works — all six orderings of the three were built in a scratch worktree and
produce one identical tree with zero conflicts, re-verified after the drop.
Origin accounting for the original content is unchanged from the reviewed head: the mode's implementation, the two
headers and their tests are new content re-expressed onto current
dev; one behaviour ofdev's is deliberatelysuperseded; the checker's CRLF leniency was corrected during review to match
dev's contract of record; twocommitted pins (the
nllvalue and theload_cfgdigest) close gaps nothing guarded before. One commit listsevidence_digest.hamongcolibri's Makefile prerequisites, because upstream's Makefile-prerequisite test (#1284)requires every header a source includes to be declared — no behaviour change. The shared logprob helper in
c/sample.hnow promotes the logit before subtracting; no number this PR writes to disk changes, because theevidence writer computes its own double subtraction. No surrounding code was reformatted.
Durable vs current state: the mode's format, its refusals, the digest binding, the derived cap and the dispatch
guard are durable. Everything measured in this body — diffstats, suite counts and timings, the cap behaviour, the
archive gates — is measured against base
cc756a63at the head being pushed. No evidence from an earlierrevision of this PR is carried into it — the decode-identity and round-trip results above were regenerated
against the current integrated tree rather than quoted from the previous round.