Validation date: 2026-09-05. This report distinguishes executed checks from supplied but unexecuted acceptance tests.
Environment: Linux x86_64 container; Python 3.13.5, Node 22.16.0, Git 2.47.3. The intended mise Node pin is 22.19.0; that exact Node version was not installed in this environment.
| Check | Result | What this establishes |
|---|---|---|
| Python task-state and delegation tests | 25 passed | Real Git worktree isolation with a strict fake Herdr; explicit session scoping; single-start behavior; preserved uncertain launches; task limits; durable questions/answers; no overwritten duplicate questions; concurrent record writes; submitted-vs-applied distinction; unknown-schema refusal; records-only backup |
| Python setup/configuration tests | 8 passed | Valid local MCP configuration; idempotent config generation; preservation of unrelated settings; conflict refusal; valid source syntax and skill links |
| Node Mesh command tests | 10 passed | Correct command construction; native prompts; bounded waits using --until; busy-target refusal; uncertain handoff errors; bounded reads; curated tool surface |
| Scripted end-to-end demo | Passed | Delegation through the helper and fake Herdr into a real Git worktree; a question surviving a busy coordinator; answer/application; an actual branch commit; unchanged primary checkout; report and backup |
| Python compilation, Bash syntax, JavaScript syntax | Passed | Source parses on the versions above; this does not prove network installation or live API compatibility |
43 automated test cases passed; none were skipped in those completed runs. The Python core tests were executed in three groups of 8, 8, and 9 because whole-suite tool invocations exceeded this container's execution window. The grouped runs contain every core test. Earlier interrupted invocations are not counted as successful runs.
Test output is retained in docs/test-results.txt.
Executed on the development host for the isolated self-development checkout work (issue #3): the Python suites (60 tests), the Node Mesh tests (10), the offline demo including its new development-checkout section, and scripts/live_smoke.py against real Herdr 0.8.2 in a named sum-test-* lab session with a short /tmp lab root.
The live smoke test now registers its lab coordinator with init before prepare, uses pane wait-output --match, and opens a real workspace whose root pane starts in a development checkout.
No model, GitHub write, or user default session was involved.
Executed on the development host for issue #4 from a task checkout: the Python suites (69 tests), the Node Mesh tests (10), and the offline demo including its new release-staging section.
The new tests cover staging into an installation reached through a symlink and a path with spaces, manifest validation and tampering, a failing installer and a missing mise, four concurrent stagings of one SHA, two installations with separate release directories, a helper call paused inside release N while N+1 is staged, a release tree refusing to own state, and the stable entrypoint serving an old absolute callback before and after staging.
A real staging probe ran in a temporary lab installation with the real installer: mise install/mise which against the bundled mise.toml, a Mesh clone from the installation's local clone, npm ci, the overlay, herdr --skill, and the MCP smoke test from the staged tree, followed by doctor and the MCP smoke test through the installation entrypoint with .local/current pointed at that release.
No model, GitHub write, live installation state, or user default session was involved; mise run setup and mise run test-live were not re-run for this slice.
Executed on the development host for issue #5 from a task checkout: the Python suites (78 tests), the Node Mesh tests (10), and the offline demo including its new brief-revision section.
The new tests cover the dispatch-time version sidecar and r1, staging r2 while a handle reads the original brief (bytes, path, and approved task unchanged), duplicate regeneration writing nothing, the machine-generated change summary and verification flag, refusal to regenerate after the approved body was altered, stale/unknown/damaged/missing revision requests, orphan revision files never overwritten, explicit request and adopt with the notice slot untouched, an unsupported sidecar inspected without migration while ask/report/show keep working, the frozen helper from b1239a4 preparing, asking, answering, reporting, resolving, and showing on records that also carry new metadata, concurrent old and new writers plus regenerations, and a records-only backup that carries every revision and sidecar and restores relocated paths.
No model, GitHub write, live installation state, or user default session was involved.
Executed on the development host for issue #6 from a task checkout: the Python suites, the Node Mesh tests, and the offline demo including its new update/rollback section.
The new tests use a bare Git repository as origin and cover: resolving only revisions merged on origin/main (a local unmerged commit and a dirty checkout are refused or reported, never reset); an atomic apply while a helper call from the old runtime is paused and while a task's absolute callbacks run before and after the switch; injected failures before the rename, at the rename, and after it (the entrypoint post-check), each leaving a complete selection; a failing installer, a mismatched manifest, a candidate needing another Herdr version, a candidate not supporting a task's brief schema, a candidate helper that cannot read the records, and a concurrent update holding the lock, each leaving the current default and records intact; rollback after a new question and report were saved, with both helper generations reading the same records, rollback to the checkout, an explicit target, and an unknown target; a simulated MCP server keeping its start tree across an apply while a new entrypoint call uses the new default; and gating of apply/rollback to the coordinator pane and of every update write to the installation's own helper.
No model, GitHub write, live installation state, or user default session was involved; mise run setup and mise run test-live were not re-run for this slice.
Issue #65 was exercised in temporary installations with real Git, the stable installation CLI, a fake dependency installer, and strict fake Herdr. The focused root run passed 31 updater and recovery tests; the separate twelve-worker scenario also passed. Faults covered post-selection failure, interruption before and after pointer replacement, an unimportable candidate, a pre-recovery prior helper, unauthorized recovery, stale generations, changed checkout revisions, mismatched checkout approval, recovery-file flush failure, and an unusable initial stable fallback. The fleet scenario sent question and report callbacks while the failed candidate was selected, then verified their records after recovery together with the existing worktree and no-restart assertions. These focused results do not certify the full repository, hosted CI, authenticated harness behavior, or deployment. The original reproduction's first case accidentally reached the real dependency installer; its preserved correction note distinguishes that capture from the later fake-installer verification.
Executed on the development host for issue #7 from a task checkout: the Python suites, the Node Mesh tests, and the offline demo including its new refresh section.
The new offline tests cover: a request persisted before the delivery attempt with the fixed instruction carrying only IDs, hashes, paths, and the machine-generated summary (question and answer prose never travel); the notice slot left untouched; busy, blocked, unknown, missing-pane, stale-cwd, and refused-prompt workers left on their brief with the exact reason recorded; repeated requests coalescing to one requested event while every delivery attempt is recorded; r2 superseded by r3 and a stale receipt refused; four concurrent ask calls racing four refresh request calls plus a report without losing a record; developer sessions excluded; an MCP contract change and a legacy record without start metadata reported as capability-deferred; CLI gating to the coordinator pane and the installation helper; and, through the installation entrypoint, two upstream updates and a rollback that stage r2, r3, r4 for the worker and r1..r3 for the coordinator, refuse every earlier receipt, and leave the open question, the report binding, the worktree, and the branch unchanged.
One run in a named Herdr 0.8.2 session sum-lab-t288-* with a lab state home under /tmp, this checkout's helper, and two throwaway repositories; the user's default session and the production .sum were never touched, and the lab session was deleted afterwards.
Both harnesses were dispatched with the same tiny brief (list the brief, answer READY, stop), given one recorded decision each, and then refreshed once with refresh request.
| Observation | Claude Code (claude, Fable 5.1, auto mode) |
Codex CLI (codex, gpt-5.6) |
|---|---|---|
| Startup in a fresh worktree | Blocked on the folder-trust dialog; Herdr returned agent_not_ready, sum recorded needs-attention and did not relaunch |
Same: directory-trust dialog, agent_not_ready, needs-attention |
| After the lab accepted the dialog | idle; the brief prompt ran, sumctl brief list was executed, READY, done in 26 s |
done, but the account had hit its usage limit: the brief prompt sat in the composer and no turn ran |
| Refresh delivery | Pane observed done; instruction submitted; state submitted-unconfirmed |
Pane observed done; instruction submitted; state submitted-unconfirmed |
| Receipt | Read briefs/r2.md, ran brief adopt, applied the recorded decision with resolve, kept its checkout and finished work, stopped; confirmed after 39 s |
None: the composer showed the answer notice and the refresh text concatenated, unprocessed; status honestly stayed submitted-unconfirmed with one attempt recorded |
| Native safe boundary | Herdr agent get/agent prompt lifecycle; agent prompt refuses blocked agents |
Same Herdr surface; no harness-specific hook was available or used |
What this establishes: the same fixed instruction reaches both harnesses only through Herdr's settled-state gate, the receipt is the only thing that turns a row confirmed, and a settled (done) client that cannot act, here for a billing reason, is reported as unconfirmed rather than updated.
What it does not establish: parity between harnesses, behavior under a blocked permission dialog mid-task, or any harness-native refresh hook; none exists in this slice, so pending state is rechecked only at ordinary interactions.
Executed on the development host for issue #8 from a task checkout: the Python suites (95 tests, of which 3 in the new tests/test_fleet.py), the Node Mesh tests (10), the offline demo including its capacity check, and scripts/live_smoke.py against real Herdr 0.8.2 in a named sum-test-* lab session.
The refactor of tests/test_core.py into shared ReleaseLab/UpdateLab fixtures also removed seven tests that the previous class inheritance ran twice; no test case was dropped.
The deterministic regression dispatches twelve scripted workers across twelve isolated Git repositories through the installation entrypoint with capacity raised to twelve, then records a long in-flight tool call, a dirty checkout, an open question, an answered-unapplied question, a pending report, a client connected under an older MCP tool contract, a closed parent, an unknown (removed) worker pane, a worker whose prompt Herdr refuses, a worker that ignores its refresh, and two cooperative workers.
It stages and activates N+1, requests a rolling refresh, answers and reports through the old absolute callbacks, activates N+2 and interrupts the refresh mid-pass with KeyboardInterrupt, inspects and recovers with an ordinary repeated request, rolls the default back, refreshes again, refuses an unmerged update while ask, report, and inbox --live continue, and finally admits a new task only after a reported task is archived.
Asserted: no agent start, worktree create, agent read, or any stop/kill call after the initial twelve launches; every worktree keeps its HEAD, branch, and uncommitted file; every question, answer, and report is present with its status; reports stay bound to the revision they were made under; the new task's sidecar records the rolled-back default; the thirteenth dispatch was refused at admission.
Two further tests cover twelve concurrent prepare calls against a global limit of six (exactly six admitted, six refused, six worktree create calls, admission order serialized) and six concurrent writers into one repository (one admitted), legacy defaults without a settings file, ten invalid settings variants refused before any side effect with the file left as found, a symlinked settings file refused, developer panes and candidate checkouts refused for settings set, a lowered limit that evicts nothing, a reported task holding its slot until archived, and the settings file in the records backup.
Measured on this host (Apple Silicon macOS, Python 3.13.5, strict fake Herdr subprocess per call). These are the numbers of one run, not a hardware-independent guarantee:
| Pass over 12 workers | Wall time (ms) | Herdr calls | agent list |
agent get |
agent prompt |
|---|---|---|---|---|---|
status --live |
87 | 1 | 1 | 0 | 0 |
refresh request after update one |
430 | 11 | 1 | 0 | 10 |
refresh request recovery after interruption |
310 | 7 | 1 | 0 | 6 |
refresh request after rollback |
459 | 11 | 1 | 0 | 10 |
inbox --live after a refused update |
92 | 1 | 1 | 0 | 0 |
Before this slice the same passes made one agent get per task (twelve observation calls, each with its own five-second ceiling) before any delivery; now one snapshot per session bounds observation, each prompt keeps its own five-second ceiling, and no transcript is read.
The real-Herdr lab (scripts/live_smoke.py) prepared thirteen tasks as shell panes without agents in one named session: status --live took 73 ms and refresh request 289 ms with exactly one real herdr agent list each, every row honestly pending-unreachable.
No model was launched; the authenticated-harness fleet canary in skills/sum-update/SKILL.md and docs/ACCEPTANCE.md section 8 remains a documented manual step and was not executed for this slice.
Executed on the development host for issue #13 from a task checkout: the Python suites (234 tests, of which 15 in the new tests/test_returns.py), the Node Mesh tests (10), and the offline demo including its new coalesced-question assertion.
The new tests cover: two distinct questions producing two bounded notices where the second names both IDs and neither carries question text; the same key asked twice recording one obligation and one send; three concurrent ask calls, a report, and a refresh request leaving every question, the report, and the requested revision open with at most one parent prompt per write; an answer visible as submitted until resolve closes it, never re-sent; a busy root reaching exactly three known-not-delivered attempts, then stalled, then one explicit notice; a closed root with two tasks keeping every return, bind --parent-only presenting one inline catch-up for the rebound task only, and a coordinator init listing the same items without a second delivery; a parent rebound between observation and send getting nothing typed and the new route refused until it is the registered coordinator; a prompt that timed out after the fake had received it recorded as uncertain, not retried by the pump or inbox --live, and sent once more only by an explicit notice; a KeyboardInterrupt during the prompt leaving an in-flight record that reads as uncertain with no duplicate turn and no claim in the legacy slot; the frozen helper from b1239a4 reporting, answering, and asking on the same task without changing a byte of returns.json, its records reconciled as pending returns; two instances on one machine keeping separate sidecars; the worker pane refused when answering its own question (in-process and through the CLI with a forged approval text); a report obligation surviving show and inbox --live and closing only on verify; a pending-busy refresh riding the next worker notice with the exact brief adopt command and recorded as submitted-unconfirmed for refresh status; and the pump CLI refused for an unregistered pane.
The fleet regression still makes exactly one Herdr call for status --live over twelve workers: returns owed to the calling coordinator are presented inline, and the closed root's returns fail against the snapshot without an extra call.
No model, GitHub write, live installation state, or user default session was involved.
Executed on the development host for issue #14 from a task checkout: the Python suites (253 tests, of which 19 in the new tests/test_hook.py), the Node Mesh tests, the offline demo with its new plugin scenario, and the explicit real-Herdr live smoke test with its new plugin section.
Plugin contract facts were taken from the installed Herdr 0.8.2 (herdr api schema plus a probe plugin in an isolated named session), not from the website: accepted manifest event names pane.agent_status_changed, pane.agent_detected, pane.exited, pane.closed, workspace.closed (an unknown name is a link warning, not a failure); handler environment HERDR_SESSION, HERDR_SOCKET_PATH, HERDR_PLUGIN_ID, HERDR_PLUGIN_EVENT, HERDR_PLUGIN_EVENT_JSON ({"event", "data": {"type", "pane_id", "workspace_id", "agent_status", "agent"}}); plugin link takes effect while the server runs and is idempotent for the same path; startup hooks do not run at link time; a failing hook is logged with its exit code and leaves the plugin enabled; agent read prints text, not JSON.
The deterministic tests cover: manifest and live link with one explicit reconciliation; enable refused for a developer pane; disable/unlink and re-enable with the synchronous ask path unchanged in between; read-only hook status with pending count and age; a question saved while the root was busy, a stale idle edge that observed working and typed nothing, then the real idle edge delivering once within at most three Herdr calls and a duplicate edge sending nothing; already-satisfied conditions on enable and startup reconciliation; a stalled return retried on an edge while an uncertain one is not; a pending-busy brief revision delivered on the worker's idle edge with the exact brief adopt command; a blocked permission UI recorded as attention with the excerpt kept out of the notice and no question or approval invented, closed when the worker resumed; a prose-only question recorded once, unchanged by newer routine output, superseded by the captured ask; an open saved question preserved through newer output and edges; exit, pane.exited, and pane.closed recorded as distinct attention with --seen; the launch handshake idle not counted; unrelated panes, other sessions, a wrong plugin id, a missing session, and a malformed pane id refused or ignored without records; two installations with distinct plugin ids and homes; a closed root leaving returns pending until --reclaim and bind --parent-only, with the rebound root then served on its own edge; six concurrent handler processes sending one notice; a handler crash and an unreadable health file leaving ask and report unaffected and the rundown reporting degraded; inbox --live reconciling only while enabled; and twelve workers served by one edge with one snapshot, then disable, blocked edges, and re-enable catching up the already-blocked three.
The live smoke test linked the lab plugin into an isolated registry, drove real pane report-agent edges, and observed the real server run the handler: event-to-attention wall time on this host was about 90-140 ms per edge with a 2-36 ms handler; an unrelated pane was ignored; unlink left the registry empty.
Repair 1 (coordinator review of f33661b): a coalesced delivery now stamps and prompts only pending/retryable items; a submitted sibling is still named in the text, an uncertain one is neither named nor restamped. Every agent read excerpt invalidates that session's snapshot, so the pump observes again before any prompt. A payload working closes attention only after a fresh observation shows it, any fresh non-blocked status clears a blocked record, an unobservable pane degrades without using the payload as truth, and every handler exception is recorded in health. Three regressions added (uncertain sibling, recipient turning busy during the read, out-of-order working); one issue-13 fleet expectation updated because the first task's already-submitted return is no longer restamped by the second task's write. Suite now 256 tests.
Repair 2: sumctl pump with reason=None no longer indexes the legacy reason table with attention (KeyError before stamping); the generic reason is used. Regression covers the inline and prompt paths with a pending attention beside an uncertain question. Suite now 257 tests.
Not measured: human-labeled question capture on real harness tasks. A shell reported as an agent is not a model; native idle alone does not detect a question asked in prose, and no such claim is made. Unsupported observation (plugin disabled, unlinked, or a Herdr build without these events) is reported as degraded with the rundown path intact.
Executed on the development host for issue #15 from a task checkout: the Python suites (267 tests, of which 10 in the new tests/test_context.py), the Node Mesh tests (10), and the offline demo with its new context section.
The new tests cover: a 45 KB report beside an unresolved early question (outline and coordinator view name the question, prose bounded with exact character counts, --max-chars 0 returns it whole); 45 questions paged three times with total/omitted/next_after and every unapplied decision listed on each page; a question asked and another answered between two pages, counted on the next page and named by --since CURSOR, with an unchanged cursor rendering no section; r1 read back byte-identical and hash-verified after r2 was adopted, a tampered file reported without content, an unknown revision refused; a missing notes artifact and handoff artifact strings classified as checkout (present or not) or outside-checkout without being opened; notes appended by worker and coordinator panes, a credential-shaped note refused, a symlinked notes.md neither read nor written through nor backed up, --section taking only fixed names; credential-shaped question text redacted in the view but intact in the record; the full show shape unchanged and the frozen helper from b1239a4 showing and asking on a task that carries notes; worker, reviewer, and coordinator views of one candidate sharing a cursor while differing in sections, decision filters, skill references, and contract; context and help read-only from a candidate checkout while notes is refused; and help/help TOPIC/help brief-adopt generated from the parser.
Measured on one representative lab fixture (six questions in three states, two reports of 5 KB and 4 KB, a handoff, a reviewer finding, a staged r2): bytes of stdout, one run, no token conversion.
| Read | Bytes |
|---|---|
show |
53330 |
context (outline) |
2598 |
context --role worker |
9389 |
context --role reviewer |
25314 |
context --role coordinator |
15294 |
context --since CURSOR (unchanged) |
1442 |
help |
4512 |
help context |
1698 |
--help text |
6027 |
Token savings were not measured; the byte counts above are the evidence.
Repair 1 (coordinator review of 131a10e): worker-supplied artifact strings are now classified by string rules alone (absolute, ~, or any .. component is outside-checkout; no realpath, stat, exists, or open on any worker path, so a checkout symlink logs -> /host-secrets is never followed); every handoff string in every view (next_action, task_ref, review_ref, files, artifacts, decisions_unresolved, checks, PR strings) is projected through redaction and --max-chars bounding instead of dumped raw; the cursor carries a digest of status, endpoints, checkout, cleanup, PR identity, and error so --since never reports unchanged after an archive or rebind; notes.md writes fsync the file and directory like every other record. Two regressions added (symlink escape with a stat spy and a realpath/exists trap; tokens in every handoff field), the tautological traversal assertion replaced by the real argparse and task-ID refusals, and an archive-after-cursor check. Suite now 269 tests. Adding the context command to the return channel changes the brief fingerprint, so existing tasks receive it through brief regenerate/refresh request (#7), never by rewriting a brief being read.
No model, GitHub write, live installation state, or user default session was involved; mise run test-live was not re-run for this slice (no Herdr call was added).
Executed on the development host for issue #18 from a task checkout: the Python suites (334 tests, of which 21 in the new tests/test_metadata.py; the file no longer re-runs the inherited cleanup cases), the Node Mesh tests (10), the offline demo with its new metadata section, and the explicit real-Herdr live smoke test with its new metadata section (three passes).
Metadata contract facts were taken from the installed Herdr 0.8.2 in an isolated named session, not from the website: pane report-metadata and workspace report-metadata print nothing on success and accept --token NAME=VALUE and --clear-token NAME under a --source; tokens ride pane get, agent get/list, and workspace get/list as tokens; any source may clear a key (the latest accepted update wins); values are normalized and capped at 80 characters; an empty value clears; errors are pane_not_found, workspace_not_found, invalid_metadata_token; notification show answers shown: false, reason: disabled while the user's [ui.toast] delivery is off; plugin pane open --plugin ID --entrypoint ID --placement split --target-pane PANE returns plugin_pane.pane.pane_id, a popup returns no pane id; api schema --json exposes PaneReportMetadataParams, WorkspaceReportMetadataParams, and NotificationShowParams, which metadata enable probes before writing anything. Token metadata is not restored across a server restart (documented), so the rundown, coordinator init, the startup hook, and metadata sync compare Herdr's held tokens with the record and rewrite what is missing.
The deterministic tests cover: disabled by default with every write path unchanged and no metadata directory created; enable probing the binary, recording the sum:<instance> source, projecting once, and touching neither report-agent, rename, titles, display names, nor state labels; enable refused for a developer pane; a build without report-metadata refused at enable and, when the command disappears later, a first refused write marking the capability off and degraded while ask, answer, and resolve keep working; a worker seen working beside a needs-decision token with the lifecycle untouched; states following the records in the order the user acts (review-ready, verified, merged-cleanup-pending with the exact PR URL); instruction-refresh-pending with sum_rev r1>r2 cleared by brief adopt; three duplicate idle edges and an unchanged pass writing nothing and making no Herdr call; opt-in notifications coalesced into one notification show per pass with task ids, states, and repository names only, credential-shaped question text absent from every token and notification, repeated passes silent, a fresh transition after leaving a state notifying once, and shown: false reported honestly; notifications off remembering transitions without replaying them when enabled later; a user's pane and workspace labels and another reporter's jj_status token preserved through enable and disable; a stale pane (another cwd) receiving nothing and losing sum's old keys while the workspace keeps the task's; an absent pane reported as such; bind --worker-pane clearing only sum's keys on the old pane and projecting the new one; two installations with two sources whose disable leaves the other's tokens intact; a blocked worker with a private permission prompt visible as attention-blocked and 1 blocked without the prompt text, superseded by the saved question; cleanup and archive releasing exactly the recorded keys (a user's other key on the coordinator pane and mine on a worker pane survive) with every --clear-token a sum_ key; the CLI ask/report output shape unchanged with projection after the record; snippet as text only with no config.toml written, the inbox entrypoint riding the hook manifest as /bin/sh -c '"$0" "$@"; ...; read _' sumctl --home HOME inbox, popup and split placements, an unknown placement refused; a simulated server restart emptying every token, an ordinary task write still believing the record, the reconciling pass forgetting three endpoints and rewriting them, and the startup hook rewriting a dropped workspace; disable clearing pane, workspace, and coordinator tokens with the synchronous path intact; and twelve dispatched workers through update apply and refresh request with every pane and workspace at instruction-refresh-pending r1>r2, the coordinator line 12 refresh · contract r1, exactly 25 writes per pass, one snapshot for the refresh and one for the projection, no agent get, an explicit sync writing nothing, three adoptions returning to running and 9 refresh, and every checkout clean.
The live smoke test enabled projection over 13 prepared tasks in the real session: 13 workspaces, 13 verified shell panes (each verified by one pane get because no agent occupied them), and the coordinator pane in about 350-430 ms with 14 observation calls; the Herdr-given workspace label survived; an unrelated workspace received no tokens; an explicit sync made one snapshot and one workspace list and wrote nothing; metadata inbox --placement split opened the plugin pane in the coordinator's workspace, whose text held the task id and the ordinary inbox JSON before the press Enter to close line; disable cleared 27 endpoints and left both workspace get and the coordinator pane get without tokens.
Fixture changes: the fake Herdr gained api schema --json, pane/workspace report-metadata (per-source token patches, real error codes, empty-value clear, FAKE_NO_METADATA for an older build), notification show (FAKE_TOAST), pane rename, workspace list, and plugin pane open with the real response shape; panes the fake created keep their own cwd and agent instead of being rewritten to the scripted root scenario whenever they act as the caller, which the demo relied on in one place and now states explicitly.
Not measured: how many model turns a real user saves by reading the sidebar instead of asking; that is step 10 of docs/ACCEPTANCE.md. The parked live-smoke flake (a plugin log row still running when listed right after the stranger pane's two edges) blocked one of four smoke runs of this slice; the check now waits up to ten seconds for no running rows and formats its message without assuming a stderr field, which is the smallest change that keeps the existing assertion honest. Existing hook plugins linked before this slice report expected_manifest_current: false until hook enable relinks the manifest with the inbox entrypoint; nothing else about them changes.
Executed on the development host for issue #30 from a task checkout: the Python suites (356 tests, of which 22 in the new tests/test_projects.py, one of them against the installed real mise), the Node Mesh tests (10), the offline demo with its new managed-project section, and the explicit real-Herdr live smoke test (unchanged sections, one pass).
Two facts were taken from the installed binaries in isolated lab directories, not from documentation: mise 2026.9.1 run in <installation>/projects/<owner>/<repo> lists sum's mise-tasks/test and demo with their parent source, so a nested clone with no configuration of its own would execute the parent's test; and Herdr 0.8.2 pane get returns the pane's start cwd and the foreground process's foreground_cwd, which init now reads before registering a role.
The deterministic tests cover: a fresh --remote enrollment cloning exactly one repository into the Git-ignored projects/<owner>/<repo> through a private staging directory with no gh call, git ls-files empty for it and git status --ignored listing it; a repeated enrollment returning already-enrolled with a local file and dirty state intact, a plain re-enrollment without --remote comparing nothing against gh, and a different remote refused rather than switched; two owners' same-named repositories and a --host repository landing at three distinct paths under three distinct names, the URL form of a name matching its registration, and a bare name refused; a user directory, a clone of another repository, an external clone with another origin, and a stray clone inside the installation all refused with contents, HEAD, and origin untouched and no registration written; a symlinked projects/<owner> component and a symlinked --path refused while a real path with spaces is adopted as external; a failing default-host clone (the strict fake gh refuses like a failed authentication) leaving no directory, no empty parent, and no registration; enrolling douglasjarquin/sum registering the installation itself with nothing cloned, its migration refused, and a --project dispatch of it branching into a separate worktree; a developer pane and a candidate checkout unable to enroll; a dirty legacy .sum/projects/<owner>/<repo> clone with a prepared task adopted as legacy where it is with no second clone, its task and linked worktree named as migration blockers, --apply refused, and a second writer into it refused at admission; a migration refused while a fake process runs inside the clone or the process table is unreadable, then applied with zero references, keeping HEAD, .git, and an uncommitted file and recording migrated_from; a clone placed by hand at the canonical path adopted, not replaced; a --project task whose Herdr worktree is outside the installation receiving a brief ## Delivered runtime section with the installed helper path and the absolute skill path, byte size, and hash of the runtime file, no ../skills anywhere, and context --role worker naming the same helper and files; a --repo path attaching the registered identity or null, --repo plus --project refused, an unknown project refused; an enrolled clone whose directory vanished refusing dispatch without re-cloning; a pane whose cwd is inside a managed clone of an unowned installation refused by init with no context.json and no registration, the same for an enrolled external clone, while a pane elsewhere still initializes and a dispatched worker pane keeps its role even with a cwd under projects/; mise_task_origins for a nested clone without configuration reporting the parent test as inherited with verification.test: false and the named problem, a project owning verify/test shadowing the parent's names, env discover on a worktree under a foreign parent mise.toml recording task_origins and the problem beside the project's own package script and env show carrying it, a missing or non-executable mise reported as such; and release stage, source_files, git ls-files, and a records backup excluding project code while the backup carries state/projects.json, plus an unknown registry schema preserved and refused.
Not measured: enrollment through a real gh repo clone against GitHub (the fake refuses it; --remote exercised git clone against local bare repositories), and no authenticated harness was launched in a nested clone; step 11 of docs/ACCEPTANCE.md covers both.
Fixture changes: a strict fake mise (tests/fixtures/mise.py) that resolves mise.toml tasks and mise-tasks/ files up the directory tree like mise, bounded by FAKE_MISE_STOP; every lab environment sets it so env discover never runs the host's mise. Existing .sum/projects/... clones and --repo paths keep working unchanged; the production installation had no such clone at the time of this slice.
Executed on the development host for issue #36 from a task checkout: the Python suites (505 tests, of which 15 offline in the new tests/test_graph.py plus one opt-in real-binary case), the Node Mesh tests (10), the offline demo with its new code-graph section, the portable runner's --check, and the maintenance audit against the base (clean; the maps docs/features/README.md and docs/features/graph.md and mise.toml are the policy files the root review must read).
mise run test-live was not re-run for this slice (no Herdr call was added or changed).
Facts were taken from the real @colbymchenry/codegraph 1.5.0 binary (installed through the mise pin into ~/.local/share/mise/installs/npm-colbymchenry-codegraph/1.5.0) in isolated lab repositories, not from its README, and the strict fake tests/fixtures/codegraph.py is scripted from them: init writes only .codegraph/ with its own .gitignore and leaves the repository's .gitignore alone, so the directory shows as untracked until .git/info/exclude (honored by both Git and the indexer) or a .gitignore entry covers it; a linked worktree without its own index answers from the main worktree's index with a warning and reports initialized: false; indexing honors ignore rules (nothing under projects/, .sum/, .local/, .deps/, .artifacts/ in sum's own 47-file index); pendingChanges counts only uncommitted edits, a committed change leaves it at zero while the query is stale, and status never syncs on its own, which is why sum records the HEAD it last built or synced; a repeated init on an existing index is a cheap no-op; CODEGRAPH_DIR accepts only a plain name; install --print-config prints without writing.
Measured on this host with the real binary against sum's checkout (47 files, 2,200 symbols, 10,204 edges): init 0.69 s wall (234 ms indexing), repeated init 0.08 s, sync after one edit 0.24 s, query 0.15 s, explore 0.14 s. In the scripted twelve-worker lab each dispatch including the graph initialization took about 520-590 ms wall through the installed entrypoint. Upstream's published speedups are not repeated as local measurements.
The deterministic tests cover: prepare initializing the index inside the new checkout after identity validation with the exclude written once, the record (tool version against the pin, identity with HEAD, branch, and common Git directory, attempt with exit and duration, index identity, freshness), a clean git status, the brief's ## Code graph section with exact CODEGRAPH_NO_DAEMON=1 commands and the rules, context --role worker and status carrying the state, and no install/serve/upgrade call ever made; two worktrees of one repository returning their own graphs for one symbol while the clone stays uninitialized; a repeated graph init verifying instead of rebuilding, an uncommitted edit reported stale with reconcile_needed: sync and a stale query returning nothing, the brief's sync command making it fresh; ignored trees excluded and a repository that already ignores .codegraph/ receiving no exclude write; a failing init keeping the task prepared and the checkout intact, the brief naming the fallback and the retry command, start still launching exactly one worker, a retry after the cause is fixed recording ready without rewriting the worker's brief; three failures recording exhausted and a fourth graph init refused; a timed-out build killed as a process group with the attempt recorded and a later retry succeeding; two build slots bounding concurrent builds (peak observed at most 2) and a held house recorded deferred, not as a failure; a runtime without the binary or with another version recorded unavailable with the reason, the checkout clean, doctor reporting the tool as optional; graph config printing snippets for claude, codex, cursor, and opencode with the pinned path, refusing others, and changing no file under a lab HOME or the lab root; .codegraph/ classified disposable for cleanup and a backup carrying graph.json with rebuild metadata and no index; verify --execute building a separate index in the detached checkout, recording it in the run evidence, removing it with the checkout, and never touching the worker's index; dev prepare initializing and a reopen reconciling; a release manifest carrying the codegraph provenance and pin, a bundle containing .codegraph refused, and an older bundle without the pin still passing verify_release; and twelve dispatched workers through the installed entrypoint each with its own index within the slot bound, unchanged across update apply and a rollback.
The real-binary case (SUM_REAL_CODEGRAPH_BIN=... python3 -m unittest tests.test_graph.RealCodegraphTest) passed on this host: separate indexes per worktree, the nested ignored tree absent from the index, an uncommitted edit reported stale until sync and then found, a repeated init verified, nothing written under a lab HOME, and no codegraph process left for the checkout.
Not measured: a real harness reading the brief's graph commands or a hand-merged MCP snippet (step 12 of docs/ACCEPTANCE.md), and the graph on a large repository; the 300 s bound is a limit, not a measurement. A worker's own codegraph serve --mcp, when a harness starts one, is that session's process and is not managed by sum.
Fixture changes: the strict fake codegraph above with FAKE_CODEGRAPH_FAIL, FAKE_CODEGRAPH_SLEEP, FAKE_CODEGRAPH_VERSION, and FAKE_CODEGRAPH_EXTRACTION knobs and a concurrency peak counter; every graph lab sets SUM_CODEGRAPH_BIN to it and HOME to an empty directory. Existing labs without that variable see the graph as unavailable and keep every earlier assertion. The worker skill changed, so existing tasks receive the new ## Code graph section only through brief regenerate/refresh request, never by rewriting a brief being read.
- A full
mise run setupdependency download/install. The container could not reach the required network endpoints. - MCP initialization/tool discovery using the actually installed upstream SDK. A smoke test is provided and is run by setup after installation; the executed Node tests instead exercise the production command handlers with a fake runner.
mise run test-liveagainst a real Herdr server. The real binary was not available here.- Authenticated Codex, Claude, Grok, Cursor, Pi, or other model execution.
- Busy/closed-coordinator behavior with actual harnesses, or human-labeled real-task question capture.
- Remote GitHub repository creation, a push, hosted CI, or PR publication. The connected GitHub interface exposed read operations only;
douglasjarquin/sumwas not created by this build. - macOS execution. A macOS/Linux CI matrix is included but has not run on GitHub.
The checked-in pin, upstream documentation, and command fixtures are not a substitute for live conformance. In particular, source review does not establish every installed harness's trust prompts, instruction loading, or native event behavior.
With Python and Node already available, no network or credentials are required:
python3 -m unittest discover -s tests -p 'test_*.py' -v
(cd go && go test ./internal/cli -run TestOfflineDemo ./internal/mesh ./cmd/herdr-mesh)On the execution host, first run setup, then mise run test-live and the steps in ACCEPTANCE.md. Use a throwaway project for the first real-model delegation.
This MVP does not guarantee unattended delivery or automatic capture of questions printed only in prose. It does not isolate credentials, enforce model spending, safely replace live workers, provide exactly-once external actions, or back up worktree code. A worker report is not verification evidence. See the README for operational limitations.