Skip to content

refactor(helper): retire the unadopted Go prototype #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
douglasjarquin wants to merge 3 commits into from
Closed

refactor(helper): retire the unadopted Go prototype #90

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fa9e69f
fix(benchmark): execute the frozen Go helper reference honestly
douglasjarquin Sep 10, 2026
9578b6a
refactor(helper): retire the unadopted Go prototype
douglasjarquin Sep 10, 2026
7cdb7a4
fix(release): bind native artifacts to bundled contracts
douglasjarquin Sep 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
227 changes: 227 additions & 0 deletions benchmarks/issue-38/raw.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,227 @@
{
"schema": 1,
"binary": "/Users/douglasjarquin/github/sum/.sum/dev/issue38-audit/.artifacts/issue38/sumctl-go",
"binary_bytes": 6109458,
"reference_revision": "b03b8020621e0d417906402a5c7ecc5d63192541",
"reference_helper": "/var/folders/5j/n805d0vn43g2bjt7ykqmyxmm0000gn/T/sum-go-benchmark-1a1wn4kb/reference/bin/sumctl",
"samples": 15,
"scenarios": [
{
"command": [
"/Users/douglasjarquin/github/sum/.sum/dev/issue38-audit/.artifacts/issue38/sumctl-go",
"--version"
],
"samples": 15,
"p50_ms": 5.919,
"p95_ms": 16.432,
"min_ms": 5.293,
"max_ms": 16.432,
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"peak_memory_bytes": 3490224,
"subprocesses": 0,
"id": "startup.version.cold"
},
{
"command": [
"/Users/douglasjarquin/github/sum/.sum/dev/issue38-audit/.artifacts/issue38/sumctl-go",
"--help"
],
"samples": 15,
"p50_ms": 5.801,
"p95_ms": 6.585,
"min_ms": 5.251,
"max_ms": 6.585,
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"peak_memory_bytes": 3523016,
"subprocesses": 0,
"id": "startup.help.cobra"
},
{
"command": [
"/Users/douglasjarquin/github/sum/.sum/dev/issue38-audit/.artifacts/issue38/sumctl-go",
"--home",
"/var/folders/5j/n805d0vn43g2bjt7ykqmyxmm0000gn/T/sum-go-benchmark-1a1wn4kb/state",
"status"
],
"samples": 15,
"p50_ms": 133.056,
"p95_ms": 156.934,
"min_ms": 126.481,
"max_ms": 156.934,
"exit_codes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"peak_memory_bytes": 3555784,
"subprocesses": 1,
"id": "read.status.fixture"
},
{
"command": [
"/Users/douglasjarquin/github/sum/.sum/dev/issue38-audit/.artifacts/issue38/sumctl-go",
"--home",
"/var/folders/5j/n805d0vn43g2bjt7ykqmyxmm0000gn/T/sum-go-benchmark-1a1wn4kb/state",
"show",
"t-000000000000"
],
"samples": 15,
"p50_ms": 132.928,
"p95_ms": 164.723,
"min_ms": 127.212,
"max_ms": 164.723,
"exit_codes": [
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1,
1
],
"peak_memory_bytes": 3572168,
"subprocesses": 1,
"id": "failure.show-missing",
"expected_exit": 1
}
],
"scope": "compiled Cobra entrypoint; status and missing-show use the explicit Python compatibility boundary",
"allocations": {
"command": [
"go",
"test",
"-run",
"^$",
"-bench",
"BenchmarkNewRoot",
"-benchmem",
"./internal/cli"
],
"exit": 0,
"ns_per_op": 6299.0,
"bytes_per_op": 33080.0,
"allocs_per_op": 133.0
},
"gate": {
"outcome": "defer",
"pass": false,
"comparisons": {
"startup.version.cold": {
"baseline_p50_ms": 136.568,
"candidate_p50_ms": 5.919,
"absolute_improvement_ms": 130.649,
"relative_improvement_percent": 95.67
},
"startup.help.cobra": {
"baseline_p50_ms": 138.56,
"candidate_p50_ms": 5.801,
"absolute_improvement_ms": 132.759,
"relative_improvement_percent": 95.81
}
},
"interactive_hot_path": {
"required_absolute_ms": 50,
"required_relative_percent": 35,
"observed_absolute_improvement_ms": 132.759,
"observed_relative_improvement_percent": 95.81,
"pass": true
},
"frequency_weighted": {
"required_ms": 500,
"observed_ms": 0,
"pass": false,
"reason": "No stateful command is native; status remains a compatibility subprocess."
},
"behavior": {
"required_regressions": 0,
"observed_regressions": null,
"pass": false,
"status": "not-evaluated",
"basis": "Expected exit codes are smoke checks, not differential output or effect parity."
},
"memory": {
"required_regression_percent": 10,
"pass": false,
"status": "not-comparable",
"reason": "The compatibility child is outside the compiled parent's /usr/bin/time memory sample."
},
"allocations": {
"command": [
"go",
"test",
"-run",
"^$",
"-bench",
"BenchmarkNewRoot",
"-benchmem",
"./internal/cli"
],
"exit": 0,
"ns_per_op": 6299.0,
"bytes_per_op": 33080.0,
"allocs_per_op": 133.0
},
"binary_and_entrypoint": {
"version": "startup.version.cold",
"help": "startup.help.cobra"
},
"reasons": [
"The native startup/help path crosses the latency gate.",
"No stateful command is native, so the 500 ms frequency-weighted gate is not established.",
"Behavior parity has not been evaluated.",
"Compatibility memory is not comparable to the Python child process."
]
}
}
74 changes: 74 additions & 0 deletions benchmarks/issue-38/report.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Go helper decision: discard the unused prototype

Do not adopt the Go helper or plan a production cutover under issue #39.
The experiment demonstrates faster native version/help handling, but no stateful operation moved to Go and the required adoption gate is not met.
Production remains on the existing Python helper, and the separately delivered Go Herdr Mesh remains available.

## Measured candidate

The corrected driver and recoverable experiment are at commit `fa9e69f6a9c95c73904e6a7316bfbec7293d0b1a`.
Its Go source is unchanged from the compiled source at `d9d0bd210aa06d4dd87215460a642e1a4a2979aa`.
The Python reference is `b03b8020621e0d417906402a5c7ecc5d63192541`, extracted from local Git and used for fixture setup and compatibility execution.
The build used Go 1.25.0 with `CGO_ENABLED=0`, `GOENV=off`, `GOTOOLCHAIN=local`, `-trimpath`, and `-buildvcs=false`.
The runtime used Python 3.13.5 on macOS arm64.
No model, native Herdr pane, live installation state, or installed runtime was used.

The executable was 6,109,458 bytes with SHA-256 `1faf6062b68d2c9d3d06557f8f4e519e90e6169fe13d4bbd0195a2ffd6de66c4`.
The retained [raw result](raw.json) is the exact 15-sample output of the corrected benchmark, with SHA-256 `6aeb8054523f69d7e58a30236e41c7759c1c43a1e60663546ec75831aebef9a5`.
Recorded temporary paths identify the measured run; those fixtures were removed after it completed.

| Compiled entrypoint | Median | Implementation |
| --- | ---: | --- |
| `--version` | 5.919 ms | Native Go |
| Native root `--help` | 5.801 ms | Native Cobra rendering, not Python help parity |
| Fixture `status` | 133.056 ms | Python compatibility subprocess |
| Missing-task `show` | 132.928 ms | Python compatibility subprocess, expected exit 1 |

Fresh Cobra tree construction measured 6,299 ns/op, 33,080 B/op, and 133 allocations/op.
The reported subprocess counts are declared compatibility-boundary counts, not measured totals of nested Python, Git, or Herdr processes.
The memory samples omit the compatibility child's peak usage and cannot establish a whole-operation memory regression bound.

## Unchanged adoption gate

The [issue #37 baseline](../issue-37/report.md) and its thresholds were not rerun or altered.
The required gate remains at least 50 ms and 35% improvement on a measured interactive path, at least 500 ms of serial frequency-weighted opportunity, zero behavior regressions, and at most 10% peak-memory regression.

- Native version/help cross the local latency threshold against the recorded baseline.
- No stateful command is native, so the required frequency-weighted opportunity is not established.
- Expected exit codes are smoke checks, not differential output, state, role, durability, or cancellation parity against Python.
- Whole-operation memory is not comparable.

The gate therefore fails; this is a discard decision, not a claimed successful port.
No command is moved to Go, so no new implementation is accepted under a weaker parity standard.
All domain operations remain in their existing implementation.

## Evidence correction

The earlier driver labeled the Python reference with the frozen commit but invoked the working checkout's helper.
A trace of the actual compiled benchmark observed configured source hash `0391d1dc8cbe8f7b2c06d3ae03e5ba289479c1ac547f761cca378fb3120797fa`, differing from the claimed reference hash `bba3eb2d171da1dfd5193e1ade4f0fb4482c0383856b5f969d21065462ac90da`.
After the repair at `fa9e69f`, the same trace observed the latter hash for the configured helper, and the behavior verdict changed from an unsupported pass to `not-evaluated`.
Earlier labeled results are not used as frozen-reference evidence here.

## Reproducing the historical experiment

Use an isolated checkout of `fa9e69f6a9c95c73904e6a7316bfbec7293d0b1a` with the pinned Go and Python tools already available.
The repository must contain the frozen Python commit locally.
Run these commands in that historical checkout, with an owned temporary output directory replacing `/tmp/sum38-reproduction`:

```sh
mkdir /tmp/sum38-reproduction
(cd go && env -u GOROOT -u GOBIN -u GOTOOLDIR -u GOOS -u GOARCH CGO_ENABLED=0 GOENV=off GOTOOLCHAIN=local GOPROXY=off go build -trimpath -buildvcs=false -o /tmp/sum38-reproduction/sumctl-go ./cmd/sumctl-go)
python3 scripts/benchmark_go.py --binary /tmp/sum38-reproduction/sumctl-go --output /tmp/sum38-reproduction/raw.json --samples 15
```

The driver extracts and cleans its own temporary reference and fake-Herdr fixture.
Retain the output if needed for comparison and remove only the owned reproduction directory afterward.
Timing varies by host and run; this is reproducibility of the experiment and decision inputs, not bit-identical performance or binary output.

## Delivery boundary

The retirement removes the unused helper implementation and its current-tree benchmark driver from new source releases.
New staging retains the independently used Go Mesh artifact and its integrity checks.
Historical releases are checked against their own artifact inventory; this does not delete old installed binaries or change a running process.
Issue #39 is deferred/not planned because its prerequisite demonstrated helper win is absent.
This decision does not block Remainder, Pinchos, or other roadmap work, and it does not authorize installation updates or fleet restarts.
7 changes: 5 additions & 2 deletions docs/DEPENDENCIES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,18 @@ The SDK's reviewed transitive graph remains visible in `go.sum`; no Viper, gener
1. Creates local runtime symlinks under `.local/bin`, once. An existing link is never retargeted, because a running process may depend on it; setup reports a differing pin instead.
2. Clones Herdr Mesh at **54adef519aa6af4dcd0bbd72586d414abab90046** into a private staging directory, runs `npm ci --omit=dev --ignore-scripts` against the upstream committed lockfile there, applies the documented runtime overlay below, and renames the finished tree to `.deps/herdr-mesh`. An existing `.deps/herdr-mesh` is never rewritten, reinstalled, or re-patched; drift between its overlay and the current `patches/` is only reported.
3. Copies the release-matched Herdr skill from `herdr --skill`.
4. Builds the cgo-free `go/cmd/sumctl-go` companion as `.local/bin/sumctl-go`; it is not selected by `bin/sumctl` and does not replace the Python helper.
4. Builds the cgo-free `go/cmd/herdr-mesh` companion as `.local/bin/herdr-mesh-go`; its opt-in launcher is `bin/herdr-mesh-go`, and the existing Node Mesh entrypoint remains available.
5. Generates repository-local MCP settings and tests MCP initialization/discovery.

Re-running setup is therefore safe while a coordinator, workers, or an MCP server are using the checkout; it changes nothing they hold open.
Newer code or dependencies go into a staged release instead (below).

The native companion is also built in a release staging directory with `CGO_ENABLED=0`.
The staged binary's source, build requirements, runtime requirements, and SHA-256 are recorded in `release.json` under `dependencies.native.sumctl-go`.
The staged binary's source, build requirements, runtime requirements, and SHA-256 are recorded in `release.json` under `dependencies.native.herdr-mesh-go`.
Running it requires no Go toolchain, module download, Node, Python, or Cobra generator.
The unused Go helper experiment is [not adopted](go-helper-prototype.md); `bin/sumctl` remains the Python entrypoint.
Native artifact requirements come from each release's own dependency inventory, so a missing or corrupt declared artifact is refused without making a retired experiment mandatory for new bundles.
Verification compares the manifest's inventory with the hash-checked bundled inventory and requires each native artifact's canonical path, source identity, version, and current-host platform to agree.

The source revision and upstream lockfile are pinned. This does not claim bit-for-bit reproducibility of every OS/runtime installation. A mise lockfile has not been invented; generate/review it on a networked machine when updating dependency pins.

Expand Down
12 changes: 0 additions & 12 deletions docs/dependency-inventory.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,18 +134,6 @@
"owner": "Model Context Protocol",
"contracts": {"cli": [], "mcp": ["MCP 2025-11-25 compatibility"]}
},
{
"id": "sumctl-go",
"source": "go/cmd/sumctl-go",
"version": "sum 0.1.0",
"checksum": "release.json#dependencies.native.sumctl-go.sha256",
"license": "MIT",
"platforms": ["darwin-arm64", "darwin-amd64", "linux-arm64", "linux-amd64"],
"requirements": ["setup/release staging", "CGO_ENABLED=0", "no runtime toolchain"],
"role": "runtime",
"owner": "sum",
"contracts": {"cli": ["sumctl compatibility argv/stdout/stderr"], "mcp": []}
},
{
"id": "herdr-mesh-go",
"source": "go/cmd/herdr-mesh",
Expand Down
2 changes: 1 addition & 1 deletion docs/features/coordination.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,6 @@ Journeys the coordinator, workers, and developers take through `sumctl` and the
| `live.herdr-smoke` | A named lab Herdr session runs a real dispatch, question, and refresh | manual: `mise run test-live` on a host with Herdr 0.9.0 | operator report |
| `live.harness-canary` | An authenticated harness worker reads its brief and calls `sumctl ask` and `sumctl report` | manual: `docs/ACCEPTANCE.md` section 2 | operator report |
| `performance.hot-path-baseline` | Opt-in local metrics and `mise run benchmark` measure production helper entrypoints in fake and isolated real-Herdr labs before any rewrite decision | automated: `tests/test_benchmark.py` | `lib/sum_measure.py`, `scripts/benchmark.py`, `scripts/benchmark_fixture.py`, `scripts/benchmark_real.py`, `scripts/benchmark_report.py`, `mise-tasks/benchmark`, and retained `benchmarks/issue-37/raw.json`, `benchmarks/issue-37/report.md`, `benchmarks/issue-37/real-herdr.log` |
| `performance.go-cobra-prototype` | A cgo-free compiled Cobra entrypoint keeps the frozen Python helper as an explicit compatibility boundary while measuring native version/help and representative fixture dispatch | automated: `go/internal/cli/root_test.go`, `go/cmd/sumctl-go/main_test.go` | `docs/go-helper-prototype.md`, `go/go.mod`, `go/go.sum`, `go/internal/cli/root.go`, `go/cmd/sumctl-go/main.go`, `scripts/benchmark_go.py`, and benchmark JSON |
| `performance.go-cobra-prototype` | The helper experiment is retired after a measured no-go; new builds retain Go Mesh without building the unused helper, while the decision and corrected measurements remain available | automated: `tests/test_native_packaging.py`, `tests/test_packaging.py` | `docs/go-helper-prototype.md`, `benchmarks/issue-38/report.md`, `benchmarks/issue-38/raw.json`, `go/go.mod`, `go/go.sum`, `lib/sumctl.py`; retired source locations under `go/cmd`, `go/internal`, and `scripts/` are covered by this removal |
| `packaging.native-companion` | Setup and immutable release staging build a cgo-free native companion beside the existing Python/Node runtime, record dependency provenance, and reuse installed artifacts without replacing them | automated: `tests/test_packaging.py`, `tests/test_native_packaging.py`, `tests/test_core.py` | `docs/dependency-inventory.json`, `docs/DEPENDENCIES.md`, `scripts/setup.py`, `lib/sumctl.py`, `mise.toml`, `bin/herdr-mesh-go`, `go/cmd/herdr-mesh/main.go`, `go/internal/mesh/auth.go`, `go/internal/mesh/cli.go`, `go/internal/mesh/config.go`, `go/internal/mesh/helpers.go`, `go/internal/mesh/inputs.go`, `go/internal/mesh/mesh_test.go`, `go/internal/mesh/protocol.go`, `go/internal/mesh/protocol_test.go`, `go/internal/mesh/runner.go`, `go/internal/mesh/service.go`, `go/internal/mesh/service_test.go`, and the staged release manifest |
| `skills.namespace` | Sum-owned skills use one canonical `sum-*` identity across source directories, frontmatter, projections, generated references, and immutable release compatibility paths; collisions are refused without replacing a working projection | automated: `tests/test_skill_namespace.py`, `tests/test_core.py` | offline suite |
Loading