Skip to content

hypatia-bench keeps its own copy of ci.yaml's rust gate list #5

Description

@Jerhaad

hypatia-bench exists so that a green run predicts a green pull request. It keeps
its own copy of the list it is being green about.

ci.yaml's rust-tests job runs five recipes:

just rust-test-core
just rust-test-db
just rust-test-ingest
just rust-test-api-unit
just rust-test-api-integration

bin/hypatia-bench runs five commands of its own that happen to mean the same
thing today. The equivalence is hand-maintained, and it has already broken once:
the bench named cargo test -p api-rest --test integration until b3f2ec7, long
after that binary stopped existing — the integration suite had been split into
one file per surface, and a gate naming a target that no longer exists is a gate
that runs nothing. Nothing reports the next divergence either, in either
direction: a recipe that changes what it runs, or a step added to the job, is
invisible here because the two lists never meet.

Fix: read the commands out of ci.yaml and run those, the way
bin/hypatia-compose-check already reads the compose invocation, instead of
keeping a second spelling. Refuse by name when the extraction comes back empty,
so an unparsed workflow cannot pass for an empty gate list.

The scope is the cargo test block. The steps around it — cargo check, the
migrate run, the table-count assertion, the test-case registration — differ from
ci.yaml deliberately, for the reasons their comments give, and reading those out
of the workflow would be wrong rather than merely different.

just is installed on the build host (mise, 1.36.0) but is absent from the PATH
a non-interactive ssh gets: ~/.local/share/mise/shims/just, or mise which just.

Done when: the bench runs the rust test list ci.yaml names, names each
command it runs, and fails when a test file under crates/api-rest/tests/ fails.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions