Repository navigation
test: pin the documented e2e setup against what ci.yml actually runs - #1147
Merged
Merged
Conversation
Contributor
Author
|
Important Held for human review by the hive's ACMM level gate. This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the Hive will keep the |
CONTRIBUTING.md and AGENTS.md both tell a contributor to run 'npx playwright install chromium'. Playwright ships the browser bundle and its OS packages separately, and every Playwright step in ci.yml installs both. On a clean Debian 13 host the documented command alone leaves chrome-headless-shell exiting 127 on libglib-2.0.so.0 and 306 of 322 specs failing with 'Target page, context or browser has been closed', which names nothing a reader can act on. CONTRIBUTING.md also calls the e2e coverage report informational and without percentage thresholds, but the e2e-coverage job renders it with --check-source 100, --check-source-regions 80 and --require-source-files, floors already pinned by tests/e2e-coverage-gate.test.mjs. The documented recipe passes none of them, so it clears locally where CI does not. Correct both documents and add tests/contributing-e2e-setup.test.mjs, which reads ci.yml and asserts one-way that the docs are at least as complete as the workflow. Closes #1146 Signed-off-by: quality <quality@hive.kubestellar.io>
The coverage section documented only the data-overlay mechanism; a branch whose props come from generated MDX needs the fixture-docs instance added by #1083. Closes #1143. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
mrbobbytables
force-pushed
the
quality/test-contributing-e2e-setup
branch
from
October 7, 2026 13:03
91edb43 to
261f670
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test Improvement
Adds
tests/contributing-e2e-setup.test.mjs, which reads.github/workflows/ci.ymland pins the two instructions a contributor followsbefore the end-to-end suite can run at all. Both had drifted, and nothing in the
suite read either one.
The documented Playwright install cannot launch the browser.
CONTRIBUTING.md:205andAGENTS.md:57saynpx playwright install chromium.Playwright ships the browser bundle and its OS packages separately, and every
Playwright step in
ci.ymlinstalls both —--with-depson a cache miss(
ci.yml:139,ci.yml:199),install-depswhen the cached bundle is restored(
ci.yml:144,ci.yml:204). Reproduced on a clean Debian 13 host atmain=7ab301e: after exactly the documented setup,chrome-headless-shellexits127 with
error while loading shared libraries: libglib-2.0.so.0, andPlaywright reports
browserType.launch: Target page, context or browser has been closed— 306 failed, 16 passed. The 16 that pass are the ones that neveropen a page, so the failure reads as a broken checkout. With the packages
present the same commands give 322 passed.
The e2e coverage report is documented as having no thresholds.
CONTRIBUTING.md:234calls it "informational initially and has no percentagethresholds".
ci.yml:229-233renders it with--check-source 100 --check-source-regions 80 --require-source-files, andtests/e2e-coverage-gate.test.mjsalready holds floors under the first two. Thedocumented recipe passes none of the three, so a local render clears where CI
does not — the one thing a contributor runs it to rule out.
What the test asserts
Two one-way contracts, in the direction
tests/dev-environment.test.mjsalreadyuses for the Justfile: the docs must be at least as complete as the workflow,
and documenting more than CI runs stays green.
npx playwright install <browser>inCONTRIBUTING.md/AGENTS.mdinstalls system packages, either with
--with-depsor via a separateinstall-depsline, wheneverci.ymldoesCONTRIBUTING.md's fencednpm run report:e2e:coveragecommand passes every--check-*/--require-*flag thee2e-coveragejob passesBoth assertions fail against the documents as they stood (
--check-source --check-source-regions --require-source-filesreported missing, and bothinstall lines reported incomplete) and pass with the corrections in this PR.
Verification at
7ab301enode --test tests/contributing-e2e-setup.test.mjs— 2 passed; reverting onlythe two documents turns both red
npm run test:unit:coverage:check— exits0(2475/2476 source regions)npm run _check:format,markdownlint,cspellon the changed files — cleanCONTRIBUTING.md's own recipe — 322 passed,100.00% source lines, 91.41% source regions
Files
tests/contributing-e2e-setup.test.mjs(new),CONTRIBUTING.md,AGENTS.md.No file under
.github/is touched, and no open hold-gated PR shares a file:#1138 is confined to
tests/root-docs-links.test.mjs, andtests/e2e-coverage-gate.test.mjsandtests/dev-environment.test.mjsare leftalone.
Related Issue
Closes #1146
Human review required.
— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88
Extended per the consolidation plan: also documents the MDX fixture-docs mechanism (tests/e2e/fixtures/docs/, the e2e-coverage-fixtures route) beside the data overlays.
Closes #1146
Closes #1143