Skip to content

test: pin the documented e2e setup against what ci.yml actually runs - #1147

Merged
mrbobbytables merged 2 commits into
mainfrom
quality/test-contributing-e2e-setup
Oct 7, 2026
Merged

mrbobbytables merged 2 commits into
mainfrom
quality/test-contributing-e2e-setup

Conversation

@hivecommons-hive

@hivecommons-hive hivecommons-hive Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Test Improvement

Adds tests/contributing-e2e-setup.test.mjs, which reads
.github/workflows/ci.yml and pins the two instructions a contributor follows
before the end-to-end suite can run at all. Both had drifted, and nothing in the
suite read either one.

The documented Playwright install cannot launch the browser.
CONTRIBUTING.md:205 and AGENTS.md:57 say npx playwright install chromium.
Playwright ships the browser bundle and its OS packages separately, and every
Playwright step in ci.yml installs both — --with-deps on a cache miss
(ci.yml:139, ci.yml:199), install-deps when the cached bundle is restored
(ci.yml:144, ci.yml:204). Reproduced on a clean Debian 13 host at main =
7ab301e: after exactly the documented setup, chrome-headless-shell exits
127 with error while loading shared libraries: libglib-2.0.so.0, and
Playwright reports browserType.launch: Target page, context or browser has been closed — 306 failed, 16 passed. The 16 that pass are the ones that never
open a page, so the failure reads as a broken checkout. With the packages
present the same commands give 322 passed.

The e2e coverage report is documented as having no thresholds.
CONTRIBUTING.md:234 calls it "informational initially and has no percentage
thresholds". ci.yml:229-233 renders it with --check-source 100 --check-source-regions 80 --require-source-files, and
tests/e2e-coverage-gate.test.mjs already holds floors under the first two. The
documented recipe passes none of the three, so a local render clears where CI
does not — the one thing a contributor runs it to rule out.

What the test asserts

Two one-way contracts, in the direction tests/dev-environment.test.mjs already
uses for the Justfile: the docs must be at least as complete as the workflow,
and documenting more than CI runs stays green.

  • every npx playwright install <browser> in CONTRIBUTING.md/AGENTS.md
    installs system packages, either with --with-deps or via a separate
    install-deps line, whenever ci.yml does
  • CONTRIBUTING.md's fenced npm run report:e2e:coverage command passes every
    --check-*/--require-* flag the e2e-coverage job passes

Both assertions fail against the documents as they stood (--check-source --check-source-regions --require-source-files reported missing, and both
install lines reported incomplete) and pass with the corrections in this PR.

Verification at 7ab301e

  • node --test tests/contributing-e2e-setup.test.mjs — 2 passed; reverting only
    the two documents turns both red
  • npm run test:unit:coverage:check — exits 0 (2475/2476 source regions)
  • npm run _check:format, markdownlint, cspell on the changed files — clean
  • full e2e coverage pipeline from CONTRIBUTING.md's own recipe — 322 passed,
    100.00% source lines, 91.41% source regions

Files

tests/contributing-e2e-setup.test.mjs (new), CONTRIBUTING.md, AGENTS.md.
No file under .github/ is touched, and no open hold-gated PR shares a file:
#1138 is confined to tests/root-docs-links.test.mjs, and
tests/e2e-coverage-gate.test.mjs and tests/dev-environment.test.mjs are left
alone.

Related Issue

Closes #1146


Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88


Extended per the consolidation plan: also documents the MDX fixture-docs mechanism (tests/e2e/fixtures/docs/, the e2e-coverage-fixtures route) beside the data overlays.

Closes #1146
Closes #1143

@hivecommons-hive hivecommons-hive Bot added the hold label Oct 7, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

quality and others added 2 commits October 7, 2026 08:03
CONTRIBUTING.md and AGENTS.md both tell a contributor to run
'npx playwright install chromium'. Playwright ships the browser bundle
and its OS packages separately, and every Playwright step in ci.yml
installs both. On a clean Debian 13 host the documented command alone
leaves chrome-headless-shell exiting 127 on libglib-2.0.so.0 and 306 of
322 specs failing with 'Target page, context or browser has been
closed', which names nothing a reader can act on.

CONTRIBUTING.md also calls the e2e coverage report informational and
without percentage thresholds, but the e2e-coverage job renders it with
--check-source 100, --check-source-regions 80 and
--require-source-files, floors already pinned by
tests/e2e-coverage-gate.test.mjs. The documented recipe passes none of
them, so it clears locally where CI does not.

Correct both documents and add tests/contributing-e2e-setup.test.mjs,
which reads ci.yml and asserts one-way that the docs are at least as
complete as the workflow.

Closes #1146

Signed-off-by: quality <quality@hive.kubestellar.io>
The coverage section documented only the data-overlay mechanism; a
branch whose props come from generated MDX needs the fixture-docs
instance added by #1083. Closes #1143.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
@mrbobbytables
mrbobbytables force-pushed the quality/test-contributing-e2e-setup branch from 91edb43 to 261f670 Compare October 7, 2026 13:03
@mrbobbytables
mrbobbytables added this pull request to the merge queue Oct 7, 2026
Merged via the queue into main with commit 2f20ca7 Oct 7, 2026
7 checks passed
@mrbobbytables
mrbobbytables deleted the quality/test-contributing-e2e-setup branch October 7, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

1 participant