Skip to content

[quality] coverage executed on e2e fixture pages is not credited: 19 src regions stay at zero across four runs #1236

Closed

Description

@hivecommons-hive

Finding

The e2e coverage build grows a dedicated fixture page or variant build whenever a
src/** arm is unreachable from published content. Four consecutive CI runs of the
End-to-end coverage job report the same 19 regions uncovered, including regions a
merged, passing fixture spec was added specifically to reach.

artifact head branch src region %
11609771627 (run 37914482229-1) b00ff4e sec/csp-script-style-src 95.92
11605926538 438185f quality/e2e-per-file-region-floor 95.86
11599787569 03459d3 quality/test-docs-link-helpers 95.89
11595887431 046e2c5 quality/require-source-files-unmapped 95.84

report.json uncoveredRegions, byte-identical in all four:

src/components/CaseStudies/index.js                  46 64
src/components/CommunityPeople/index.js              134
src/components/GroupLinkStatus/index.js              23 33
src/components/hooks/useFocusTrap.js                 35
src/components/MemberDirectory/DirectoryFreshness.js 19 30 44 49
src/components/MetricsDashboard/index.js             65 85
src/components/PeopleFreshness/index.js              22
src/components/RadarReports/index.js                 6 24
src/components/ReferenceArchitectures/index.js       13 34

Why this is a defect and not an ordinary residual

src/components/CommunityPeople/index.js line 134 is
const people = peopleData.people[section] || [];. The || [] arm is exactly what
tests/e2e/community-people-unknown-section.spec.js exists for — its header names the
file and the line — and that spec landed in #1136 together with
tests/e2e/fixtures/docs/community-people.mdx, which renders
<CommunityPeople section="coverage-fixture-no-such-section" /> on a second docs
instance served at /e2e-coverage-fixtures/ when E2E_COVERAGE=1.

Three things are true at once in run 37914482229-1, and they cannot all be:

  1. The spec ran and passed. The run sealed status: passed, and
    report.json diagnostics is {"warnings": [], "errors": []}.
  2. The fixture page was visited and its V8 coverage was captured. Two capture
    files in the artifact carry
    http://localhost:3000/e2e-coverage-fixtures/community-people as their document
    entry — worker-0-parallel-0-retry-0-page-0-246037424792b923.json and
    …-4d50d7d0fd24511d.json, 11 coverage entries each. So the arm was executed in a
    browser whose coverage reached the artifact.
  3. The region is still reported at zero. src/components/CommunityPeople/index.js
    → uncoveredRegions: [134].

The report cannot be discarding it for want of a union, either: regions merge by
maximum across every script and artifact —
tests/tools/e2e-coverage-report.mjs:676-681, if (!existing || region.count > existing.count) — so one covering capture is enough on the merge side.

So coverage that was captured on a fixture page is not reaching the file's region
tally. The other eight files show the same shape: every one of the 19 regions is an arm
that only a variant build or a fixture page can execute, and not one of them is
credited. This reads as one attribution defect with nine symptoms rather than nine
unrelated gaps, which is why it is one issue.

I could not localise it further here. The two candidate mechanisms I could see by
reading, neither of which I am asserting:

  • the converted.sourceFiles.length > 1 && !converted.attributedPaths.has(path)
    guard at tests/tools/e2e-coverage-report.mjs:656-661, which drops a source
    entirely when a multi-source script does not list it in attributedPaths;
  • the contradiction/phantom folding (isContradictedRegion, isPhantomRegion)
    reached from the same loop.

Recommendation

One deliverable: make a region executed on a fixture page or a variant build count
towards its source file, and prove it with the case that is already instrumented.

  1. Reproduce from the artifact rather than from a fresh browser run — the capture
    directory 37914482229-1 plus the build/ tree of b00ff4e is a complete,
    deterministic input to node tests/tools/e2e-coverage-report.mjs --input … --build ….
    Narrow to the two …246037424792b923 / …4d50d7d0fd24511d captures and watch
    whether src/components/CommunityPeople/index.js region 134 is dropped before or
    during the merge.
  2. Fix the attribution path in tests/tools/e2e-coverage-report.mjs.
  3. Pin it with a unit test over collectE2ECoverage() driving a capture whose script
    maps to several sources, asserting the rare arm is credited.
  4. The gate then wants ratcheting: --check-source-regions is 95 in
    .github/workflows/ci.yml and the measured number is 95.84–95.92, so today the job
    is green with all 19 regions missing. (Ratcheting the threshold is a one-line
    workflow edit and must be done by a human or an agent with workflow-write
    permission — see Note on scope below.)

Evidence and provenance

  • Unit. npm run test:unit:coverage (TZ=UTC node tests/tools/coverage-report.mjs),
    node v26.10.0, run locally 2026-10-09 at 593b9fe. All nine files report
    100.00 | 100.00; src files totals 100.00 | 99.96 (2595/2596 regions). So every
    region above is unit-covered, which is why this is priority 2 and not priority 1.
  • End-to-end. The four e2e-coverage artifacts tabulated above, produced by the
    End-to-end coverage job in .github/workflows/ci.yml
    (npm run build:e2e:coverage → npm run test:e2e:coverage →
    npm run report:e2e:coverage -- --check-source 100 --check-source-regions 95).
    Artifact 11609771627 was downloaded and both report.json and the raw capture
    directory inspected directly; the other three were read for report.json only.
  • Not combined. Unit coverage is node's own --experimental-test-coverage data and
    the e2e side is V8-to-istanbul over built bundles; the two were analysed separately.
  • Not reproduced locally. The e2e half could not be re-run on this machine —
    chrome-headless-shell fails to start with
    error while loading shared libraries: libglib-2.0.so.0, and installing system
    packages is not available here. Every e2e claim above therefore rests on the CI
    artifacts, not on a local run.

Note on scope — no PR from me

Step 4 edits .github/workflows/ci.yml, which a contributor-tier token cannot push;
GitHub rejects a diff touching .github/workflows/** server-side. More importantly,
steps 1-3 cannot be verified here for the reason in the provenance note above: a fix
to the attribution path has to be shown to move the measured number, and that needs a
browser this environment cannot launch. Opening a PR with an unverified fix would be
worse than filing this. The reproduction in step 1 is deterministic from the artifact,
so whoever picks it up does not need to guess.

Priority

  • Impact: medium (the e2e gate is green while nine components' rare-path arms are
    uncredited, so every fixture page and variant build added to reach them is currently
    buying nothing measurable)
  • Effort: medium

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 593b9fe

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Activity

  1. added
    qualityApproved by a Hive merger/owner for auto-merge on green CI
    testingApproved by a Hive merger/owner for auto-merge on green CI
    agent/qualityApproved by a Hive merger/owner for auto-merge on green CI
    hive/covered-by-prHive verified that an open PR references or claims this issue; still actionable until confirmed
    hive/likely-doneHive verified that a merged PR references or claims this issue; pending confirmation
    and removed
    hive/covered-by-prHive verified that an open PR references or claims this issue; still actionable until confirmed
    on Oct 9, 2026
  2. hivecommons-hive commented on Oct 9, 2026

    @hivecommons-hive
    ContributorAuthor

    A merged PR referenced this issue without closing it. Carrying the remainder back here so the issue timeline shows what is still open.

    Remainder from the PR body:

    src/components/CommunityPeople/index.js:134 is still uncovered after this fix, and ratcheting --check-source-regions in .github/workflows/ci.yml needs workflow-write permission this agent does not have

    This comment is edited in place by the task-list sweep on every cycle; it is not duplicated.

  3. hivecommons-hive commented on Oct 9, 2026

    @hivecommons-hive
    ContributorAuthor

    What to reply

    • /hive approve: go with the recommendation
    • /hive decision <your instructions>: go ahead, following your instructions
    • /hive help: show these again

    If this issue is waiting only for reporter confirmation after a merged fix, the reporter or a maintainer can reply /fixed instead.

    Only a maintainer with write access can un-park; everything else is ignored. Full command reference: https://github.com/hivecommons/hive/blob/v5/src/docs/maintainer-commands.md.

  4. hivecommons-hive commented on Oct 9, 2026

    @hivecommons-hive
    ContributorAuthor

    Verified against merged #1240: the source-boundary defect it identified is fixed — convertScript now credits runs that span a boundary, and src/components/MemberDirectory/DirectoryFreshness.js drops lines 30, 44 and 49 from uncoveredRegions. That was the bulk of the 19.

    Two things remain and neither is closeable from this lane:

    1. src/components/CommunityPeople/index.js:134 is still reported uncovered. test(e2e): credit coverage for runs that span a source boundary #1240 says so explicitly, and I could not take it further here: reproducing it needs the browser, and chromium will not launch in this environment — chrome-headless-shell exits with error while loading shared libraries: libglib-2.0.so.0, and installing the Playwright system dependencies needs root. So I am recording the state rather than guessing at a cause. The fixture (tests/e2e/fixtures/docs/community-people.mdx) and its spec are both present and the spec passes, so the symptom is unchanged from the issue text.
    2. Ratcheting --check-source-regions past 95 (the measured number is now 97.59) edits .github/workflows/ci.yml; a contributor-tier App token is rejected server-side on any diff touching .github/workflows/**.

    Labelled hive/verified-open so this stops being re-asked. Both remainders need a human or a workflow-write agent.


    🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 09b635a

    — hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

  5. added a commit that references this issue on Oct 11, 2026
    4184f33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CIhive/hosted-available-lke648397-260827-5n31Approved by a Hive merger/owner for auto-merge on green CIhive/likely-doneHive verified that a merged PR references or claims this issue; pending confirmationhive/verified-openneeds-humanqualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions