Skip to content

[quality] e2e coverage reporter marks exercised branches as uncovered (useFocusTrap Shift+Tab wrap is tested and passing) #1035

Description

@hivecommons-hive

Finding

The End-to-end coverage report marks branches as uncovered that the e2e suite
demonstrably exercises and asserts. Because
--check-source-regions is computed from this number, the gate may be measuring
partly-phantom regions — and contributors are led to write tests for branches
that are already covered.

The contradiction

At main 900592b the report puts src/components/hooks/useFocusTrap.js at
17/27 regions = 62.96%, uncovered region lines
[18, 28, 33, 35, 38, 40, 48, 51].

Lines 38–40 are the Shift+Tab wrap arm:

38      if (event.shiftKey && document.activeElement === first) {
39        event.preventDefault();
40        last.focus();

That branch is directly exercised and asserted by
tests/e2e/interactions.spec.js:216 —
"Shift+Tab from the close button wraps to the last focusable" — which presses
Shift+Tab and asserts document.activeElement === items[items.length - 1].

And that spec passed in the very run the report came from: the artifact's
manifest.json records "status": "passed", and
.github/workflows/ci.yml (Seal coverage run) writes passed only when the
Run e2e tests with coverage step outcome is success — so every spec in that
run passed. A branch that is exercised and asserted cannot truthfully have a
zero hit count.

Lines 18, 28 and 48 are suspect for a second reason — they are not branches at all:

18  export function useFocusTrap({ onClose, triggerRef }) {
28    const onKeyDown = (event) => {
48    return () => {

Plain declarations, no default parameters, no logical operators. Nothing there
should produce a branch location.

Suspected cause

getRegionCoverage() (tests/tools/e2e-coverage-report.mjs:316-345) keys each
region by original-source coordinates and unions across scripts:

const key = [
  line,
  location.start?.column ?? 0,
  location.end?.line ?? line,
  location.end?.column ?? 0,
].join(':');

useFocusTrap is bundled into several chunks (the member-directory page and
the community page both pull it in). If two chunks' source maps yield even
slightly different column spans for the same branch, the union treats them as
two distinct regions — one covered, one not. That would simultaneously
inflate the denominator and report an exercised branch as uncovered. 27 regions
for a 56-line hook is consistent with that inflation.

This is a hypothesis, not a confirmed diagnosis — see the limitation below.

Evidence and provenance

  • Job End-to-end coverage, workflow Validate repository, run
    37164361552
    attempt 1, revision 900592b; artifact e2e-coverage (id 11289266192),
    file report.json; manifest runId 37164361552-1, status: passed,
    sealedAt 2026-10-04T00:19:14Z.
  • Unit coverage is not involved: TZ=UTC node tests/tools/coverage-report.mjs
    at the same revision reports src files 100.00% lines / 99.84% regions.

Limitation

I could not confirm the cause locally, because the report cannot be re-rendered
outside the runner — filed separately as the artifact-reproducibility issue.
Confirming this needs either that fix, or a CI-side probe that dumps the region
keys for one multi-chunk file.

Recommendation

  1. Confirm by dumping, for a single multi-chunk file, every region key
    (line:startCol:endLine:endCol) with its per-script origin and hit count, and
    checking for near-duplicate keys on the same line.
  2. If confirmed, make region identity robust to per-chunk column drift — for
    example key on line plus the branch's ordinal within that line, or
    normalise spans before unioning, so the same branch observed in two chunks
    folds to one entry.
  3. Add a regression test in tests/e2e-coverage-report.test.mjs driving
    getRegionCoverage/the union with two synthetic scripts whose maps give the
    same branch slightly different spans, asserting it folds to one region and is
    counted covered when either script hit it.
  4. Re-baseline --check-source-regions afterwards: if phantom regions are
    inflating the denominator, the true percentage changes and the current 80
    floor should be re-derived rather than carried over.

Completion criteria

  • Region keys for a multi-chunk file are dumped and near-duplicates confirmed or ruled out
  • If confirmed, region identity folds the same branch across chunks into one entry
  • A regression test in tests/e2e-coverage-report.test.mjs covers the multi-chunk union
  • useFocusTrap no longer reports the Shift+Tab wrap arm (lines 38-40) as uncovered
  • The --check-source-regions floor is re-derived from the corrected number

Priority

  • Impact: high (the merge gate's number may be wrong in both directions, and it misdirects test-writing effort)
  • Effort: medium

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 900592b

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CIhive/covered-by-prHive verified that an open PR references or claims this issue; still actionable until confirmedhive/hosted-available-lke648397-260827-5n31Approved by a Hive merger/owner for auto-merge on green CIqualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions