Skip to content

[quality] e2e accepted-region staleness does not distinguish a moved span from a vanished one #1811

Closed as not planned
Closed as not planned
[quality] e2e accepted-region staleness does not distinguish a moved span from a vanished one#1811
Labels
agent/qualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI

Description

@hivecommons-hive
hivecommons-hive
bot
opened on Oct 22, 2026

Finding

An accepted-regions entry whose source line shifts is reported with the same
words as one whose region is genuinely gone, and the lane's failure text tells
the reader to delete it.

A local end-to-end coverage run at 8abcb3f (362 chromium specs passed;
npm run build:e2e:coverage, npm run test:e2e:coverage,
npm run report:e2e:coverage) reproduces the six stale entries now failing the
End-to-end coverage / Render e2e coverage report gate. All six are line
shifts introduced by #1807 — identical start and end columns, identical height,
one line lower:

entry span region the run reports
DirectoryFreshness.js 18:66-18:71 20:66-20:71
DirectoryFreshness.js 29:12-29:28 31:12-31:28
DirectoryFreshness.js 43:12-43:31 45:12-45:31
DirectoryFreshness.js 48:8-48:41 50:8-50:41
profile-links.mjs 68:9-68:23 69:9-69:23
profile-links.mjs 68:23-68:31 69:23-69:31

Every one of those regions is still present, still zero-count, and still
unreachable for exactly the reason its entry records. Only the line moved.

The reporter says not a region this run reports, and the lane fails with
... are stale; remove them. Nothing distinguishes a moved region from a
vanished one, so the message reads as an instruction to delete the rationale.
Measured against the same run:

  • entries re-pointed → src files | 100.00 | 100.00 | 2292/2292 lines | 660/660 regions, no uncovered-region list
  • entries deleted → 654/660 regions and six entries back under Uncovered regions

The second outcome is the standing defect noise the ADR 0007 accepted-regions
amendment was written to remove (#1676, #1692, #1736). The gate still passes
either way against the 96% floor, so nothing catches the regression — the
registry just quietly loses an entry on every edit above an accepted region,
and this will recur on the next one.

ADR 0007 is not wrong here: a stale entry must stop matching and fail the lane.
What is missing is that the tool knows which case it is looking at and does not
say so.

Recommendation

Teach tests/tools/e2e-coverage-report.mjs to detect the shift and name it.
When a stale entry's file reports exactly one uncovered region with the same
start column, same end column and same height at a different line, report:

not a region this run reports; the same span is uncovered at 20:66-20:71 -- re-point this entry rather than deleting it

and carry the span as a shiftedSpan field on the JSON artifact. Change the
lane's failure heading from are stale; remove them to
no longer match this run; re-point the ones whose span moved and remove the rest.

The match must stay strict — identical columns, identical height, exactly one
candidate — because a guessed span in this registry silently excuses a region
nobody triaged. Where no single candidate matches, the message is unchanged.

Scope is one file plus its tests; this is a single deliverable and a single PR.

Not in scope

Re-pointing the six committed entries is #1809 / #1810's ground and is
deliberately excluded here, so the two changes do not collide over
tests/e2e/fixtures/accepted-regions.json. This issue touches
tests/tools/e2e-coverage-report.mjs and its tests only.

Priority

  • Impact: medium — the gate stays green, but the triage registry erodes silently and the uncovered-region list refills
  • Effort: low

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 8abcb3f

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Activity

  1. hivecommons-hive
    added
    testingApproved by a Hive merger/owner for auto-merge on green CI
    agent/qualityApproved by a Hive merger/owner for auto-merge on green CI
    on Oct 22, 2026
  2. hivecommons-hive
    linked a pull request that will close this issuetest: name the span a stale accepted region moved to #1812on Oct 22, 2026
  3. hivecommons-hive
    added
    stage/040-covered-by-prAn open PR covers this issue
    on Oct 22, 2026

  4. hivecommons-hive commented on Oct 23, 2026

    @hivecommons-hive
    hivecommons-hiveBot
    on Oct 23, 2026 – with hivecommons-hive
    ContributorAuthor

    Folded finding. This was filed as a separate issue, but 1 open agent-filed issue (plus 3 closed as completed in the last 14 days) already cite tests/tools/e2e-coverage-report.mjs, so it was folded into the oldest of them instead of opening another. Treat this issue as the tracker for that component: prefer one structural fix that covers every folded variant over a PR per variant.

    [quality] e2e-coverage-run.mjs's one unreachable region carries no residual note and no premise test

    Finding

    tests/tools/e2e-coverage-run.mjs has exactly one uncovered region — the re-throw in readProvenance:

    try {
  payload = JSON.parse(raw);
} catch (error) {
  if (!(error instanceof SyntaxError)) throw error;

    That arm is unreachable, not untested. The try wraps only JSON.parse(raw); raw is bound from readFile(..., 'utf8'), so it is always a string; and JSON.parse applied to a string throws nothing but SyntaxError (probed on Node 26 with 500k-deep nested arrays and objects, a lone surrogate, and truncated input — every one a SyntaxError). The reachable half, the SyntaxError warning, is already exercised by tests/e2e-coverage-runtime.test.mjs.

    Two things are missing around it:

    1. No residual note. tests/tools/e2e-coverage-report.mjs carries a residual note that names each unreachable arm by function and expression, explains why no probe could close it, and totals the regions. e2e-coverage-run.mjs carries nothing, so its 99.15% gets re-investigated from scratch every cycle.
    2. No premise test. The arm is dead because of a fact about the surrounding code, not about the arm. Widening the try to wrap the readFile, or a validator call, makes the re-throw live — and nothing fails. tests/e2e-coverage-error-arms.test.mjs is the repository's established answer to exactly this shape, and it has no counterpart here.

    This also leaves the harness gate argued from half its evidence. tests/coverage-gate-thresholds.test.mjs derives the --check-harness-file-regions floor from e2e-coverage-report.mjs's note alone, asserting it is "the lowest harness file" — but e2e-coverage-run.mjs sits at the same percentage with no note, so a probe that closed the report's residuals would let the floor be raised past an undocumented ceiling and break the gate.

    Measured on main (8abcb3f), Node v26.12.0, npm run test:unit:coverage:

    tests/tools/e2e-coverage-report.mjs  99.15% (580/585 regions; uncovered at 724 848 943 1274 1371)
tests/tools/e2e-coverage-run.mjs     99.15% (116/117 regions; uncovered at 158)
src files     100.00% lines | 100.00% regions (3110/3110)
harness files 100.00% lines |  99.78% regions (2732/2738)

    src/ is at 100% lines and 100% regions, so this is the remaining harness residual, not a source coverage gap. No end-to-end claim is made: this is a harness module and is not reachable from the browser lane.

    Recommendation

    • Add a residual note to the top of tests/tools/e2e-coverage-run.mjs in the same form tests/coverage-gate-thresholds.test.mjs already parses — "Together they are N regions, which caps this file at P% regions" — naming the arm by function and expression rather than by line.
    • Add a premise test, modelled on tests/e2e-coverage-error-arms.test.mjs, that fails if the premise stops holding: that readProvenance's try block wraps nothing but the JSON.parse assignment, that raw is bound from a 'utf8' read, and that JSON.parse of a string throws only SyntaxError.
    • Generalise tests/coverage-gate-thresholds.test.mjs so the --check-harness-file-regions floor is argued against every harness file that carries a residual note, taking the lowest ceiling, instead of assuming one file is the lowest.

    Priority

    • Impact: medium
    • Effort: low

    Filed by quality agent (hold-gated mode)

    🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 8abcb3f

    — hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

  5. mrbobbytables commented on Oct 23, 2026

    @mrbobbytables
    mrbobbytables
    on Oct 23, 2026
    Member

    Closing as not planned under ADR 0008 rule 2: the finding's only effect is a repository metric (accepted-region staleness reporting in the e2e coverage harness), which is out of agent remit. The paired PR #1812 is closed on the same basis. Under rule 4 this decision binds later findings on the same rule or file; leave a comment here rather than re-filing.

  6. mrbobbytables
    closed this as not plannedon Oct 23, 2026
  7. hivecommons-hive
    removed
    stage/040-covered-by-prAn open PR covers this issue
    on Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions