Skip to content

Report QA coverage and scope: say which worksheets were not assessed - #10

Open
roosch269 wants to merge 4 commits into
digital-guerrilla:devfrom
roosch269:feat/qa-coverage-and-scope
Open

Report QA coverage and scope: say which worksheets were not assessed#10
roosch269 wants to merge 4 commits into
digital-guerrilla:devfrom
roosch269:feat/qa-coverage-and-scope

Conversation

@roosch269

@roosch269 roosch269 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

The QA report says nothing about worksheets it never looked at. The schema describes 12; a COBie workbook can also carry Job, Resource, Spare, Impact, Assembly, Connection and Issue, and those are currently parsed by nothing, checked by nothing, and absent from the output. Coverage was never computed at all — the workbook knows its sheet names, the schema knows its own, and the difference was never taken.

Adds a Coverage and scope block to the QA view and the PDF cover. Against the example workbook in this repo it reads:

The active profile describes 12 of 19 recognised COBie worksheets present. Does not affect QA score.

Not assessed — recognised COBie worksheets this profile does not describe: Assembly — 0 rows; Connection — 0 rows; Impact — 0 rows; Issue — 0 rows; Job — 0 rows; Resource — 0 rows; Spare — 0 rows

Three categories, because a plain present-minus-described set isn't true — it reports a client's own Notes tab as COBie work that failed assessment:

  • recognised COBie, not described by the profile → Not assessed
  • not a COBie worksheet → Additional worksheet outside this profile
  • described but no data rows → Present, empty

Instruction is excluded from the count: it's part of the COBie template but carries guidance, not project data, so calling it either "outside the profile" or "not assessed" would be wrong.

Two things deliberately not done the obvious way:

  • the described set is the profile's full sheet list, not the stage-filtered one — a sheet excluded by the selected stage is described, not unassessed, and using stageSheets would make coverage change whenever the user switches stage
  • iteration is per source workbook, not per logical facility — _qaLogicalFacilityRows() keeps one representative per facility identity, so a second file describing the same facility would disappear from the report

Coverage is descriptive, never evaluative. It lives in its own object and never touches qaFindings or qaRuleResults, so it cannot reach _qaScoreTally() and move a score or the donut. No severity colour, no icon.

Also fixes npm test on macOS. tests/schema-regression.js spawned bare python, which macOS hasn't shipped since 12.3, so the suite died at the first file with spawnSync python ENOENT before running anything. It now probes for a working interpreter; PYTHON= still overrides.

12 checks in tests/qa-coverage-regression.js, wired into npm test. Score invariance asserts every tallied value is byte-identical before and after coverage is computed and rendered. Three of them exist because running the built tool in a real browser against the example workbook found what the structural tests couldn't see — spacing in the rendered sentence, the filename repeated on every entry, and the Instruction misclassification.

Verified: four suites green with no PYTHON set, and two consecutive release builds byte-identical.


Targeted at dev so the call on main stays yours.

dev and main differ only by three merge commits with no content between them, so this diff is identical either way — 8 files, +447/-16.

Andrew-Waring and others added 4 commits August 16, 2026 21:14
quick update to schema to include column colours
The QA report is silent about worksheets it never looked at. The schema
describes 12 worksheets; a real COBie workbook can carry Job, Resource,
Spare, Impact, Assembly, Connection or Issue, and today those are parsed
by nothing, checked by nothing, and mentioned nowhere in the output. A
reader sees a score and no indication that part of their deliverable was
outside it.

Nothing computed coverage at all: the workbook knows its sheet names and
the schema knows its own, and the difference was never taken.

Adds a neutral "Coverage and scope" block to the QA view and the PDF
cover:

  The active profile describes X of Y recognised COBie worksheets
  present; Z additional worksheets were outside this profile. Does not
  affect QA score.

The wording is deliberate. "Covers X of Y" would count a described but
empty sheet in X while the same block reports that it had no data rows
to assess, which invites coverage to be read as data assessment.

Three categories, because a plain present-minus-described set is not
true. It would report a client's own "Notes" or "Revisions" tab as COBie
work that failed assessment, which is a false statement about their data:

  - recognised COBie, not described by the profile -> Not assessed
  - not a COBie worksheet                          -> Additional worksheet
  - described but no data rows                     -> Present, empty

Two details that are wrong the obvious way round:

  - the described set is the profile's full sheet list, not the
    stage-filtered one. A sheet excluded by the selected stage is
    described, not unassessed; using stageSheets would make coverage
    change every time the user switches stage.
  - iteration is per source workbook, not per logical facility.
    _qaLogicalFacilityRows() keeps one representative per facility
    identity, so a second file describing the same facility would
    disappear from the report.

Coverage is descriptive, never evaluative. It is kept in its own object
and never touches qaFindings or qaRuleResults, so it cannot reach
_qaScoreTally() and move a score or the donut. It carries no severity
colour and no icon. tests/qa-coverage-regression.js asserts that every
scored value is unchanged before and after coverage is computed and
rendered, alongside multi-workbook, case-varied, empty-sheet, non-COBie
tab, unprofiled-COBie-sheet, stage-invariance and HTML-escaping checks.

The inventory/profile alignment check reads the profile through
_qaParseSchema() rather than scraping the XML, and asserts the described
set is non-empty before judging it. An earlier version matched
"<worksheet" where the schema writes "<sheet", so it compared an empty
set and passed unconditionally: a gate that could not fail. Both arms are
mutation-proven -- removing Picklist from the inventory names it as a
stray, and emptying the described set trips the non-empty control.

Also fixes npm test on macOS. tests/schema-regression.js spawned bare
"python", which macOS has not shipped since 12.3, so the suite died at
the first file with spawnSync python ENOENT before running anything. It
now probes for a working interpreter; PYTHON=... still overrides.
Found by running the built tool against the repository's own example
workbook in a real browser, which the unit tests could not see:

  - the headline rendered as "present;1additional worksheet wasoutside
    this profile" -- template literal boundaries had eaten the spaces
  - the source filename repeated on all eight entries of a single-file
    report, which is noise rather than provenance; it now appears only
    when more than one workbook is loaded
  - Instruction was reported as an additional worksheet outside the
    COBie profile. It is part of the COBie template, but carries
    guidance rather than project data, so it is now excluded from the
    coverage count entirely rather than being called either outside the
    profile or unassessed.

The suite was 9/9 green through all three, because structure assertions
cannot read a sentence. A10-A12 read the rendered text instead, and each
is mutation-proven against the defect it was written for.

Verified: all four suites green with no PYTHON set, coverage suite 0/9
against this commit's parent and 9/9 here, and two consecutive release
builds byte-identical.
@roosch269
roosch269 changed the base branch from main to dev August 31, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants