Report QA coverage and scope: say which worksheets were not assessed - #10
Open
roosch269 wants to merge 4 commits into
Open
Report QA coverage and scope: say which worksheets were not assessed#10roosch269 wants to merge 4 commits into
roosch269 wants to merge 4 commits into
Conversation
quick update to schema to include column colours
Updated SVG Mapping Limits
The QA report is silent about worksheets it never looked at. The schema
describes 12 worksheets; a real COBie workbook can carry Job, Resource,
Spare, Impact, Assembly, Connection or Issue, and today those are parsed
by nothing, checked by nothing, and mentioned nowhere in the output. A
reader sees a score and no indication that part of their deliverable was
outside it.
Nothing computed coverage at all: the workbook knows its sheet names and
the schema knows its own, and the difference was never taken.
Adds a neutral "Coverage and scope" block to the QA view and the PDF
cover:
The active profile describes X of Y recognised COBie worksheets
present; Z additional worksheets were outside this profile. Does not
affect QA score.
The wording is deliberate. "Covers X of Y" would count a described but
empty sheet in X while the same block reports that it had no data rows
to assess, which invites coverage to be read as data assessment.
Three categories, because a plain present-minus-described set is not
true. It would report a client's own "Notes" or "Revisions" tab as COBie
work that failed assessment, which is a false statement about their data:
- recognised COBie, not described by the profile -> Not assessed
- not a COBie worksheet -> Additional worksheet
- described but no data rows -> Present, empty
Two details that are wrong the obvious way round:
- the described set is the profile's full sheet list, not the
stage-filtered one. A sheet excluded by the selected stage is
described, not unassessed; using stageSheets would make coverage
change every time the user switches stage.
- iteration is per source workbook, not per logical facility.
_qaLogicalFacilityRows() keeps one representative per facility
identity, so a second file describing the same facility would
disappear from the report.
Coverage is descriptive, never evaluative. It is kept in its own object
and never touches qaFindings or qaRuleResults, so it cannot reach
_qaScoreTally() and move a score or the donut. It carries no severity
colour and no icon. tests/qa-coverage-regression.js asserts that every
scored value is unchanged before and after coverage is computed and
rendered, alongside multi-workbook, case-varied, empty-sheet, non-COBie
tab, unprofiled-COBie-sheet, stage-invariance and HTML-escaping checks.
The inventory/profile alignment check reads the profile through
_qaParseSchema() rather than scraping the XML, and asserts the described
set is non-empty before judging it. An earlier version matched
"<worksheet" where the schema writes "<sheet", so it compared an empty
set and passed unconditionally: a gate that could not fail. Both arms are
mutation-proven -- removing Picklist from the inventory names it as a
stray, and emptying the described set trips the non-empty control.
Also fixes npm test on macOS. tests/schema-regression.js spawned bare
"python", which macOS has not shipped since 12.3, so the suite died at
the first file with spawnSync python ENOENT before running anything. It
now probes for a working interpreter; PYTHON=... still overrides.
Found by running the built tool against the repository's own example
workbook in a real browser, which the unit tests could not see:
- the headline rendered as "present;1additional worksheet wasoutside
this profile" -- template literal boundaries had eaten the spaces
- the source filename repeated on all eight entries of a single-file
report, which is noise rather than provenance; it now appears only
when more than one workbook is loaded
- Instruction was reported as an additional worksheet outside the
COBie profile. It is part of the COBie template, but carries
guidance rather than project data, so it is now excluded from the
coverage count entirely rather than being called either outside the
profile or unassessed.
The suite was 9/9 green through all three, because structure assertions
cannot read a sentence. A10-A12 read the rendered text instead, and each
is mutation-proven against the defect it was written for.
Verified: all four suites green with no PYTHON set, coverage suite 0/9
against this commit's parent and 9/9 here, and two consecutive release
builds byte-identical.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The QA report says nothing about worksheets it never looked at. The schema describes 12; a COBie workbook can also carry Job, Resource, Spare, Impact, Assembly, Connection and Issue, and those are currently parsed by nothing, checked by nothing, and absent from the output. Coverage was never computed at all — the workbook knows its sheet names, the schema knows its own, and the difference was never taken.
Adds a Coverage and scope block to the QA view and the PDF cover. Against the example workbook in this repo it reads:
Three categories, because a plain present-minus-described set isn't true — it reports a client's own
Notestab as COBie work that failed assessment:Instructionis excluded from the count: it's part of the COBie template but carries guidance, not project data, so calling it either "outside the profile" or "not assessed" would be wrong.Two things deliberately not done the obvious way:
stageSheetswould make coverage change whenever the user switches stage_qaLogicalFacilityRows()keeps one representative per facility identity, so a second file describing the same facility would disappear from the reportCoverage is descriptive, never evaluative. It lives in its own object and never touches
qaFindingsorqaRuleResults, so it cannot reach_qaScoreTally()and move a score or the donut. No severity colour, no icon.Also fixes
npm teston macOS.tests/schema-regression.jsspawned barepython, which macOS hasn't shipped since 12.3, so the suite died at the first file withspawnSync python ENOENTbefore running anything. It now probes for a working interpreter;PYTHON=still overrides.12 checks in
tests/qa-coverage-regression.js, wired intonpm test. Score invariance asserts every tallied value is byte-identical before and after coverage is computed and rendered. Three of them exist because running the built tool in a real browser against the example workbook found what the structural tests couldn't see — spacing in the rendered sentence, the filename repeated on every entry, and theInstructionmisclassification.Verified: four suites green with no
PYTHONset, and two consecutive release builds byte-identical.Targeted at
devso the call onmainstays yours.devandmaindiffer only by three merge commits with no content between them, so this diff is identical either way — 8 files, +447/-16.