[quality] add cross-catalog invariant tests — lock current state, catch new regressions - #3278
Conversation
… new regressions
Adds scripts/__tests__/cross-catalog-invariants.test.mjs — 12 tests
covering the cross-catalog invariants that the mission generators
depend on but that scripts/__tests__/catalog-invariants.test.mjs
does not cover:
* CNCF_PROJECTS ∩ OTHER_PROJECTS by name (silent clobber in
generate-cncf-missions.mjs:1441 ALL_PROJECTS merge)
* CNCF_PROJECTS ∩ K8S_PLATFORMS by name (defensive)
* CNCF internal repo duplicates
* CNCF_PROJECTS ∩ OTHER_PROJECTS by repo (double-crawl)
* K8S_PLATFORMS ∩ OTHER_PROJECTS by repo
* OTHER_PROJECTS.category ⊆ Object.keys(CATEGORY_TO_DIR)
* OTHER_PROJECTS.sources.{stackoverflow.tags,reddit.subreddits}
non-empty string[] when present
While writing the suite, five distinct current-state violations
surfaced on master (see #3277 for the full inventory). Rather than
silently patch the data or leave the suite red, each violation is
captured in a named KNOWN_*_DEBT allowlist so:
* master stays green today
* every NEW collision or unmapped category is caught immediately
* companion 'allowlist stays honest' tests prevent bit-rot — an
entry cannot be left in an allowlist after the underlying debt
is fixed
No production data change; no workflow change (scripts-tests.yml
already runs 'vitest run' via 'npm test', which auto-discovers new
*.test.mjs files under scripts/__tests__/).
Refs #3277
Signed-off-by: kubestellar-hive[bot] <223556219+Copilot@users.noreply.github.com>
|
Important Held for human sign-off on the direction, not on the code. This PR's only tracked rationale is #3277, which the hive filed itself — issue #3277 was filed by kubestellar-hive[bot] and no human has acknowledged it. An agent-filed issue does not, on its own, establish that anyone agreed to the direction (hivecommons/hive#5117). The change may well be right; nothing here is a review of it. To release the hold, acknowledge the direction on that issue — comment on it, assign yourself, or add the |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Direction acknowledged by hub admin ( |
|
Thank you for your contribution! Your PR has been merged. Check out what's new:
Stay connected: Slack #kubestellar-dev | Multi-Cluster Survey |
Test Improvement
Adds
scripts/__tests__/cross-catalog-invariants.test.mjs— 12 tests covering cross-catalog invariants that the mission generators depend on but thatscripts/__tests__/catalog-invariants.test.mjsdoes not check.Why
Two generators merge these three catalogs:
generate-cncf-missions.mjs:1441—[...CNCF_PROJECTS, ...OTHER_PROJECTS.map(...)], keyed by.name, usesCATEGORY_TO_DIR[category]for thefixes/subdirectorygenerate-platform-missions.mjs:508—[...K8S_PLATFORMS, ...OTHER_PROJECTS], keyed by.nameSilent violations of the merge assumptions corrupt generated output without failing CI today. The existing per-catalog suite plus one cross-cut (K8S ∩ OTHER by name) leaves the rest unguarded.
What this PR adds
CNCF ∩ OTHERby name,CNCF ∩ K8Sby name (defensive)CNCF ∩ OTHERby repo,K8S ∩ OTHERby repo.category ⊆ CATEGORY_TO_DIR keys,sources.stackoverflow.tagsshape,sources.reddit.subredditsshapeWhile writing the suite, five distinct pre-existing data-debt violations surfaced on
master(details filed as #3277). Rather than silently patch data or leave the suite red, each is captured in a namedKNOWN_*_DEBTallowlist — the same pattern already used bycatalog-invariants.test.mjs'sKNOWN_TRAILING_HYPHEN_DEBT:KNOWN_CNCF_OTHER_NAME_COLLISIONS— 6 entriesKNOWN_CNCF_K8S_NAME_COLLISIONS— 4 entriesKNOWN_CNCF_INTERNAL_REPO_DUPES— 1 entryKNOWN_CNCF_OTHER_REPO_COLLISIONS— 5 entriesKNOWN_UNMAPPED_OTHER_CATEGORIES— 27 distinct categoriesEach allowlist is paired with a companion
... stays honesttest that fails if an allowlist entry no longer describes a real violation — so entries cannot be left in the list after the underlying debt is paid down.Verification
The new suite alone:
Test Files 1 passed | Tests 12 passed. The full existing 1146-test suite still passes end-to-end.Scope
cncf-projects.mjs,k8s-platforms.mjs,other-projects.mjsare untouched..github/workflows/scripts-tests.ymlalready runsnpm test(=vitest run) which auto-discovers*.test.mjsfiles underscripts/__tests__/.scripts/__tests__/catalog-invariants.test.mjs.Refs #3277
Filed by quality agent (ACMM L4/L6 — full mode)
— hive: agent=quality backend=copilot model=claude-opus-4.7