Skip to content

ci: gate all scripts test suites in CI and pre-commit (closes #2540) - #2685

Merged
dem-extra1 merged 6 commits into
mainfrom
fix/gate-test-suites-coverage
Aug 31, 2026
Merged

ci: gate all scripts test suites in CI and pre-commit (closes #2540)#2685
dem-extra1 merged 6 commits into
mainfrom
fix/gate-test-suites-coverage

Conversation

@dem-extra1

@dem-extra1 dem-extra1 commented Aug 30, 2026

Copy link
Copy Markdown
Collaborator

Closes #2540

Summary of changes

  • Missing test suites wired in CI: Added scripts/test_doctor.py and scripts/test_fences.py to .github/workflows/validate.yml.
  • Test suite discovery and coverage checker: Added scripts/check-test-suites-covered.py which discovers all scripts/test_*.py files and verifies that each is executed on an active run: line in validate.yml, failing if any test suite is omitted.
  • Unit test suite: Added scripts/test_check_test_suites_covered.py (13 tests) verifying discovery, active run command extraction, comment exclusion, and CLI behaviors.
  • Pre-commit hooks: Wired doctor-tests, fences-tests, check-test-suites-covered, and test-check-test-suites-covered in .pre-commit-config.yaml.
  • Link fix: Fixed broken relative link in shared/workflow/fully-clean.md (../../skills/mwc/SKILL.md).

Design Rationale

Per #2540, we evaluated wildcard auto-discovery vs. explicit step enumeration. We chose explicit step enumeration in validate.yml paired with automated discovery verification in check-test-suites-covered.py. This retains individual step execution timings and collapsible log sections in GitHub Actions UI for easier pinpoint debugging, while guaranteeing 100% test suite coverage in CI.

@dem-extra1 dem-extra1 changed the title ci: gate all script test suites in CI and pre-commit (closes #2540) ci: gate all scripts test suites in CI and pre-commit (closes #2540) Aug 31, 2026
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dem-extra1
dem-extra1 force-pushed the fix/gate-test-suites-coverage branch from 6b945f0 to 1731ee9 Compare August 31, 2026 00:26
@dem-extra1

Copy link
Copy Markdown
Collaborator Author

Addressed both review findings in commit {1ee93:

  1. **Rebased onto current \main**: Resolved the merge conflict with \main\ (commit LJ1d37c\ which retired local Ollama inference and deleted \ est_check_ollama_locality.py). The gate \scripts/check-test-suites-covered.py\ now discovers and verifies all 41 test suites cleanly.
  2. Design rationale: We chose explicit step enumeration in \�alidate.yml\ backed by automated discovery enforcement in \check-test-suites-covered.py\ rather than a single wildcard/runner step. This retains individual step timings, per-suite failure isolation, and clear log collapsible groups in GitHub Actions UI, while eliminating the risk of un-gated suites through strict CI/pre-commit enforcement.

Posted by Antigravity (AI agent) --- not written by a human.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dem-extra1
dem-extra1 marked this pull request as ready for review August 31, 2026 00:35
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

👀 Claude Review Dispatchedrun reviewing PR #2685 at 2c71f0b.

@github-actions

Copy link
Copy Markdown
Contributor

Claude finished reviewView run


All checks confirm this round introduces no new content to evaluate.

Review: #2685 — "ci: gate all scripts test suites in CI and pre-commit"

Confirming review — no new findings. The commit range for this round (1731ee93..2c71f0b1) contains only a merge of main into the PR branch, pulling in an unrelated commit (19e7e28, "ums: record lean VS Code profiles and glab list syntax", #2694) that touches only memories/MEMORY.md, memories/gitlab.md, and memories/vscode-profiles.md — none of which this PR's own diff (workflow gating, check-test-suites-covered.py, pre-commit config) touches or is affected by. git diff --stat 1731ee93 2c71f0b1 confirms exactly those three files changed, 27 insertions, 0 deletions, nothing under .github/, scripts/, or .pre-commit-config.yaml.

This confirms the prior round posted 2026-08-31T00:36:52Z, verdict Ready for merge, which itself confirmed the two rounds before it. Re-verified independently this round:

  • .github/workflows/validate.yml on disk is byte-identical to origin/main.
  • scripts/test_check_test_suites_covered.py (13 tests), scripts/test_doctor.py (7 tests), and scripts/test_fences.py (25 checks) all pass locally.
  • gh pr view reports state: OPEN, merged: false, mergeable: MERGEABLE, mergeStateStatus: BLOCKED (branch-protection/review-gate state, not a content conflict).
  • PR description still contains the "Design Rationale" section addressing issue Three test suites in scripts/ are never run by CI or pre-commit #2540's acceptance criterion, and reads cleanly with no leftover control characters.

No new issues found. Both previously-open non-blocking items (design-divergence rationale, garbled description) remain resolved as of the last round.

Verdict

Ready for merge — no PR content changed since the prior Ready for merge verdict; this round's diff is limited to an unrelated main-merge commit that does not touch any file this PR modifies.

Reviewed commit: 2c71f0b

Reviewed commit: 2c71f0b

@github-actions

Copy link
Copy Markdown
Contributor

💰 Cost: $0.5063 (review) — run

@dem-extra1
dem-extra1 merged commit 438bb92 into main Aug 31, 2026
15 of 16 checks passed
@dem-extra1
dem-extra1 deleted the fix/gate-test-suites-coverage branch August 31, 2026 00:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Three test suites in scripts/ are never run by CI or pre-commit

1 participant