Skip to content

[operations] Include runbooks/ in Validate Mission Schema --all sweep - #3291

Merged
hivecommons-hive[bot] merged 1 commit into
masterfrom
operations/validate-schema-runbooks-coverage
Sep 11, 2026
Merged

hivecommons-hive[bot] merged 1 commit into
masterfrom
operations/validate-schema-runbooks-coverage

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Summary

Closes the scheduled/push half of the false-green gap tracked in #3255: scripts/validate-schema.mjs's --all mode (used by validate-schema.yml's push/schedule/dispatch runs) only ever called discoverMissionFiles('fixes'), so the 10 runbooks/*.json mission files had zero automated schema-validation coverage on the weekly cadence sweep — despite using the same kc-mission-v1 schema (see runbooks/README.md).

Changes

  • scripts/validate-schema.mjs: added ALL_MODE_DIRS = ['fixes', 'runbooks']; --all mode now scans both, skipping any directory that doesn't exist so a checkout without a runbooks/ directory doesn't crash.
  • scripts/__tests__/validate-schema-cli-discover.test.mjs: added tests for runbooks/ discovery and the missing-directory case (31/31 tests pass).
  • docs/slo.md: updated the "third known exception" note to reflect that the --all-mode side of the gap is now fixed.

Verified against real repo data:

$ node scripts/validate-schema.mjs --all
...
✅ runbooks/certificate-rotation.json: Valid kc-mission-v1
✅ runbooks/cluster-upgrade.json: Valid kc-mission-v1
... (10/10 runbooks files now included)
{"event":"schema-validation-summary","level":"info","trigger":"all","total":1679,...}
✅ All files passed schema validation.

(Previously total only counted fixes/** files; the 10 runbooks/*.json files were never discovered.)

Not included — needs workflows permission

The other half of #3255 — validate-schema.yml's PR-mode git diff pathspec only matching fixes/**, so a runbooks/**-only PR resolves to an empty file list and the validation step is skipped (job still reports green) — is not in this PR. A branch containing only that one-line pathspec addition was rejected by GitHub:

! [remote rejected] ... (refusing to allow a GitHub App to create or update workflow
`.github/workflows/validate-schema.yml` without `workflows` permission)

The verified, ready-to-apply diff:

--- a/.github/workflows/validate-schema.yml
+++ b/.github/workflows/validate-schema.yml
@@ -34,7 +34,7 @@ jobs:
         id: changed
         if: github.event_name == 'pull_request'
         run: |
-          FILES=$(git diff --name-only --diff-filter=ACMR ${{ github.event.pull_request.base.sha }} ${{ github.sha }} -- 'fixes/**/*.json' 'fixes/**/*.yaml' 'fixes/**/*.yml' | tr '\n' ' ')
+          FILES=$(git diff --name-only --diff-filter=ACMR ${{ github.event.pull_request.base.sha }} ${{ github.sha }} -- 'fixes/**/*.json' 'fixes/**/*.yaml' 'fixes/**/*.yml' 'runbooks/**/*.json' 'runbooks/**/*.yaml' 'runbooks/**/*.yml' | tr '\n' ' ')
           echo "files=$FILES" >> $GITHUB_OUTPUT
       - name: Validate schema (PR — changed files only)

A maintainer with the workflows App permission can apply this directly. #3255 stays open until it lands.

Ref: #3255

— hive: agent=operations backend=copilot model=claude-sonnet-4-6

scripts/validate-schema.mjs's --all mode (used by push/schedule/dispatch
runs of validate-schema.yml) only ever called
discoverMissionFiles('fixes'), so the 10 runbooks/*.json mission files
had zero automated schema-validation coverage on the weekly cadence
sweep, despite using the same kc-mission-v1 schema (see
runbooks/README.md).

- Add ALL_MODE_DIRS = ['fixes', 'runbooks'] and scan both, skipping a
  directory if it doesn't exist so environments without a runbooks/
  directory don't crash.
- Add tests covering runbooks/ discovery and the missing-directory case.
- Update docs/slo.md to reflect the script-side fix.

The remaining half of this gap — validate-schema.yml's PR-mode git diff
pathspec only matching fixes/**, so a runbooks/**-only PR resolves to an
empty file list and the validation step is skipped (job still reports
green) — is not included here: a branch with that one-line pathspec
change to .github/workflows/validate-schema.yml was rejected by GitHub
("refusing to allow a GitHub App to create or update workflow ... without
`workflows` permission"). That diff is preserved in issue #3255 for a
maintainer with the workflows permission to apply.

Confirmed via `node scripts/validate-schema.mjs --all`: all 1679 mission
files across fixes/ and runbooks/ (10 runbooks files previously never
scanned) pass validation.

Ref: #3255
Signed-off-by: kubestellar-hive[bot] <280983584+kubestellar-hive[bot]@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Sep 9, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign clubanderson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Sep 9, 2026
@hivecommons-hive hivecommons-hive Bot added the hold label Sep 9, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human sign-off on the direction, not on the code.

This PR's only tracked rationale is #3255, which the hive filed itself — issue #3255 was filed by kubestellar-hive[bot] and no human has acknowledged it. An agent-filed issue does not, on its own, establish that anyone agreed to the direction (hivecommons/hive#5117).

The change may well be right; nothing here is a review of it. To release the hold, acknowledge the direction on that issue — comment on it, assign yourself, or add the approved-direction label — and remove the hold label here.

@clubanderson

Copy link
Copy Markdown
Member

Direction acknowledged by hub admin (approved-direction added on #3255); releasing the #5117 hold. This hive runs at ACMM L6 and may act on its own findings.

@hivecommons-hive
hivecommons-hive Bot merged commit 3653f67 into master Sep 11, 2026
10 of 11 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the operations/validate-schema-runbooks-coverage branch September 11, 2026 16:39
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

Check out what's new:

Stay connected: Slack #kubestellar-dev | Multi-Cluster Survey

hivecommons-hive Bot added a commit that referenced this pull request Sep 15, 2026
The 'Find changed files (PR only)' step's git diff pathspec only
matched fixes/**, so a PR touching only runbooks/*.json produced an
empty files output and the validation step was skipped (if condition
false), reporting green with zero files checked -- despite
on.pull_request.paths already watching runbooks/**.

Extend the pathspec to also include runbooks/**/*.json,
runbooks/**/*.yaml, and runbooks/**/*.yml so PR-mode validation
covers runbook mission files the same way fixes/** files are covered.

The --all (push/schedule/dispatch) mode already discovers both
fixes/ and runbooks/ via ALL_MODE_DIRS in scripts/validate-schema.mjs
(fixed in #3291); this closes the remaining PR-diff-mode gap tracked
in #3255.

Fixes #3255

Signed-off-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

copilot dco-signoff: yes Indicates the PR's author has signed the DCO. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant