Skip to content

test(drift): CI guard for source-of-truth drift (backend hosts + workload names)#218

Merged
saadqbal merged 1 commit into
developfrom
test/drift-check
Jun 8, 2026
Merged

test(drift): CI guard for source-of-truth drift (backend hosts + workload names)#218
saadqbal merged 1 commit into
developfrom
test/drift-check

Conversation

@LukasWodka

Copy link
Copy Markdown
Contributor

What

Net-new CI drift check — fails when two sources of truth that must agree have silently diverged. From the test-hardening analysis; the only roadmap item with no existing PR. Tracks backend#746.

Two checks (scripts/tests/check-drift.sh):

1. Backend API host parity — the dev/stg/prod hosts are hardcoded in three files (preflight.sh::_pf_backend_host, install-client-helm.sh::_backend_url, install-k8s.ps1::Get-BackendUrl). Asserts all three carry the identical *.tracebloc.io host set.

2. Workload-name contractsummary.sh (readiness wait, rollout status deployment/<name>) and diagnose.sh (--diagnose bundle, logs deploy/<name> + daemonset/<name>) reference mysql-client, <release>-jobs-manager, <release>-requests-proxy, tracebloc-resource-monitor by name. A chart rename breaks readiness + diagnostics silently — same class as the app=manager selector bug. Renders the chart (helm template) and asserts each name exists, and that the scripts still reference each (so the contract can't go stale on either side).

How

  • .github/workflows/drift-checks.yaml runs on scripts/** OR client/** (either side moving is the drift), sets up helm, runs the check.
  • scripts/tests/check-drift.bats — 8 cases (parity match / mismatch / removed; contract drop; render-missing; helm-absent skip).
  • check-drift.sh added to the static job's shellcheck list.

Testing

  • bash scripts/tests/check-drift.sh → green on develop (both checks pass; the chart render confirms the four workload names).
  • bats scripts/tests/check-drift.bats → 8/8.
  • Full suite: no new failures (same 4 pre-existing macOS-env failures).
  • shellcheck (error severity) clean on the new script.

Follow-ups (separate)

  • Cross-repo selector drift: Mintlify docs -l app=… selectors ↔ chart labels (the original app=manager surface) — needs a cross-repo guard.
  • Preflight thresholds ↔ docs sizing numbers (cross-repo).
  • Promote the Source-of-truth drift job to a required status check once proven stable.

Refs tracebloc/backend#746

🤖 Generated with Claude Code

…load names)

Mocked unit tests can't catch a chart rename that breaks the names summary.sh /
diagnose.sh grep for, or a backend host changed in one of the three files that
hardcode it — they ship green and break in the field. Add a drift checker:

- backend API host parity across preflight.sh / install-client-helm.sh /
  install-k8s.ps1 (dev/stg/prod hosts must match).
- workload-name contract: the Deployments/DaemonSet that summary.sh (readiness
  wait) and diagnose.sh (--diagnose bundle) reference by name must be rendered by
  the chart (helm template), and the scripts must still reference each.
- 8 bats cases for the checker; drift-checks.yaml runs on scripts/ or client/
  changes (helm set up in CI); check-drift.sh added to the shellcheck list.

Refs tracebloc/backend#746

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Contributor Author

👋 Heads-up — Code review queue is at 24 / 8

Above the WIP limit. The team convention is to review existing PRs before opening new work.

Open PRs currently in Code review (oldest first):

Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.)

@saadqbal saadqbal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid net-new drift guard — CI-only, can't affect runtime. Check 1 (backend host parity across preflight.sh / install-client-helm.sh / install-k8s.ps1) and Check 2 (workload-name contract: 2a scripts ↔ 2b helm template render) both look correct, and sourcing is side-effect-safe (set + main gated on direct execution) so the bats suite can exercise the helpers. 8/8 bats green. LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants