The E2E budget guard counts workflow runs, not Supabase — it cannot see a backend swap

[TortoiseWolfe]

Findings 2026-08-19 — two proposals shipped, one is not buildable as written

Proposal 2, done literally, would have shipped a bug. E2E_BUDGET_BACKEND_EPOCH was
read but never passed, and its documented contract — "an ABSENT variable falls back to the
constant, an empty one opts out" — is unreachable from a workflow. GitHub evaluates
${{ vars.FOO }} to an empty string when the variable is unset, so env: cannot express
"absent". Wiring it up as asked would have flipped the default to opt-out:

	window start
with the epoch	2026-08-07T06:00:00Z
with an empty string	2026-08-02T00:00:00Z

5 days wider, counting runs billed to the deleted project. Empty now means "use the
default"; opting out is spelled none.

Proposal 3 cannot be built as described. The Supabase Management API exposes per-project
request counts (analytics/endpoints/usage.api-counts), not the three dimensions that
actually blew the quota — egress GB, MAU and realtime messages are org-level and
dashboard-only, with no documented public endpoint. So "read real usage from the API"
has no API to read. The project-ref cross-check now shipped is the one real backend signal
available without it.

Two sub-findings in the original body are already fixed and should not be re-investigated:
the three names existing as both variable and secret are gone (deleted 2026-08-18), and
SUPABASE_KEEPALIVE_REFS is now a single live ref (#815).

Proposal 4 is #575, an open epic.

---

The E2E circuit breaker does not measure Supabase. It counts GitHub workflow runs and treats
that as a proxy for quota. As a result it will block a brand-new backend with 100% of its free
tier unused, and it cannot notice that the backend changed at all.

Observed on PR #722, 2026-08-14:

NEXT_PUBLIC_SUPABASE_URL: https://ozbdyopxmeqmwnfsmglp.supabase.co
last 24h ....... 6 / 10
this cycle ..... 30 / 30   (since 2026-08-07)
verdict ........ MONTH_EXCEEDED

The run immediately before it read 29 / 30 ... OK and was green. Nothing changed but the
counter: the green run was the 30th and counted itself into the limit for the next one.

Why the proxy cannot be right

• scripts/ci/e2e-budget-guard.mjs calls api.github.com/.../workflows/e2e.yml/runs?created>=<since>.
  There is no Supabase API call anywhere in the file.
• The window starts at a hardcoded BACKEND_EPOCH = '2026-08-07T06:00:00Z' (:66).
  E2E_BUDGET_BACKEND_EPOCH exists as an override (:350-353) but is not plumbed through
  e2e.yml, so today only a code edit can move it.
• The project ref appears in the script exactly once — in a comment. SUPABASE_PROJECT_REF is
  never read, so the counting key is (workflow file, creation time, did-any-shard-run).
  Provisioning a new project cannot reset the counter, and the guard cannot tell that CI is
  now pointed somewhere else.
• DEFAULT_LIMITS = { day: 10, month: 30 } (:51) are policy numbers derived from "44 runs ≈
  100% of a month", not a measurement.

The #640/#648/#700 hardening is genuinely present and correct — blocked and cancelled runs are
not counted. The count of 30 is real. The question is what it is a count of.

The Aug-7 swap was done correctly — so this is latent, not broken today

Worth stating, because "the counter cannot see the ref" invites the guess that CI is pointed at
a stale project. It is not:

BACKEND_EPOCH                  2026-08-07T06:00:00Z   (e2e-budget-guard.mjs:66)
NEXT_PUBLIC_SUPABASE_URL  set  2026-08-07T06:27:23Z
SUPABASE_PROJECT_REF      set  2026-08-07T06:27:24Z
NEXT_PUBLIC_SUPABASE_ANON_KEY  2026-08-07T06:27:24Z

The epoch was moved to the provisioning time, 27 minutes before the variables were written, and
the local .env points at the same project. ozbdyopxmeqmwnfsmglp is the current backend
and the 30 counted runs are genuine runs against it. Someone did the right thing by hand.

That is exactly the risk: it worked because a human remembered. Nothing links the counter to the
ref, so the next swap silently keeps the old count if the epoch edit is forgotten.

Live trap: three names exist as BOTH a variable and a secret

Found while confirming the above. Same names, different values, different ages:

name	variable	secret	what workflows read
NEXT_PUBLIC_SUPABASE_URL	08-07T06:27	08-07T04:59	vars.* ×12
NEXT_PUBLIC_SUPABASE_ANON_KEY	08-07T06:27	08-07T06:22	vars.* ×8
SUPABASE_PROJECT_REF	08-07T06:27	08-07T04:59	vars.* ×2

Every reference in .github/workflows/ is vars.*. There are zero secrets.* references to
these three. The secrets are dead, they hold older values than the variables, and updating
one — the obvious move when pointing CI at a new backend — changes nothing and reports no error.
Delete them, or the next swap will look done and not be.

(SUPABASE_KEEPALIVE_REFS names three projects: ozbdyopxmeqmwnfsmglp, vswxgxbjodpgwfgsjrhq,
utxdunkaropkwnrqrsef. Only the first is referenced anywhere else — worth confirming the other
two are still wanted rather than being kept alive out of habit.)

Proposed

1. Make the project ref part of the counting key, so a backend swap resets the window and a
   forgotten epoch bump cannot silently block a healthy project.
2. Plumb E2E_BUDGET_BACKEND_EPOCH through e2e.yml so moving the window is config, not a code
   edit.
3. Read actual usage where possible, so the breaker protects the thing it is named after.
4. The durable answer is [EPIC] Point E2E at an ephemeral per-runner Supabase — retires the mutex, the 25% cancellation rate, and #567 #575 — local Supabase in CI. If the default PR path runs locally,
   cloud quota stops being the scarce resource and this guard shrinks to covering only the jobs
   that genuinely need the hosted backend.

Also note the gate is total: budget → build → smoke/rate-limiting/auth-setup → e2e.
A spec that touches no backend at all still cannot run — and chromium-gen carries a
storageState from auth.setup.ts, so it would hit Supabase anyway.

Currently blocking PR #722 and every other PR that needs E2E.

Supersedes the "BACKEND_EPOCH + limit both stale" note.