feat: remix worker engine (t0.5) — RemixJob orchestration + claim/report/upload

[mtmtian]

What

Adds a second, worker-driven remix engine alongside the existing in-app single-shot text2video path (which is untouched). This phase ships tier t0.5 only: per-beat text-described regeneration — the storyboard from buildRemixBrief is generated beat-by-beat by an external worker (Cloud Run Job, separate repo), assembled with ffmpeg, brand-QC re-scanned (whisper + OCR), and lands as the same review-gated Creative(source:'remix', reviewStatus:'pending'). Competitor material stays analysis-only — no competitor pixels enter generation, consistent with the "borrow structure, not pixels" rules in remix-brief.ts / competitor-media.ts.

• RemixJob model + migration: brief snapshot, per-beat progress, qcReport, costTokens; tier / segmentPlan fields reserved for later tiers (API rejects anything but t0_5 for now).
• POST/GET /api/creatives/remix-jobs (session auth): reuses buildRemixBrief / competitorCreativeToAnalysis; shares the existing remix burst + daily-cap guardrails.
• Worker endpoints under /api/worker/remix-jobs/* (HMAC via ingest-auth.verifyHmac, new WORKER_WEBHOOK_SECRET):
  • claim — atomic pickup (updateMany count check) with REMIX_JOB_LEASE_MINUTES stale-lease recycling
  • report — atomic status state machine (pending→claimed→running→assembling→qc→succeeded/failed, terminal states locked, 409 on illegal transitions); outputUrl must match the canonical upload path; QC-fail is surfaced to reviewers via Creative.reviewNotes (promotion not blocked — reviewer decides)
  • upload — raw mp4 → GCS via server-side uploadToGCS (content-sha256-bound HMAC, pre-buffer header checks, 100MB cap)

Design notes

• Zero modifications to the existing remix/competitor files — only imports. storage.ts gains one additive export (gcsPublicPrefix).
• Both engines feed the same /creatives/review gate; remix history keeps working via Creative(source:'remix').
• Worker runtime lives in a separate repo (ffmpeg/whisper/tesseract image); dry-run mode reproduces the full flow at zero generation cost.

Test plan

• npm run lint / npx tsc --noEmit clean; migration verified against schema with prisma migrate diff (no drift).
• e2e/remix-jobs.spec.ts: 15 passed — happy path (ingest → job → claim → running/assembling/qc → succeeded promotes Creative to ready), claim atomicity, HMAC rejection, terminal-state 409, unclaimed→succeeded 409, non-canonical outputUrl 400, QC-fail reviewNotes.
• Live cross-repo integration against the real worker: full sequence verified, including the 409 terminal guard.

Context: this is the first slice of the tier-ladder proposal (t0 = your existing path / t0.5 = this PR / t1 & t2 gated pending the policy question in the alignment doc we shared). Happy to adjust RemixJob modeling or StoryboardBeat integration to your preference.

🤖 Generated with Claude Code

[oratis]

Review — remix worker engine (t0.5)

Thanks for this — the control plane is well-built. Claim atomicity via the single guarded updateMany genuinely holds up, the secret is fail-closed when unset, timingSafeEqual is used correctly, org scoping is derived from session/DB rather than request input everywhere it matters, and the migration is a clean net-new table with zero drift. The sign() extraction into e2e/helpers/hmac.ts is coverage-neutral, verified line by line.

The findings below are concentrated in one endpoint plus a transactionality gap. Severity labels are ours; happy to discuss any of them.

---

🔴 Blocking — /upload bypasses the state machine entirely

Three things combine in src/app/api/worker/remix-jobs/upload/route.ts:

1. jobId is outside the signature. It's read from the query string at :31, but verifyWorkerHmac(headers, actualSha256) at :58 signs only ${timestamp}:${contentSha256}. claim and report both sign the full JSON body including jobId, so upload is the one asymmetric case.
2. No status or ownership gate. findRemixJobById is a bare findUnique with no status filter; :62-66 goes straight from "job exists" to writing bytes.
3. Deterministic key, unconditional overwrite. uploadToGCS (src/lib/storage.ts:57) issues a plain uploadType=media insert with no ifGenerationMatch, to remix/{orgId}/{jobId}.mp4.

The benign path is reachable through this feature's own design. Worker A claims job J and is merely slow — not dead — past REMIX_JOB_LEASE_MINUTES. Worker B reclaims via the stale-lease branch (claim/route.ts:63-71), completes, QCs, reports succeeded; the Creative is promoted and a human approves it. Worker A then finishes its own un-QC'd render and calls /upload — which succeeds, overwriting the bytes the approved Creative points at. A's subsequent /report correctly 409s, so the DB row stays consistent, but the artifact actually being served no longer matches what was QC'd or reviewed. There's no fencing token from /claim, so nothing tells A it was preempted.

The adversarial variant: because the signature doesn't bind jobId, a captured upload request can be replayed within the 5-minute window (ingest-auth.ts:55) against ?jobId=<another org's job>.

This is what makes "terminal states locked" true of the DB row but not of the deliverable. It's also the one path with no test — e2e/remix-jobs.spec.ts:375-393 skips the upload happy path.

Suggested fix: bind jobId (and ideally the route) into the signed payload; require the job to be in an expected in-flight status before writing; and either use ifGenerationMatch or a claim-scoped/versioned object key so a preempted worker physically cannot clobber a finished one.

🟠 Non-transactional RemixJob → Creative writes leave unrecoverable state

report/route.ts:106-145: the status updateMany commits independently, then creative.update runs as a separate statement — no $transaction, no try/catch. If the second write throws (deleted Creative, transient DB error), the job is already committed as succeeded, but terminal states are excluded from both ALLOWED_PREV and the lease-recycling set, so every retry 409s forever and the Creative is stranded at generating. Job creation has the same shape (creatives/remix-jobs/route.ts:158-200) — a failed second create orphans a Creative(status:'generating') no job can ever promote.

🟠 The new e2e suite does not run in CI

e2e/remix-jobs.spec.ts:26 gates the whole file on INGEST_WEBHOOK_SECRET + WORKER_WEBHOOK_SECRET in the runner env. .github/workflows/e2e.yml:37-41 sets only AUTH_TOKEN_SECRET and PORT, and the workflow has no Postgres service container. So all 15 tests report as skipped, not passed — the "15 passed" figure can only come from a local run, and the green checkmark on this PR asserts nothing about the claim/report/tier logic.

Worth wiring properly rather than just amending the description: prompt-eval.yml:34 already sets a precedent for injecting a conditional secret via ${{ secrets.* }}. Note also that playwright.config.ts:30 hardcodes the server-side secret to 'e2e-worker-secret' while the spec reads process.env for the client side — enabling this in CI with any other value would fail every HMAC test for an unrelated reason.

🟡 outputUrl canonical-path check is a prefix + substring test

report/route.ts:90 is startsWith(canonicalPrefix) && includes(canonicalSubstring), not an exact match — and gcsPublicPrefix() (storage.ts:113) returns the bucket root without the uploads/ segment that uploadToGCS always prepends (storage.ts:54). So https://storage.googleapis.com/<bucket>/anything/remix/<orgId>/<jobId>/x.mp4 passes. Bounded to the configured bucket, so not independently critical, but the "must match the canonical upload path" guarantee isn't literally what's implemented. The one negative test varies org/job IDs rather than surrounding path structure.

🟡 Burst limit is not shared between the two remix paths

creatives/remix/route.ts:54 uses key:'remix', creatives/remix-jobs/route.ts:52 uses key:'remix-jobs'; the bucket key is ${key}:${identity} (rate-limit.ts:55), so these are independent buckets and an org gets 2× effective burst across the two endpoints. The daily cap is correctly shared — both count Creative(source:'remix') and both stamp that source, so that half of the claim holds.

🟡 No rate limiting or audit logging on the worker surface

None of the three worker routes import rate-limit.ts, and logAudit() is absent from all four new routes. The sibling session-authed route does rate-limit, so the pattern exists. On audit: AuditAction already carries competitor.video_store precisely because storing competitor video is IP-sensitive and wants provenance — using that material in a remix currently leaves no trail.

🟡 @@index([orgId, status]) doesn't serve the claim hot path

schema.prisma:1098 leads with orgId, but claim/route.ts:58-70 — the endpoint the whole worker fleet polls — filters on status alone with no orgId predicate, so that index can't be used. @@index([orgId, createdAt]) correctly matches the list GET.

🔵 Minor

• REMIX_JOB_LEASE_MINUTES is undocumented in .env.example (the other two new vars are there).
• No cleanup path for orphaned GCS blobs when /upload succeeds but /report never lands or is rejected — deleteFromGCS exists but is never called here.
• Stale-worker progress reports can non-monotonically overwrite costTokens / beats / qcReport while a job is still non-terminal, since those are plain overwrites rather than merges.

---

The claim/report state machine itself is sound and we'd be glad to see it land — the blocking items are the /upload gating and the report-route transactionality. Happy to look at a revision.

_{Review assisted by Claude Code; every finding above was manually verified against the diff.}

[mtmtian]

Fixes pushed — ea46840..f52482b (4 commits)

Point-by-point against the review:

🔴 /upload state-machine bypass → claim fencing (25160d6 + e6a406f)

• RemixJob gains claimToken (fresh randomUUID minted on every claim) and attempt (increments on every claim); both returned by /claim.
• Signature now binds job identity: upload signs ${ts}:${jobId}:${claimToken}:${contentSha256} (new required header x-adex-claim-token). A captured request can no longer be replayed against a different jobId — the signature won't verify.
• Status + ownership gate: after HMAC, upload requires the job to exist (404), claimToken to match the current holder's (409 stale claim), and status to be in-flight (409 otherwise).
• Versioned object key: remix/{orgId}/{jobId}/v{attempt}.mp4. Your benign scenario is now dead twice over: preempted worker A holds a stale token (409 before any write), and even in the check-to-write window its would-be object is v1 while B's is v2 — different keys, and /report's canonical check only ever accepts the current attempt's URL.
• /report is fenced too: claimToken required in the body and included in the atomic updateMany where-clause. This also closes the 🔵 "stale progress reports non-monotonically overwrite" item wholesale — a stale worker can no longer write anything.

🟠 Non-transactional writes (e6a406f)

/report's state transition + Creative promotion now run in one $transaction — if the Creative write fails, the status change rolls back and the job stays retryable instead of stranding a generating Creative. Job creation (Creative + RemixJob) is likewise transactional.

🟠 e2e suite not running in CI (792ae32)

e2e.yml now runs a postgres:16 service container, DATABASE_URL, prisma migrate deploy, both webhook secrets (testing-only literals), and INVITE_CODES_DISABLED=true so registration works on the runner. playwright.config.ts's webServer env now reads process.env first — the hardcoded-server-secret drift you flagged is gone. The remix suite runs for real in CI now (20 tests incl. the new negatives).

🟡 Items (e6a406f, f52482b, 25160d6)

• outputUrl: exact match against ${gcsPublicPrefix()}uploads/remix/{orgId}/{jobId}/v{attempt}.mp4 — prefix+substring test gone, and the missing uploads/ segment you spotted is in the canonical string.
• Burst limit: remix-jobs now uses the shared 'remix' rate-limit key — one bucket across both endpoints.
• Worker surface: all three routes rate-limited (generous abuse-guardrails, commented as such) and audited — new remix.job_create/claim/upload/report AuditActions.
• Index: added @@index([status, updatedAt]) for the claim hot path.

🔵 Minor

• REMIX_JOB_LEASE_MINUTES documented in .env.example.
• Failed reports best-effort delete the current attempt's blob; upload-then-vanish orphans remain uncovered (commented in-code as a known partial-coverage gap).

Verification

tsc clean · full Playwright suite 45/45 green against a live Postgres locally · the worker repo is updated to the new contract and re-verified against its mock, including the 409 stale-claim abandon path. The upload GCS happy path still isn't e2e-covered (local/CI creds are read-only) — all fencing rejections (401/409) fire pre-GCS and are covered.