ci: run e2e suite against Vercel preview deployments - #641
Open
karrui wants to merge 2 commits into
Open
Conversation
Next.js rejects `dynamic(..., { ssr: false })` inside Server
Components, so the dev server failed to compile not-found.tsx and the
local e2e suite could not boot. Read window.history in an effect
instead and import the button statically.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds .github/workflows/e2e-preview.yml, triggered by the deployment_status event Vercel's GitHub integration fires automatically, filtered to successful Preview deployments, plus a manual workflow_dispatch with a preview-url input. The workflow needs no secrets to run: the Datadog Test Optimization step skips itself when DD_API_KEY / DD_SERVICE_NAME are absent (same gating as ci.yml), and VERCEL_AUTOMATION_BYPASS_SECRET is passed through so gated previews work when configured, while unprotected previews run with no secrets at all. Posts an explicit commit status (E2E against Vercel preview) that branch protection can require, and uploads the Playwright HTML report as an artifact. Playwright side: the config is now dual-mode. When PLAYWRIGHT_TEST_BASE_URL is set (preview mode) it skips the local webServer, attaches the x-vercel-protection-bypass header, and skips t3-env validation for the runner. The smoke tests drop the incidental container fixture import so they run against deployed previews; fixture-based tests (testcontainers Postgres/Redis) self-skip in preview mode since they need local stack control. Also ports a playwright-cache composite action that caches the Chromium binary keyed on the installed Playwright version. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds CI support for running the Playwright smoke E2E suite against Vercel preview deployments, while keeping local E2E behavior intact and fixing a Next.js Server Component compatibility issue in the error UI.
Changes:
- Add a new
deployment_status-driven GitHub Actions workflow to run E2E against successful Vercel preview deployments and publish an explicit commit status + artifacts. - Update Playwright config/tests to support “local mode” (spawn webServer) vs “preview mode” (use deployed base URL, bypass Vercel protection header, skip fixture-based tests).
- Fix local E2E breakage by making
GoBackButtonSSR-safe and statically importing it in the error card.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/e2e-preview.yml |
New workflow to run Playwright smoke tests on successful Vercel preview deployments (and via manual dispatch), upload reports, and post commit statuses. |
tooling/github/playwright-cache/action.yml |
New composite action to cache/install Playwright Chromium binaries keyed by Playwright version. |
apps/web/playwright.config.ts |
Add preview-vs-local mode behavior, optional Vercel protection bypass header, and CI reporters/artifacts. |
apps/web/tests/e2e/smoke.test.ts |
Remove dependency on ~/env so smoke tests can run against deployed previews. |
apps/web/tests/e2e/app-fixture.ts |
Skip fixture-based (testcontainers) tests in preview mode; make DB/Redis fixtures nullable. |
apps/web/src/app/_components/errors/go-back-button.tsx |
Make history access SSR-safe via useEffect to allow import from Server Components. |
apps/web/src/app/_components/errors/error-card.tsx |
Replace dynamic no-SSR import with static import of GoBackButton. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+9
to
+13
| // The Playwright runner may import test files that transitively pull in | ||
| // `~/env` (t3-env). The runner itself only needs a couple of vars - the | ||
| // *server* it talks to validates its own env - so skip the t3-env validation | ||
| // here, decoupling the suite from the full server env schema. | ||
| process.env.SKIP_ENV_VALIDATION = '1' |
Contributor
|
question: technically once apps get productionized and move to ECS, testing against vercel might make much sense since there might be AWS-specific issues? also, i was thinking about it and im not sure how much upside there is by testing against a built app v.s. testing against a vercel preview? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an e2e workflow that runs the Playwright smoke suite against every successful Vercel preview deployment, modeled on confetti's e2e-preview.yml but adapted for a generic starter kit.
Workflow (
.github/workflows/e2e-preview.yml)deployment_status- fired automatically by Vercel's GitHub integration, so it works out of the box for any repo connected to Vercel (no webhook bridge needed, unlike confetti'srepository_dispatch). Filtered to successfulPreviewdeployments. Also supports manualworkflow_dispatchwith a preview URL.DD_API_KEY/DD_SERVICE_NAMEare absent (same pattern asci.yml)VERCEL_AUTOMATION_BYPASS_SECRETis passed through; empty is fine for unprotected previews, and gated previews work once the secret is setE2E against Vercel previewcommit status (pending → success/failure) that branch protection can require, sincedeployment_statusruns don't surface reliably as PR checks.resolve-branchjob pinsDD_GIT_BRANCHfor Datadog PR attribution on detached-HEAD checkouts.Playwright dual-mode
playwright.config.ts: whenPLAYWRIGHT_TEST_BASE_URLis set (preview mode), skip the localwebServer, attach thex-vercel-protection-bypassheader, and skip t3-env validation for the runner. Local mode is unchanged.smoke.test.ts: drops the incidental container-fixture and~/envimports so it runs against deployed previews.app-fixture.ts: fixture-based tests (testcontainers Postgres/Redis) self-skip in preview mode - they need local stack control, so the preview run is a deployment-verification gate, not a full functional suite.tooling/github/playwright-cachecomposite action (Chromium cache keyed on Playwright version).Drive-by fix
Local e2e was broken on main:
error-card.tsxuseddynamic(..., { ssr: false })from a Server Component, which Next.js rejects, so the dev server never compiled.GoBackButtonnow readswindow.historyin an effect (SSR-safe) and is imported statically.Test plan
pnpm e2e: 2 passed, boots dev server + testcontainers)playwright test --listloads, and a fixture-based test correctly reportsskippedworkflow_dispatchagainst any preview URL)🤖 Generated with Claude Code