diff --git a/README.md b/README.md index e8143ac..f242a56 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,46 @@ # infra There is dust in the clouds, infrastructure as code for perishdev. + +## What this repo manages + +- **Cloudflare** — the `perish.dev` zone, all DNS records (Email Routing + GitHub Pages), via the [`cloudflare/cloudflare`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest) v5 Terraform provider. +- **GitHub** — `perishdev/infra` and `perishdev/perishdev.github.io` repo settings, branch protection on `main`, the `safe-to-plan` label, via the [`integrations/github`](https://registry.terraform.io/providers/integrations/github/latest) v6 Terraform provider. +- **HCP Terraform** — remote state, runs, and the workspace variables that hold the API credentials. The repo's own `terraform/cloud {}` block lives in each leaf's `versions.tf`. + +No hosts of our own (no Salt, no Ansible). Everything in scope is SaaS-shaped. + +## Where things live + +``` +terraform/ + cloudflare/ one HCP workspace (cloudflare), zone + DNS + github/ one HCP workspace (github-org), repos + protection + labels +.github/ + workflows/ fork-safe terraform fmt + validate gates +docs/ + secrets.md secrets store, GitHub App, rotation + state.md HCP backend, workspace layout + ci.md workflow contract, fork-PR policy + setup.md out-of-band bootstrap runbook + import.md cf-terraforming runbook for adopting existing Cloudflare state +``` + +For the contracts the repo is built on — secrets, state, CI — see [`CLAUDE.md`](./CLAUDE.md). For the live design decisions table, look there first. + +## Contributing + +Branch protection requires four green checks before any merge to `main`: + +- `terraform fmt` +- `terraform validate (terraform/cloudflare)` +- `terraform validate (terraform/github)` +- `Terraform Cloud/perishdev/...` (the HCP aggregated commit status) + +Fork PRs only get GitHub Actions; HCP plans require a maintainer to apply the `safe-to-plan` label first. See [`docs/ci.md`](./docs/ci.md) for the full policy. + +Conventional Commits, Conventional Branches, Conventional PR titles. + +## License + +[MIT](./LICENSE) diff --git a/docs/ci.md b/docs/ci.md index 6a74a15..6fb6432 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -1,46 +1,55 @@ # CI -This repo is **public**. Anyone can fork it and open a PR. The CI workflow must assume a hostile PR body and protect every secret accordingly. +This repo is **public**. Anyone can fork it and open a PR. The CI workflow assumes a hostile PR body and protects every secret accordingly. ## Trust boundary | Surface | Visibility | Holds secrets? | |---|---|---| | Repo source, Issues, PRs, Actions logs | Public | no | -| HCP Terraform workspace | Private (HCP) | yes — all Terraform-time secrets | -| GitHub Actions encrypted secrets | Private (org/repo settings) | yes — only `TF_API_TOKEN` | -| `terraform plan` output | Public (posted to PR by HCP VCS integration) | redacted by HCP | +| HCP Terraform workspaces | Private (HCP) | yes — all Terraform-time secrets | +| GitHub Actions encrypted secrets | Private (repo settings) | none today (placeholder for `TF_API_TOKEN` if needed later) | +| Speculative `terraform plan` output | Linked from PR; lives in HCP UI | redacted by HCP | -## Workflows +## What runs on a PR -### `plan` — runs on PRs +| Job | Where it runs | Triggered for fork PRs? | Secrets in scope | +|---|---|---|---| +| `terraform fmt` | GitHub Actions | yes | none | +| `terraform validate (terraform/cloudflare)` | GitHub Actions | yes | none (`init -backend=false`) | +| `terraform validate (terraform/github)` | GitHub Actions | yes | none (`init -backend=false`) | +| HCP speculative plan per workspace | HCP Terraform via VCS integration | **only when maintainer labels `safe-to-plan`** | yes — full workspace variables | -- **Trigger**: `pull_request` against `main`. -- **PRs from branches in this repo** (collaborators): plan runs automatically. -- **PRs from forks**: plan runs only when a maintainer applies the `safe-to-plan` label. Until labeled, CI runs lint/validate only (no secrets, no HCP API token). -- **What it does**: calls the HCP Terraform API to start a speculative plan in the relevant workspace. HCP posts the plan summary back to the PR. -- **What it does NOT do**: never runs `terraform apply`, never echoes secret values, never reads `${{ secrets.* }}` into shell variables. +The first three are defined in [`.github/workflows/ci.yml`](../.github/workflows/ci.yml). The HCP plan is triggered by HCP's own VCS integration when it detects a push to a watched branch — not by a GitHub Actions job. There is no `TF_API_TOKEN` in use today. -### `apply` — runs on main +## What runs on merge to `main` -- **Trigger**: `push` to `main` (i.e. merged PR). -- HCP creates a run for each VCS-watched workspace. -- **Manual confirmation required** in HCP UI by a workspace admin. No auto-apply for the prod workspaces. -- Apply logs are visible in HCP, not in GitHub Actions. +- HCP creates a real run for each workspace whose path filter matches the merged commit (`terraform/cloudflare/**` for the `cloudflare` workspace, `terraform/github/**` for `github-org`). +- The run plans then **stops at "needs confirmation"** — applies require a human click in HCP UI (or an authenticated `POST /runs//actions/apply`). +- Apply logs live in HCP, not GitHub Actions. -### `lint` — runs on every PR including forks +A docs-only push to `main` triggers no workspace runs. HCP still posts an aggregated commit status (success), so branch protection treats it as a passing rollup. -- `terraform fmt -check`, `terraform validate`, `tflint` if adopted. -- No secrets, no network calls beyond provider schema downloads. -- Safe to run unconditionally on fork PRs. +## Branch protection on `main` + +Enforced via [`terraform/github/branch_protection.tf`](../terraform/github/branch_protection.tf). All four required status checks must be green before merge: + +- `terraform fmt` +- `terraform validate (terraform/cloudflare)` +- `terraform validate (terraform/github)` +- `Terraform Cloud/perishdev/repo-id-CffUfWW6H1x6Bauq` — HCP's aggregated commit status + +Plus: linear history, no force-push, no branch deletion, conversation resolution required. Admins can bypass for emergencies (`enforce_admins = false`). + +> ⚠️ The HCP check name embeds a per-installation VCS-repo ID (`repo-id-CffUfWW6H1x6Bauq`). If the GitHub–HCP OAuth/App connection is ever rebuilt, that string changes and branch protection silently blocks every PR until [`terraform/github/branch_protection.tf`](../terraform/github/branch_protection.tf) is updated to match. ## Rules 1. **Never use `pull_request_target`** unless the workflow is reviewed line-by-line for fork-PR safety. The default trigger is `pull_request`, which gives fork PRs no access to secrets. 2. **Never `echo` secrets**, never pass them as command-line args (visible in `ps`). Use env vars and let the tool read them. -3. **Sensitive Terraform outputs**: mark `sensitive = true` on any output that could leak a value. HCP redacts these from plan output. -4. **Fork-PR plans are opt-in.** A maintainer reviews the diff, decides whether it's safe to run against the real Cloudflare/GitHub account, then applies the label. -5. **Apply requires a human.** No automated apply to a production workspace, ever. +3. **Sensitive Terraform outputs**: mark `sensitive = true`. HCP redacts these from plan output. +4. **Fork-PR plans are opt-in.** A maintainer reviews the diff, decides whether it's safe to run against the real Cloudflare/GitHub account, then applies the `safe-to-plan` label. +5. **Apply requires a human.** No automated apply, ever. (The exception during this repo's bootstrap was deliberate API-driven applies after the speculative plan had been read; see commit history.) ## What to watch for in PR diffs from forks diff --git a/docs/import.md b/docs/import.md index a585ea5..c996fff 100644 --- a/docs/import.md +++ b/docs/import.md @@ -81,7 +81,7 @@ cf-terraforming import \ `--modern-import-block` emits Terraform 1.5+ `import { to = ... id = "..." }` blocks rather than the legacy CLI commands. Append to the same file so resources and their imports stay co-located. -## Verify and apply +## Verify ```sh terraform fmt generated.tf @@ -89,12 +89,17 @@ terraform validate terraform plan # expect: every existing resource shown as "will import", nothing as "will create" ``` -If `plan` shows any `create` for a resource that already exists, the resource name or import ID in `generated.tf` is wrong — fix before applying. +`terraform plan` from the CLI is allowed against a VCS-connected HCP workspace (it runs as a speculative plan in HCP); `terraform apply` from the CLI is intentionally blocked. The plan output streams back to your terminal with a link to the HCP run. -Once the plan is clean, commit `generated.tf` and let HCP Terraform run the apply (manual confirmation per `docs/ci.md`). +If `plan` shows any `create` for a resource that already exists, the resource name or import ID in `generated.tf` is wrong — fix before opening a PR. The real apply happens when the PR merges and a maintainer confirms in HCP (or scripts the confirm via `POST /api/v2/runs//actions/apply`). ## When to re-run Re-run `cf-terraforming generate` whenever new resources appear in Cloudflare that you want Terraform to manage. The cleanest workflow is to write new resources directly in Terraform from the start; cf-terraforming is for one-time onboarding of legacy state, not steady-state operations. > Note: cf-terraforming is **not** intended for use in CI. It runs locally during onboarding, output is reviewed by a human, then committed. + +## What we've actually used this for + +- **`cloudflare_dns_record`** — the six email-routing DNS records on `perish.dev` were imported via this flow. The four GitHub Pages apex records and the `www` CNAME were written directly into [`terraform/cloudflare/dns.tf`](../terraform/cloudflare/dns.tf) (no prior existence to import). +- **`cloudflare_pages_project`**, **`cloudflare_r2_bucket`** — discovered during the first run but explicitly **excluded** from import. The Pages projects were unrelated personal repos; the R2 bucket was empty. Scope decision: this repo manages `perish.dev` infra, not the whole account. diff --git a/docs/secrets.md b/docs/secrets.md index a5c7d06..c707eda 100644 --- a/docs/secrets.md +++ b/docs/secrets.md @@ -6,36 +6,47 @@ This repo is **public**. Plaintext secrets must never be committed — not in co | Secret | Store | Consumed by | |---|---|---| -| Cloudflare API token | HCP Terraform workspace variable (sensitive) | `terraform` runs | -| GitHub App ID, installation ID, private key | HCP Terraform workspace variable (sensitive) | `terraform` runs | -| HCP Terraform API token | GitHub Actions encrypted secret (`TF_API_TOKEN`) | CI workflow, to trigger HCP runs | +| Cloudflare API token | HCP Terraform workspace variable on `cloudflare`, marked sensitive | `terraform` runs in HCP | +| GitHub App ID, installation ID, private key (PEM) | HCP Terraform workspace variables on `github-org`, marked sensitive | `terraform` runs in HCP | +| HCP Terraform user API token | `~/.terraform.d/credentials.tfrc.json` (set by `terraform login`), per maintainer | local Terraform CLI; scripted HCP API calls | **Nothing encrypted is committed to the repo.** No SOPS, no `git-crypt`. If we ever run our own hosts and need runtime secrets, we pick an out-of-band store then; until that day, all secrets in scope live in HCP workspace variables. ## Why HCP Terraform as the vault - Workspace variables marked `sensitive` are encrypted at rest and redacted from run logs. -- A leaked CI token can trigger a plan but can't read the underlying secret values — they're injected into the run environment, not exposed to the workflow. +- Runs execute on HCP infrastructure; sensitive values are injected into the run environment, never echoed back to the laptop or to GitHub Actions logs. - One place to rotate Terraform-time credentials. +- HCP's own state encryption covers everything Terraform writes during a run. + +## Cloudflare API token scopes + +The token loaded into the `cloudflare` workspace needs **Edit** on every kind of resource we manage. For the current set: + +- Zone — DNS — Edit +- Zone — Zone Settings — Edit + +If/when we add more resource types, add scope before declaring the resource. For one-shot discovery via [`cf-terraforming`](./import.md), a separate token with **Read** scopes works fine — the user generates it locally, uses it to populate `generated.tf`, then deletes it. The persistent HCP token only needs the scopes that match the resources Terraform actively manages. ## GitHub App setup The GitHub provider authenticates as a GitHub App, not a PAT. Apps are not tied to a user, support fine-grained permissions, and rotate cleanly. 1. Create the app under the `perishdev` org (Settings → Developer settings → GitHub Apps → New). -2. Permissions: only what Terraform needs to manage (start narrow — repos, teams, secrets — expand on demand). -3. Install the app on the org, scoped to the repos Terraform will touch. -4. Generate a private key, store the App ID, installation ID, and PEM in the HCP Terraform workspace as three sensitive variables. -5. Configure the [`integrations/github`](https://registry.terraform.io/providers/integrations/github/latest/docs#github-app-installation) provider with those three values. +2. Permissions: Repository — Administration (read/write), Contents (read), Metadata (read), Pull requests (read/write). Organization — Members (read), Administration (read/write). +3. Install the app on the org, scoped to the repos Terraform manages (currently `perishdev/infra` and `perishdev/perishdev.github.io`). +4. Generate a private key, store the **App ID**, **installation ID**, and **PEM** in the `github-org` HCP workspace as three sensitive variables (`github_app_id`, `github_app_installation_id`, `github_app_pem`). +5. The PEM must include the full `-----BEGIN/END RSA PRIVATE KEY-----` lines. Pasting just the base64 body works for the first request, then breaks on key rotation. ## Rotation -- **Cloudflare token**: rotate yearly or on any suspected exposure. Generate the replacement before deleting the old one; update the HCP workspace variable; next run uses the new token. -- **GitHub App private key**: rotate yearly or on any suspected exposure. GitHub Apps support multiple active keys — generate a new one, swap the workspace variable, delete the old key. -- **HCP Terraform API token**: rotate when any maintainer with access leaves; update the GH Actions secret. +- **Cloudflare token**: rotate yearly or on suspected exposure. Generate the replacement before deleting the old one; update the HCP workspace variable; next run uses the new token. +- **GitHub App private key**: rotate yearly or on suspected exposure. GitHub Apps support multiple active keys — generate the new one, swap the HCP variable, delete the old key. +- **HCP user API token**: rotated when a maintainer leaves. Each maintainer regenerates their own via `terraform login` after revocation. ## Things that look like secrets but aren't -- Cloudflare account ID, zone IDs — not secret, but reconnaissance signal. Fine to commit. -- GitHub org name, repo names — public anyway. +- Cloudflare account ID, zone ID — not secret, but reconnaissance signal. Fine to commit. Both live as `local`s in `terraform/cloudflare/main.tf`. +- GitHub org / repo names, GitHub App ID, installation ID — public anyway. App ID and installation ID are *marked* sensitive in the HCP variables for defense-in-depth, but they're not credentials. - Resource IDs in Terraform state — state itself lives in HCP, never in the repo. +- The HCP organization name, workspace names — non-secret identifiers. diff --git a/docs/setup.md b/docs/setup.md index a909653..1a021a5 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -1,6 +1,6 @@ # Setup -One-time, out-of-band steps to wire up the systems this repo manages. The repo's Terraform code is speculative until these are done. +One-time, out-of-band steps to wire up the systems this repo manages. The Terraform code is speculative until these are done. Run in order. Each step explains what it produces and where the value goes. @@ -19,29 +19,30 @@ For each Terraform leaf directory, create a VCS-linked workspace under the `infr | `terraform/cloudflare/` | `cloudflare` | `terraform/cloudflare` | | `terraform/github/` | `github-org` | `terraform/github` | -For each workspace: +For each workspace, in Settings → Version Control: - VCS provider: GitHub (connect via OAuth, scope to this repo only). -- **Working directory**: set as above. Settings → Version Control → "Terraform Working Directory". +- **Terraform Working Directory**: set as above. - **Automatic Run Triggering**: set to **"Only trigger runs when files in specified paths change"**, path pattern `/**` (e.g. `terraform/cloudflare/**`). Without this, every push to `main` triggers every workspace — a docs-only commit will spuriously plan against Cloudflare and may fail on an unrelated change. -- **Automatic speculative plans**: **enabled**. This is the master toggle for plans on PRs; without it, PRs get no speculative plan and the GitHub status check never appears. Settings → Version Control → "Automatic speculative plans". +- **Automatic speculative plans**: **enabled**. This is the master toggle for plans on PRs; without it, PRs get no speculative plan and the GitHub status check never appears. - **Speculative plans on PRs from forks**: **disabled**. This is a separate, fork-specific toggle. Without disabling it, anyone opening a fork PR can read the workspace's sensitive variables via a malicious `.tf` file. The label-gated plan flow in [`ci.md`](./ci.md) replaces it for fork PRs. -- Execution mode: **Remote**. -- Auto-apply: **disabled** (manual confirmation required). + +In Settings → General: + +- **Execution mode**: Remote. +- **Auto-apply**: disabled (manual confirmation required). ## 3. Cloudflare API token 1. Cloudflare dashboard → My Profile → API Tokens → Create Token. -2. Use the **"Custom token"** template with these permissions for the `perish.dev` zone and the account it belongs to: +2. **Custom token** with these permissions for the `perish.dev` zone and the account it belongs to: - Zone — DNS — Edit - Zone — Zone Settings — Edit - - Zone — Workers Routes — Edit - - Account — Workers Scripts — Edit - - Account — Workers R2 Storage — Edit - - Account — Cloudflare Pages — Edit 3. Account Resources: **Include — your account only**. Zone Resources: **Include — Specific zone — `perish.dev`**. 4. Copy the token (shown once). -5. In HCP Terraform → workspace **`cloudflare`** → Variables → add `cloudflare_api_token` as a **Terraform variable**, mark **Sensitive**, paste the token. +5. In HCP → workspace **`cloudflare`** → Variables → add `cloudflare_api_token` as a **Terraform variable**, mark **Sensitive**, paste the token. + +If you want to onboard *more* Cloudflare resource types via [`cf-terraforming`](./import.md) later, the discovery token will need Read scopes matching those resource types (e.g. `Account.Rulesets:Read` for redirect rules). That's separate from the HCP token; generate a short-lived token, use it locally, delete it. ## 4. GitHub App @@ -49,35 +50,58 @@ The `github-org` workspace authenticates as a GitHub App, not a PAT. 1. Org settings → Developer settings → GitHub Apps → New GitHub App. 2. Permissions (start narrow, widen on demand): - - Repository permissions: Administration (read/write), Contents (read), Metadata (read), Pull requests (read/write). - - Organization permissions: Members (read), Administration (read/write). + - Repository: Administration (R/W), Contents (R), Metadata (R), Pull requests (R/W). + - Organization: Members (R), Administration (R/W). 3. Where can this app be installed: **Only on this account**. 4. Create. Note the **App ID**. 5. Generate a **private key** (downloads as a `.pem`). Treat it like a password. -6. Install the app on the `perishdev` org, scoped to "All repositories" (or to specific ones; can be widened later). Note the **Installation ID** from the install URL. -7. In HCP Terraform → workspace **`github-org`** → Variables → add three sensitive Terraform variables: +6. Install the app on the `perishdev` org, scoped to the repos Terraform manages (`perishdev/infra` and `perishdev/perishdev.github.io`). Note the **Installation ID** from the install URL (`.../installations/`). +7. In HCP → workspace **`github-org`** → Variables → add four Terraform variables: - `github_owner` = `perishdev` (not sensitive) - `github_app_id` = the App ID (sensitive) - `github_app_installation_id` = the Installation ID (sensitive) - - `github_app_pem` = paste the PEM contents (sensitive) + - `github_app_pem` = paste the **full PEM contents**, including the `-----BEGIN/END RSA PRIVATE KEY-----` lines (sensitive) ## 5. HCP Terraform API token for CI -Only needed if/when CI workflows trigger HCP runs (the current [`ci.yml`](../.github/workflows/ci.yml) doesn't — it only runs fmt/validate locally on the runner). Skip until needed. +Not currently used. The CI workflow does only fork-safe `fmt` and `validate`; HCP triggers plans and applies via its own VCS integration, not from GitHub Actions. -When needed: +Set this up only if a future CI workflow needs to call the HCP API directly: -1. HCP Terraform → User settings → Tokens → Create API token. Scope: minimal. +1. HCP → User settings → Tokens → Create API token, scope minimal. 2. GitHub → repo settings → Secrets → New repository secret → `TF_API_TOKEN`. ## 6. Local development -For running `make validate` / `make plan` from your laptop: - 1. Install Terraform ≥ 1.9. -2. `terraform login` — opens a browser, asks HCP for an API token tied to your user. Token lands in `~/.terraform.d/credentials.tfrc.json`. -3. From any leaf dir: `terraform init` (will authenticate to HCP automatically). +2. `terraform login` — opens a browser, asks HCP for a user API token. Token lands in `~/.terraform.d/credentials.tfrc.json`. The same token is what authenticates HCP API calls if you want to script anything (see [`state.md`](./state.md#api-access)). +3. From any leaf directory: `terraform init` (authenticates to HCP automatically), then `terraform plan`. A VCS-connected workspace allows `plan` from CLI but blocks `apply` — that gate is intentional. ## 7. Importing existing Cloudflare resources -See [`import.md`](./import.md) for the `cf-terraforming` runbook. It uses Cloudflare's own CLI to generate HCL + `import` blocks for the existing zone, R2 buckets, and Pages projects. +See [`import.md`](./import.md) for the `cf-terraforming` runbook. It uses Cloudflare's own CLI to generate HCL and Terraform 1.5+ `import` blocks for the existing zone, DNS records, R2 buckets, and Pages projects. + +## Troubleshooting + +### HCP doesn't post a status check on a PR + +Usually one of three things: + +1. The workspace's "Automatic speculative plans" master toggle is off (see step 2). +2. The path filter excludes the PR's diff. Check Settings → Version Control → Trigger Patterns. A docs-only PR correctly produces no per-workspace check; HCP rolls up to a single SUCCESS aggregated status. +3. The workspace's VCS webhook subscription drifted. Open Settings → Version Control and click **Update VCS settings** with no changes — that re-registers the webhook with GitHub. + +### GitHub Pages cert stuck at `null` + +If `gh api repos///pages` shows `https_certificate: null` and `protected_domain_state: null` for more than ~15 min after the DNS resolves, the cert provisioning flow has wedged. Fix: + +```sh +gh api -X PUT repos///pages -f 'cname=' # remove +gh api -X PUT repos///pages -f 'cname=' # re-add +``` + +Removing and re-adding the custom domain re-emits the event that kicks Let's Encrypt. Cert usually issues within a few minutes after that. + +### Branch protection blocks a PR with no HCP check + +If a PR's diff doesn't match any workspace's path filter, HCP posts a single SUCCESS for the aggregated commit status (we tested this — it works). If it doesn't, the workspace VCS webhook is probably the issue (see above). diff --git a/docs/state.md b/docs/state.md index 1c6622a..9cebdc9 100644 --- a/docs/state.md +++ b/docs/state.md @@ -5,43 +5,56 @@ State lives in **HCP Terraform** (formerly Terraform Cloud). It is never stored ## Why HCP Terraform - Managed remote backend with built-in locking — no S3 + DynamoDB to operate. -- Runs execute on HCP infrastructure, so the Cloudflare token / GitHub App key never leave HCP and never appear in GitHub Actions logs. -- Free tier covers small teams. Easy to migrate off later (state is just a JSON blob). +- Runs execute on HCP infrastructure, so the Cloudflare token and GitHub App key never leave HCP and never appear in GitHub Actions logs. +- Free tier covers small teams. Easy to migrate off later (state is a JSON blob). ## Organization -- **HCP org**: `perishdev` (one org per company). +- **HCP org**: `perishdev`. - **Project**: `infra` (one project per repo). ## Workspaces -One workspace per concern. Start with the minimum and split per environment only when a change to one environment must not affect another (different account, different apex domain, or a real isolation requirement). +One workspace per concern. We split per environment only when a change to one environment must not affect another (different account, different apex domain, or a real isolation requirement). Today neither concern needs that. -| Workspace | Purpose | Auto-apply? | -|---|---|---| -| `cloudflare` | Cloudflare zone, DNS, Workers, R2 for the one apex domain we manage | no — manual apply | -| `github-org` | GitHub org, repos, teams, branch protection | no — manual apply | +| Workspace | Manages | VCS path filter | Auto-apply? | +|---|---|---|---| +| `cloudflare` | the `perish.dev` zone + all DNS records (Email Routing + GitHub Pages) | `terraform/cloudflare/**` | no — manual apply | +| `github-org` | `perishdev/infra` and `perishdev/perishdev.github.io` repo settings, `main` branch protection on both, `safe-to-plan` label | `terraform/github/**` | no — manual apply | -Today there is no separate `cloudflare-staging` workspace because we manage a single apex domain in a single Cloudflare account. Staging surfaces (`staging.`, `app-staging` Worker, etc.) live as additional resources inside the `cloudflare` workspace. Revisit if we add a second apex domain dedicated to staging or move staging into a separate Cloudflare account. +Both are VCS-linked to this repo's `main` branch. Speculative plans run on PRs (see [`ci.md`](./ci.md)); applies stop at "needs confirmation" until a human (or authenticated API call) approves them. -Workspaces are wired to the `main` branch of this repo via VCS integration. Plans run on PRs (see [`ci.md`](./ci.md)); applies require manual confirmation in HCP. +There is no `cloudflare-staging` workspace because we manage a single apex domain in a single Cloudflare account. Staging surfaces (`staging.`, `app-staging` Worker, etc.) would live as additional resources inside the `cloudflare` workspace. Revisit if we ever add a second apex domain or move staging into a separate account. ## Variables Each workspace declares: -- **Terraform variables** (non-sensitive): account IDs, zone IDs, region names, repo names. -- **Sensitive variables**: Cloudflare API token, GitHub App credentials (App ID, installation ID, PEM). Marked `sensitive` so HCP redacts them from logs. -- **Environment variables**: rarely needed; only for provider-level config that must be in env (e.g. `TF_LOG`). +- **Terraform variables (non-sensitive)** — set in HCP UI: Cloudflare account ID is a *local* in `terraform/cloudflare/main.tf` rather than a variable, because it's not a credential; zone ID likewise. The GitHub workspace's `github_owner` is set as a non-sensitive var. +- **Sensitive variables** — Cloudflare API token, GitHub App credentials (App ID, installation ID, PEM). Marked `sensitive`; HCP redacts from logs and never echoes back. +- **Environment variables** — none today. Used only for provider-level config that has to live in env (`TF_LOG`, etc.). -Variable values live in HCP, not in the repo. The repo declares `variable "x" {}` blocks; HCP supplies the values. +Variable values live in HCP, not in the repo. The repo declares `variable "x" {}` blocks; HCP supplies the values at run time. ## Access -- **Read state**: workspace members in HCP (small, named list). -- **Trigger runs**: anyone who can open a PR triggers a plan; only workspace admins can confirm an apply. -- **Manage workspace settings**: org admins only. +- **Read state**: workspace members in HCP. +- **Trigger speculative plan**: anyone who can push a branch (own-repo) or open a PR with the `safe-to-plan` label (fork). +- **Confirm apply**: workspace admins, or an authenticated `POST /api/v2/runs//actions/apply` with a personal HCP API token. +- **Manage workspace settings**: org admins. + +## API access + +A `terraform login` writes an HCP user API token to `~/.terraform.d/credentials.tfrc.json`. The same token authenticates every HCP REST endpoint, so anything you can do in the UI you can script via: + +```sh +HCP_TOKEN=$(jq -r '.credentials["app.terraform.io"].token' ~/.terraform.d/credentials.tfrc.json) +curl -s "https://app.terraform.io/api/v2/organizations/perishdev/workspaces/cloudflare" \ + -H "Authorization: Bearer $HCP_TOKEN" | jq '.data.id' +``` + +Useful for reading plan summaries before merging a PR, confirming applies after a clean plan, and inspecting runs that didn't post status back to GitHub. ## Migrating away -If HCP ever stops being the right choice: `terraform state pull` from each workspace → `terraform state push` into the new backend. Code stays unchanged except for the `backend` block. This is the main reason for picking a managed backend now rather than self-hosting state on day one. +If HCP ever stops being the right choice: `terraform state pull` from each workspace, `terraform state push` to the new backend. Code stays unchanged except for the `cloud {}` block (replace with `backend "..." {}`). This is the main reason for picking a managed backend now rather than self-hosting state on day one. diff --git a/terraform/README.md b/terraform/README.md index 86620cf..d727afa 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -1,37 +1,50 @@ # terraform/ -Terraform code, organized as **one leaf directory per HCP Terraform workspace**. Each leaf is a self-contained root module with its own `versions.tf` (declaring the `cloud {}` block), `providers.tf`, `variables.tf`, and `main.tf`. +Terraform code, organized as **one leaf directory per HCP Terraform workspace**. Each leaf is a self-contained root module with its own `versions.tf` (declaring the `cloud {}` block), `providers.tf`, `variables.tf`, and the resource files. ## Layout ``` terraform/ - cloudflare/ -> HCP workspace: cloudflare - github/ -> HCP workspace: github-org + cloudflare/ -> HCP workspace: cloudflare + versions.tf terraform / provider / cloud block + providers.tf cloudflare provider config + variables.tf sensitive vars only (api token) + main.tf locals + zone import + dns.tf all DNS records (email + GH Pages) + github/ -> HCP workspace: github-org + versions.tf + providers.tf github app auth + variables.tf sensitive vars (app id, installation id, pem) + main.tf placeholder + repos.tf repository resources (infra, landing page) + branch_protection.tf main protection on both repos + labels.tf safe-to-plan label ``` There is no per-environment split today: -- **Cloudflare**: we manage a single apex domain in a single Cloudflare account. Staging surfaces (e.g. `staging.`, `app-staging` Worker) live as additional resources inside the same zone, in the same workspace. Split into `cloudflare/{production,staging}/` only when we add a second apex domain dedicated to staging or move staging into a separate Cloudflare account. +- **Cloudflare**: one apex domain, one Cloudflare account. Staging surfaces (e.g. `staging.`, `app-staging` Worker) would live as additional resources inside the same workspace. Split into `cloudflare/{production,staging}/` only when we add a second apex domain dedicated to staging or move staging into a separate Cloudflare account. - **GitHub**: the org is the org. No staging variant exists. ## Why one workspace per leaf -- A `terraform apply` for one concern can never touch another concern's state, even by accident — different workspaces, different state, different (sensitive) credentials. -- HCP's VCS integration triggers exactly one run per workspace per push, so blast radius is bounded. -- Mirrors the convention in [`pypi/infra`](https://github.com/pypi/infra) and aligns with the environment separation rule in the top-level `CLAUDE.md`. +- A `terraform apply` for one concern can never touch another's state, even by accident — different workspaces, different state, different (sensitive) credentials. +- HCP's VCS integration filters by path (`terraform//**`), so a PR that only touches one leaf only triggers that workspace. +- One file per resource category inside each leaf keeps diff review focused. ## Adding a new workspace -1. Create `terraform//` (or `terraform///` if a real isolation boundary justifies the split) with `versions.tf`, `providers.tf`, `variables.tf`, `main.tf`. +1. Create `terraform//` with `versions.tf`, `providers.tf`, `variables.tf`, and a resource file (`main.tf` or per-category `*.tf`). 2. Set the `cloud { workspaces { name = "" } }` block in `versions.tf`. -3. Create the matching workspace in HCP Terraform under the `perishdev` org, VCS-linked to this repo, working directory set to the leaf dir. -4. Populate sensitive variables (provider tokens, app credentials) as workspace variables in HCP, marked `sensitive`. -5. Open a PR; the `safe-to-plan` label gates fork-PR plans (see [`../docs/ci.md`](../docs/ci.md)). +3. Create the matching workspace in HCP under the `perishdev` org / `infra` project, VCS-linked to this repo, working directory set to the leaf dir, path filter `terraform//**` (see [`../docs/setup.md`](../docs/setup.md) step 2 for the full per-workspace settings). +4. Populate sensitive variables (provider tokens, credentials) in HCP. +5. If protected status checks live on `main`, add the new workspace's check name to [`github/branch_protection.tf`](./github/branch_protection.tf) in the same PR. +6. Open a PR; the `safe-to-plan` label gates fork-PR plans (see [`../docs/ci.md`](../docs/ci.md)). -## What's not here yet +## What's not here -- `terraform/modules/` — reusable building blocks. Add when the second resource needs the same shape in two places. Don't pre-build. -- Real resources. The current files are scaffolding; they declare providers and variables but create nothing. +- `terraform/modules/` — reusable building blocks. Add when the same shape is needed in two places. Don't pre-build. +- An HCP-managing workspace (i.e. Terraform managing the HCP workspaces themselves via the `tfe` provider). Considered, deferred — the manual workspace setup is documented in [`../docs/setup.md`](../docs/setup.md) and runs once. -See also: [`../docs/secrets.md`](../docs/secrets.md), [`../docs/state.md`](../docs/state.md), [`../docs/ci.md`](../docs/ci.md). +See also: [`../docs/secrets.md`](../docs/secrets.md), [`../docs/state.md`](../docs/state.md), [`../docs/ci.md`](../docs/ci.md), [`../docs/import.md`](../docs/import.md).