Complete and harden the main-based PR, dev deployment, and release lifecycle

[ebhills]

Context

PR #1115 consolidated the CI and deployment workflows and established a strong base for the new process. We have since decided to simplify the integration model:

• feature branches merge directly into main;
• main is the source of releasable code;
• development packages and deployments are created from a selected commit already on main;
• the legacy dev branch is not an integration destination and will be retired after its wanted changes are recovered.

This issue tracks the remaining documentation, security, and automation work. The checkboxes are intentionally separable so they can be completed over time.

Lifecycle and documentation

• Update workflow documentation and repository instructions to show feature branch → PR → main as the normal integration path.
• Remove instructions and workflow behavior that treat dev as a PR target or integration branch.
• Define the development-package lifecycle: select a commit on main, build a traceable development package, publish it to CodeArtifact, and deploy that exact version to the development environment.
• Add .github/RELEASE_RUNBOOK.md covering development-package cadence, production release steps, rollback, verification, and responsible roles.
• Document that legacy dev changes must be recovered on clean branches from current main; do not merge dev wholesale into main.
• Update workflow comments, names, PR summaries, and diagrams so they match the final main-based lifecycle.

Pull-request CI

• Make PR and feature-branch CI secretless: do not expose production-capable credentials or perform external publishing/deployment from untrusted PR code.
• Run the required test suite for every PR targeting main, and make those checks required before merge.
• Ensure PR container jobs build and test locally without pushing images.
• Keep permissions at the workflow/job minimum; grant id-token: write and package-write permissions only to jobs that require them.
• Pin third-party GitHub Actions to full commit SHAs and record the corresponding release versions in comments.

Development package and deployment

• Make deploy-dev.yml accept or resolve an explicit full commit SHA from main.
• Fail before building or publishing unless the selected SHA is an ancestor of current main.
• Build the development distribution once and deploy/publish that exact artifact rather than rebuilding independently.
• Give development artifacts an unambiguous version or label containing the source commit SHA; include the version, SHA, and artifact links in the workflow summary.
• Use OIDC rather than long-lived AWS credentials, with a role scoped only to the development repositories and deployment targets.
• Attach the deployment job to a protected GitHub development environment.
• Add concurrency protection so two development deployments cannot race or overwrite one another unexpectedly.
• Stop publishing Docker :latest merely because code reached main; reserve :latest for an approved production release and use traceable development/SHA tags beforehand.

Production release

• Require a production tag's commit to be an ancestor of main.
• Retain the existing validation that the tag version matches the package version.
• Complete all required tests, package builds, container tests, and release validation before the first irreversible publication.
• Build each distribution once and promote the same immutable artifacts to CodeArtifact and PyPI.
• Use a protected GitHub production environment with required approval and appropriate tag restrictions.
• Use least-privilege OIDC roles for production publication; keep PyPI trusted publishing or equivalent short-lived authentication.
• Add workflow concurrency and explicit failure summaries so partially completed releases are visible and cannot be silently retried into an ambiguous state.

Repository controls

• Configure a main ruleset or branch protection requiring the agreed CI checks, at least one approval, dismissal of stale approvals, and resolution of review conversations.
• Add a real .github/CODEOWNERS file covering workflows, packaging, and release documentation, or disable the code-owner requirement until ownership is defined.
• Define and restrict bypass rights, including whether administrators may bypass release and merge protections.
• Protect development and production environments so secrets and deployment permissions are released only to their intended workflows.
• Freeze new feature merges into dev; after wanted changes are recovered, archive or delete the branch and remove obsolete dev triggers.

Verification

• Run a development deployment from an explicit main SHA and attach the successful workflow run here.
• Confirm that an invalid/non-main SHA is rejected before any publication or deployment.
• Confirm that PR CI cannot access deployment secrets and cannot push packages or images.
• Use the next controlled production release to verify tag ancestry, approval gates, immutable artifact promotion, and post-publish availability checks.
• Mark the lifecycle documentation complete only after the implemented workflows and repository settings match it.

Related: #1115