CoR: drive full deploy+verify chain and harden first-attempt deploys - #1658
CoR: drive full deploy+verify chain and harden first-attempt deploys#1658Nathan (nturinski) wants to merge 4 commits into
Conversation
Deploy flow: startDeploymentCommand and the deployment-plan approval hand-off now drive the azure-deploy agent through the full azure-prepare -> azure-validate -> azure-deploy skill chain and verify live endpoints, instead of only preparing a plan. Progress copy updated to 'Deploying to Azure...'. azure-deploy.agent.md: add skill-chain ownership (do not stop after preparation), Static Web Apps and Container Apps + ACR/managed-identity first-deploy invariants, and Step A-E ordering with endpoint verification. New Container Apps invariant: a file-backed/SQLite DB on the ephemeral filesystem is not durable -- do not relocate it to /tmp and report success; provision a managed database (PostgreSQL Flexible Server / Azure SQL with Entra auth) or surface the data-loss trade-off. Step E adds a write->restart->read persistence check. azure-project-scaffold/instructions.md: add a Step 4 datastore breadcrumb steering Container Apps targets away from file-based embedded databases so the persistence gap is avoided before deploy.
There was a problem hiding this comment.
🟡 Changes recommended
The updated deploy command prompt instructs the agent to “Generate and approve” the plan, which conflicts with the explicit user-approval gate and could cause the workflow to be bypassed.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Pull request overview
This PR strengthens the Copilot on Rails Deploy stage by driving an end-to-end deployment workflow (plan → validate → deploy → endpoint verification) rather than stopping after plan/artifact preparation, and adds agent/scaffold guidance to reduce first-attempt deployment failures (especially for Static Web Apps and Container Apps + embedded DB persistence pitfalls).
Changes:
- Update the deploy command and deployment-plan approval handoff prompts to explicitly require the
azure-prepare → azure-validate → azure-deploychain plus live endpoint verification. - Update deploy-stage progress UI copy to reflect full deployment execution (“Deploying to Azure…”).
- Expand agent + scaffold instructions with first-deploy invariants and persistence guardrails (Static Web Apps output assets, Container Apps + embedded DB durability).
File summaries
| File | Description |
|---|---|
| src/webviews/copilotOnRails/extension/controllers/DeploymentPlanViewController.ts | Strengthens the approval handoff prompt to drive the full deploy/verify chain. |
| src/commands/copilotOnRails/registerCopilotOnRailsCommands.ts | Updates deploy command prompt and progress UI copy to represent a full deployment workflow. |
| resources/agents/azure-project-scaffold/instructions.md | Adds scaffold-stage “deploy breadcrumbs” to catch Static Web Apps / Container Apps deployment pitfalls earlier. |
| resources/agents/azure-deploy.agent.md | Expands deploy agent responsibilities/invariants to ensure end-to-end deployment and verification (incl. durability checks). |
Review details
Suppressed comments (1)
resources/agents/azure-project-scaffold/instructions.md:233
- This text says it is a "one-line datastore-choice pointer" but is a long, detailed paragraph. Consider rewording to avoid promising a one-liner when the guidance is intentionally more extensive.
> 🧭 **Deploy breadcrumb (Container Apps / stateless-container target).** If the plan deploys the backend to Azure Container Apps (or any stateless container), do **not** make a file-based embedded database (SQLite / `node:sqlite`, on-disk LiteDB/H2, a `DB_PATH` file) the app's **durable** store. Container Apps run on an **ephemeral** filesystem, and the Azure Files volume mount that would persist it authenticates with a storage-account **key** — which locked-down subscriptions (`allowSharedKeyAccess: false` / no-local-auth policy) block, so at deploy time the only choices become a blocked mount or a `/tmp` file that silently loses all data on the next revision (see the `azure-deploy` agent's Container Apps invariants). Keep every DB call behind the `IDatabaseService` interface so the deployed app can point at the plan's **managed** database (PostgreSQL Flexible Server / Azure SQL with Entra auth) via env config; a file DB, if used at all, stays strictly for local dev behind that same seam. **If the plan pairs a file/embedded database with a Container Apps target, flag the mismatch now** — it is far cheaper to fix here than at deploy. Do not author infra or `azure.yaml` here — the `azure-deploy` agent owns those; this is a one-line datastore-choice pointer, not a task for this phase.
- Files reviewed: 4/4 changed files
- Comments generated: 3
- Review effort level: Lite
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
| export async function startDeploymentCommand(context: CopilotOnRailsContext, prompt?: string): Promise<void> { | ||
| await openChatWithAgent(context, copilotOnRailsCustomAgents.azureDeployCustomAgent, prompt ?? 'Prepare the project for deployment to Azure — generate `.azure/deployment-plan.md`, then the infrastructure (Bicep or Terraform), `azure.yaml`, and any Dockerfiles needed for `azd up`.', { | ||
| await openChatWithAgent(context, copilotOnRailsCustomAgents.azureDeployCustomAgent, prompt ?? 'Deploy the project to Azure. Generate and approve `.azure/deployment-plan.md`, prepare the required infrastructure and application artifacts, then complete the mandatory azure-prepare → azure-validate → azure-deploy skill chain and verify the live endpoints.', { | ||
| stage: 2, |
There was a problem hiding this comment.
Updated in d6e6faf: the deploy prompt now generates the plan, opens its preview, and waits for user approval before continuing.
There was a problem hiding this comment.
Have you tested this to make sure it does not take this as self approval to deploy? I had a lot of issues with this in the past where it took me saying "I want to eventually deploy this project to azure" in my intial prompt as a self approval when using the azure-prepare skill.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: nturinski <5290572+nturinski@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
The updated azure-deploy.agent.md contains conflicting Step C guidance around azd package gating (especially with remoteBuild: true) and a small frontmatter wording inconsistency that should be corrected to avoid ambiguous/blocked agent behavior.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Review details
Suppressed comments (2)
resources/agents/azure-deploy.agent.md:141
- Step C currently states not to proceed until
azd packagesucceeds, but immediately after introduces an exception fordocker.remoteBuild: truewhere localazd packageis not a meaningful gate. As written, the mandatory wording conflicts with the exception and may cause the agent to block unnecessarily on a local Docker dependency even when remote builds are intended.
After generating `azure.yaml` and the infra, **run `azd package`** (from the workspace root) to validate the manifest and confirm the app's build output is produced (for a Functions app, that the host actually discovers functions). Do **not** report the deployment as ready — and do **not** enter a retry-`azd deploy` loop — until `azd package` succeeds.
If `azd package` (or a later `azd` step) fails with a hook error such as `The '<x>' kind is not supported for hook` or `Inline scripts are only supported for shell hooks`, the fix is the `azure.yaml` hook itself (see the rules above), **not** re-running the same command. Correct the hook, then re-validate. Never retry the identical failing command more than once without changing the underlying artifact.
After `azd package` succeeds, continue immediately to Step D. Packaging success proves only that the service can be packaged; it does not validate Azure infrastructure, permissions, environment configuration, or endpoint health.
> **Containerized services with `remoteBuild: true`:** `azd package` builds images with the local Docker engine, so when the image is built remotely on ACR (or the local engine is unavailable/unhealthy) local packaging is not a meaningful gate — see Container Apps invariant #2 above. In that case validate the image through the remote build (`azd deploy` / `az acr build`) instead of blocking on local `azd package`.
resources/agents/azure-deploy.agent.md:3
- Frontmatter description says "verify the live endpoint" (singular) but the rest of the instructions require verifying multiple endpoints/URLs. Keeping this consistent avoids ambiguity about whether only one service URL needs checking.
This issue also appears on line 135 of the same file.
description: Plan, prepare, validate, and execute an Azure deployment. Generate Bicep/Terraform infrastructure, `azure.yaml`, Dockerfiles, and other required artifacts, then continue through the mandatory `azure-prepare` → `azure-validate` → `azure-deploy` skill chain and verify the live endpoint. WHEN: "deploy to Azure", "prepare for deployment", "generate infra", "generate Bicep", "generate Terraform", "create azure.yaml", "ship to Azure", "host on Azure", "create and deploy".
- Files reviewed: 4/4 changed files
- Comments generated: 0 new
- Review effort level: Lite
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
| export async function startDeploymentCommand(context: CopilotOnRailsContext, prompt?: string): Promise<void> { | ||
| await openChatWithAgent(context, copilotOnRailsCustomAgents.azureDeployCustomAgent, prompt ?? 'Prepare the project for deployment to Azure — generate `.azure/deployment-plan.md`, then the infrastructure (Bicep or Terraform), `azure.yaml`, and any Dockerfiles needed for `azd up`.', { | ||
| await openChatWithAgent(context, copilotOnRailsCustomAgents.azureDeployCustomAgent, prompt ?? 'Deploy the project to Azure. Generate and approve `.azure/deployment-plan.md`, prepare the required infrastructure and application artifacts, then complete the mandatory azure-prepare → azure-validate → azure-deploy skill chain and verify the live endpoints.', { | ||
| stage: 2, |
There was a problem hiding this comment.
Have you tested this to make sure it does not take this as self approval to deploy? I had a lot of issues with this in the past where it took me saying "I want to eventually deploy this project to azure" in my intial prompt as a self approval when using the azure-prepare skill.
Summary
Strengthens the Copilot on Rails deploy stage so a first
azddeployment is more likely to succeed and be verified end-to-end, rather than stopping at plan preparation.Changes
Deploy flow (extension)
startDeploymentCommandnow instructs theazure-deployagent to complete the mandatoryazure-prepare→azure-validate→azure-deployskill chain and verify live endpoints (not just prepare a plan). Progress copy updated to "Deploying to Azure…".DeploymentPlanViewControllerapproval hand-off drives the same full chain + endpoint verification.Agent guidance
azure-deploy.agent.md: skill-chain ownership (don't stop after preparation), Static Web Apps and Container Apps + ACR/managed-identity first-deploy invariants, and Step A–E ordering. New Container Apps invariant: a file-backed/SQLite DB on the ephemeral filesystem is not durable — don't relocate it to/tmpand report success; provision a managed database (PostgreSQL Flexible Server / Azure SQL with Entra auth) or surface the data-loss trade-off. Step E adds a write→restart→read persistence check.azure-project-scaffold/instructions.md: Step 4 datastore breadcrumb steering Container Apps targets away from file-based embedded DBs so the persistence gap is caught before deploy.Evidence
Grounded in a real scaffolded sample (Container Apps + SQLite): the no-local-auth policy blocked the Azure Files mount, so the DB was relocated to
/tmpwithmaxReplicas: 1— the deploy returned 200 while silently losing all relational data on every revision. These guardrails catch that at scaffold/deploy time.Notes
feat/CoR.docs/copilot-create-project.md(owned by docs: add Create New Project with Copilot guide and support runbook #1644).