Port the factory loop from the factory-control-plane branch - #1028
Draft
pcapriolo-yc wants to merge 19 commits into
Draft
Port the factory loop from the factory-control-plane branch#1028pcapriolo-yc wants to merge 19 commits into
pcapriolo-yc wants to merge 19 commits into
Conversation
Source: yc-software/qm-yc branch factory-control-plane at ed98be4f8, fork base f0ba6556775a3189e473dd6cc12f88c8594ea4a2. Adds src/loops/factory/ (contract, credentials, effects, evaluate, forge-evaluate, linear-intake, preflight, process-work, ship), the factory-config admin resource and its FactoryConfig record, factory surface routing in loop-fire, the factory_configs durable map and factory deps in wiring, the Software factory admin card, and the ten loop-factory test suites. Drift adaptations against public main a5e43ef: - loop-fire.ts: kept public main's stageFailure -> { error, userMessage } contract; the three moved effect bodies throw failure.error. - config-store.ts: appended factoryConfig:${org} to flushScope's org drain-key list. - wiring.ts: reused the existing buildApp-scope orgScope local. - index.html: card class adapted to sv-customize for the subview filter; fork-base browse-model context dropped; card enrolled in prepareParityCards. - Three credential fixtures gained deployments: false, now required on DecryptedServiceCredential. Deviation: test/loop-factory-evaluate.test.ts drops the converge-vector.sh case (tools/factory/ is out of scope for this port), shipping 8 tests instead of 9; FACTORY_CHECKS is module-private since nothing else on the ported tree reads it. (cherry picked from commit ca09179)
…po (#1036) Closes QM-31. ### Changes **Why this matters:** The factory loop started its coding-agent wrapper from the subject repository it was about to work on, and pointed the wrapper's factory-source directory at that same place. The wrapper, its workflows and its factory tools live in a different repository, and nothing ever put them in the sandbox, so a work run launched a script that was not there. The failure was silent: the run degraded to a "no pull request" verdict, burned an attempt, and gave the operator no signal about the real cause. **What changes:** - Before a work run starts, the factory materialises its own control plane in the sandbox: a shallow clone of `yc-software/qm-yc` at a pinned branch into `/workspace/qm-yc`. A sandbox that already holds that checkout fetches and re-checks-out the pinned branch rather than re-cloning, so a warm sandbox converges instead of running a stale copy. - The wrapper is launched by absolute path out of that checkout while the working directory stays the subject repo, so the wrapper still clones `IO_REPO_CLONE_URL` and runs `IO_REPO_SETUP_CMD` exactly where it did before. - `IO_FACTORY_SOURCE_DIR` now names the control-plane checkout and `IO_REPO_DIR` the subject repo — two distinct directories, as the wrapper's contract requires. Every other variable in the wrapper's environment is unchanged. - A bootstrap that fails or times out now fails the run loudly with `factory_source_bootstrap_failed` and never starts the wrapper. The GitHub token authenticates the clone through the process environment only, so it appears in no command string and in no checked-out git remote. **Acceptance stories:** - The factory works a ticket on a cold sandbox: before, the wrapper was started from a path that did not exist; now the control plane is cloned into the sandbox first and the wrapper runs from it. - The factory works a ticket on a sandbox it has used before: the existing checkout is fetched and moved onto the pinned branch, so it never re-clones and never runs the control plane it first happened to fetch. - The wrapper gets the control-plane directory and the subject-repo directory as two different values, so its control-plane snapshot step finds the files it copies instead of aborting. - An operator looking at a run whose clone failed sees `factory_source_bootstrap_failed`, where before the item quietly burned an attempt and reported "no pull request". - The factory clones with its GitHub credential: the token travels only in the process environment, so no logged command string and no persisted git remote carries it. ### Test Plan - `npm run typecheck` - `npm run lint` - `node --test test/loop-factory-process-work.test.ts test/loop-factory-effects.test.ts test/loop-factory-wiring.test.ts` - `npm run test:all` — the three remaining failures reproduce identically on a clean `origin/main` worktree and are unrelated to this change. - Manual: drove the bootstrap and the wrapper launch against a real local git remote standing in for the control-plane repository — cold clone, warm fetch and checkout, a flipped branch pin, and a missing branch all behaved as intended. ## Proof it works Proven at runtime: the pre-fix code was first reproduced failing, then the fixed code was driven against a real git remote and a faithful wrapper stub, covering all five acceptance stories plus adversarial probes. Captured output is archived with the run, not in the diff: - story-all-before-prefix-baseline.txt — `.io-agent-qm-31/screenshots/` - story-all-after-runtime-proof.txt — `.io-agent-qm-31/screenshots/` - adversarial-probes.txt — `.io-agent-qm-31/screenshots/`
…plied (#1035) Closes QM-32. ### Changes **Why this matters:** The software factory could be configured but never actually existed. Nothing in the system could create a loop carrying the `factory` surface, and that surface is the only gate on every factory code path, so an operator could fill in the "Software factory" admin card, click Apply, and get nothing but stored settings. Turning the factory on now brings the factory into existence. **What changes:** - Applying `factory-config` on an org scope (the "Software factory" admin card) now also creates a "Software factory" loop owned by the applying admin at that org scope, which appears on the loops page and can be opened, fired, paused, and put on autopilot like any other loop. - Applying again is idempotent: the same config, a changed config, or a second admin applying all leave exactly one factory loop with its original id and owner. - Clear (`{ "reset": true }`) removes the configuration only and leaves the loop standing; a rejected apply (invalid body, non-org scope, unauthorized actor) behaves exactly as before and creates no loop. - The loop declares the ship actions the factory actually emits, `open_pr` and `close_already_fixed`, both gated on `auto`, so a converged run ships without a human click: the review happens on the pull request, not on the loop output. **Acceptance stories:** - As an org admin, I apply the factory configuration and get a real "Software factory" loop I can open and fire; before, Apply saved settings and created nothing. - As an org admin, I apply the configuration repeatedly, unchanged or changed, and still see exactly one factory loop with the same id and owner. A second org admin's apply also creates no second loop, but that admin cannot yet see or administer the loop over HTTP: `canAdministerLoop` has no org branch, so only the first applying admin passes. That widening is a separate change. - As an org admin, I click Clear and the configuration is gone while the loop stays, so re-applying restores a working factory without duplicating it. - As an admin submitting an invalid or non-org-scoped configuration, I get the same rejection as before and no factory loop is left behind. - As Quartermaster firing the factory loop, I now reach the factory effects — previously unreachable code — and fail on missing factory credentials rather than having no loop to fire at all. ### Test Plan - `NODE_ENV=test ALLOW_UNSIGNED_TEST_IDENTITY=1 node --experimental-test-module-mocks --test test/loop-factory-loop.test.ts` - `NODE_ENV=test ALLOW_UNSIGNED_TEST_IDENTITY=1 node --experimental-test-module-mocks --test test/admin-resources.test.ts` - `NODE_ENV=test ALLOW_UNSIGNED_TEST_IDENTITY=1 node --experimental-test-module-mocks --test test/loop-factory-wiring.test.ts` - `npm run lint` - Manual: in a browser against a live instance, filled in and applied the "Software factory" admin card at an org scope, confirmed the loop appears on the loops page, that a second apply adds none, that Clear leaves the loop, and that Fire now and Autopilot work on it. ## Proof it works All seven acceptance stories pass, verified in a browser against a live instance and backed by green targeted test runs.                 page@0c3cf56c3172d9b36dc6e06b01ea5b24.webm — /tmp/claude-recordings page@76ceaf47be3a6f35de0415b201a784d1.webm — /tmp/claude-recordings
…ds (#1086) The factory wrapper runs `claude -p` inside the sandbox, but `renderFactoryEnv` carried only the Linear key and the GitHub token, so the first model call had nothing to authenticate with. Normal QM sessions run `claude` on the server through the harness; the factory does not. - `credentials.ts`: a third org service credential, `factory-anthropic`, read alongside `factory-linear` and `factory-github`. A missing or disabled one is reported by slug and fails the fire before the sandbox, as the other two do. - `effects.ts`: `FactoryContext` and `loadFactoryContext` carry `anthropicApiKey`; `work` passes it to `renderFactoryEnv`. - `process-work.ts`: `ANTHROPIC_API_KEY` in the sandbox process env, next to the two `IO_*` secrets. - Tests: the credential suite covers three slugs in every unusable combination; the env suite pins the new key and includes it in the leak sentinels; the fixtures in the effects, loop-fire, wiring, and admin-resources suites carry the third credential. Verification: targeted suites 147 pass; `npm run test:all` 5896 tests, 0 failures; typecheck, eslint, oxlint, prettier clean. Removing the `ANTHROPIC_API_KEY` env line fails the process-work test. Targets the integration branch, not `main`. <!-- codesmith:footer --> --- <a href="https://app.blacksmith.sh/yc-software/codesmith/qm/pr/1086"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v2.svg"><img alt="View with [code]smith" src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"></picture></a> <a href="https://backend.blacksmith.sh/track/enable-autofix?expires=1791676280&installation_model_id=19911&pr_number=1086&repository=yc-software%2Fqm&return_to=https%3A%2F%2Fgithub.com%2Fyc-software%2Fqm%2Fpull%2F1086&signature=a34ff10d7d61340cd66406a88aceff505c1f8600f3a72299941105f6f17ce7e2"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img alt="Autofix with [code]smith" src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"></picture></a> <sup>Need help on this PR? Tag <code>@codesmith-bot</code> with what you need. Autofix is disabled.</sup> <!-- codesmith:autofix:disabled --> <!-- /codesmith:footer --> **Second gap found on the first local run:** the wrapper refused to start without a numeric ECS session id. `factorySessionIdFor(runId)` hashes the run id into a stable positive integer and `renderFactoryEnv` passes it as `IO_FACTORY_SESSION_ID`; the `layer/factory` wrapper on `qm-30-s18477` reads it when no session URL is present.
The sandbox runs the wrapper as root, and `claude` refuses `--dangerously-skip-permissions` under root unless `IS_SANDBOX=1` is set. The first local end-to-end run stopped at that check before its first model call. One env entry in `renderFactoryEnv`, pinned by the env test. Targets the integration branch, not `main`. <!-- codesmith:footer --> --- <a href="https://app.blacksmith.sh/yc-software/codesmith/qm/pr/1087"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v2.svg"><img alt="View with [code]smith" src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"></picture></a> <a href="https://backend.blacksmith.sh/track/enable-autofix?expires=1791677533&installation_model_id=19911&pr_number=1087&repository=yc-software%2Fqm&return_to=https%3A%2F%2Fgithub.com%2Fyc-software%2Fqm%2Fpull%2F1087&signature=322130f6c5a0d46e3fa737efb8c6ad8cb447992a43911529687c88bc65eed8b2"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img alt="Autofix with [code]smith" src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"></picture></a> <sup>Need help on this PR? Tag <code>@codesmith-bot</code> with what you need. Autofix is disabled.</sup> <!-- codesmith:autofix:disabled --> <!-- /codesmith:footer -->
… pull request (#1108) On the first local end-to-end runs the item read only `no pull request` while the wrapper's real reason (a missing session id, then a root check in `claude`) sat in the sandbox at `/root/.agent-proc/<id>/out`. `noPrVerdict` now appends the last three diagnostic lines from the wrapper's stdout (its own `[io-coding-agent*]`, `[claude-stderr]`, `[converge]`, `error=` and `FAIL` lines only), with the Linear, GitHub, and Anthropic secrets masked. A run with no such lines keeps the plain reason, so the existing assertions hold. Pinned by a new effects test. Targets the integration branch, not `main`. <!-- codesmith:footer --> --- <a href="https://app.blacksmith.sh/yc-software/codesmith/qm/pr/1108"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v2.svg"><img alt="View with [code]smith" src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"></picture></a> <a href="https://backend.blacksmith.sh/track/enable-autofix?expires=1791737507&installation_model_id=19911&pr_number=1108&repository=yc-software%2Fqm&return_to=https%3A%2F%2Fgithub.com%2Fyc-software%2Fqm%2Fpull%2F1108&signature=631685c94b9b77d4bed67682f29b6ee9f1a049c51afe80e3347905e23566a33a"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img alt="Autofix with [code]smith" src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"></picture></a> <sup>Need help on this PR? Tag <code>@codesmith-bot</code> with what you need. Autofix is disabled.</sup> <!-- codesmith:autofix:disabled --> <!-- /codesmith:footer -->
`src/loops/factory/evaluate.ts` shells out to `tools/factory/converge-vector.sh`, which the forge-read evaluator (`forge-evaluate.ts`) replaced. Its only importer was its own test, and its `FACTORY_CHECKS` list had already drifted from `FORGE_CHECKS`. Found by the whole-branch pre-merge review. Typecheck, knip, and the factory suites are clean without it. Targets the integration branch, not `main`. <!-- codesmith:footer --> --- <a href="https://app.blacksmith.sh/yc-software/codesmith/qm/pr/1109"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v2.svg"><img alt="View with [code]smith" src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"></picture></a> <a href="https://backend.blacksmith.sh/track/enable-autofix?expires=1791737709&installation_model_id=19911&pr_number=1109&repository=yc-software%2Fqm&return_to=https%3A%2F%2Fgithub.com%2Fyc-software%2Fqm%2Fpull%2F1109&signature=a65da1333b8539f9ef382140f7200525d2ac10a12826821fe9070f48e2824a1d"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img alt="Autofix with [code]smith" src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"></picture></a> <sup>Need help on this PR? Tag <code>@codesmith-bot</code> with what you need. Autofix is disabled.</sup> <!-- codesmith:autofix:disabled --> <!-- /codesmith:footer -->
Adds `docs/factory.md` and a README link. Covers the three credential slugs and the scopes they need, every config field, what a run does step by step, the plain statement that the wrapper runs as root with `IS_SANDBOX=1` and `claude --dangerously-skip-permissions` inside the sandbox with the three tokens in its environment, that the loop has no schedule, and the org-admin ownership gap. Item 5 of the whole-branch pre-merge review. Targets the integration branch, not `main`. <!-- codesmith:footer --> --- <a href="https://app.blacksmith.sh/yc-software/codesmith/qm/pr/1111"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v2.svg"><img alt="View with [code]smith" src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"></picture></a> <a href="https://backend.blacksmith.sh/track/enable-autofix?expires=1791737856&installation_model_id=19911&pr_number=1111&repository=yc-software%2Fqm&return_to=https%3A%2F%2Fgithub.com%2Fyc-software%2Fqm%2Fpull%2F1111&signature=aace8ec26999e1bb5c72b2359fc3f169f4f36a7443e40576238d419b3c3c4cd0"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img alt="Autofix with [code]smith" src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"></picture></a> <sup>Need help on this PR? Tag <code>@codesmith-bot</code> with what you need. Autofix is disabled.</sup> <!-- codesmith:autofix:disabled --> <!-- /codesmith:footer -->
The Software factory card said the credentials are `factory-linear` and `factory-github`. Since the sandbox model credential landed, the loop also requires `factory-anthropic`, and a first fire without it fails with a slug the card never mentioned. Copy only. Targets the integration branch, not `main`. <!-- codesmith:footer --> --- <a href="https://app.blacksmith.sh/yc-software/codesmith/qm/pr/1110"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v2.svg"><img alt="View with [code]smith" src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"></picture></a> <a href="https://backend.blacksmith.sh/track/enable-autofix?expires=1791737744&installation_model_id=19911&pr_number=1110&repository=yc-software%2Fqm&return_to=https%3A%2F%2Fgithub.com%2Fyc-software%2Fqm%2Fpull%2F1110&signature=443b10a8fe76a28c19f83b1eb169f6e4786003c43e2764410a731746747d5997"><picture><source media="(prefers-color-scheme: dark)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img alt="Autofix with [code]smith" src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"></picture></a> <sup>Need help on this PR? Tag <code>@codesmith-bot</code> with what you need. Autofix is disabled.</sup> <!-- codesmith:autofix:disabled --> <!-- /codesmith:footer -->
…tegration-sync-main
Merge of current main into qm-29-port-factory-loop. No conflicts; typecheck and CI green on the union. The Co-author trailers check flagged two commits that belong to main itself and ride along in a merge PR.
The build script pinned linux/amd64. On an arm64 host the image ran under emulation, where node runs without its JIT. The factory verify stage then took hours for the full suite and hung on the process tests. The Fly remote builder path stays amd64. Set LOCAL_SANDBOX_PLATFORM to override the default.
The dev supervisor finds a port's holder with lsof -ti tcp:<port>. The sandbox image had no lsof, so the port-ownership tests failed, the squatter child was never killed, and test/dev-supervisor-child.test.ts hung npm run test:all inside the factory sandbox. procps adds ps.
Read a fourth org credential, factory-slack, open the run's root Slack message from the loop, and pass SLACK_BOT_TOKEN, SLACK_CHANNEL_ID, and SLACK_THREAD_TS to the wrapper when the factory config names a channel. A configured channel with no credential fails closed like the other three credentials.
canAdministerLoop gated every per-loop route on the loop owner, so no admin could administer the org-owned factory loop. An org admin with an active grant on the loop's owner scope now passes, checked inline through isOrgAdmin in shared.ts. Personal loops are unchanged.
The loop reads the wrapper's output through one sandbox exec call with a bounded timeout. Under a saturated container one poll exceeded it, the error propagated, and the loop parked the item and killed a run that had passed Verify and Review. runFactoryProcess now retries up to six consecutive read failures with a two second pause. A vanished process session is still fatal at once.
Each attempt hashed its own run id into the wrapper's session id, so a feedback re-run that kept the previous attempt's worktree sat on the old session branch while Ship expected the new one and failed with session_mr_identity_missing. The id now hashes the loop and item key, so every attempt of one item shares one branch and one PR identity.
The loop evaluates a PR seconds after Ship, when the forge has registered the check runs but not finished them. ci_green_on_head treated any unfinished run as a failure and sent the item back to work with guidance naming a check that later passed. The check now polls until every run has completed, or the pipeline has left its pending states, within a 30 minute window, re-reading the pull request on each poll so mergeability is judged from the settled state.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-opens the content of #1008 (reverted in #1026) as a draft. This stays open until the QM-native factory path runs end to end on staging;
qm-ycpins staging to this PR's head withpin.sh --candidate-pr. Not for merge before then.Content is the squash commit
ca09179echerry-picked onto currentmain. See #1008 for the port notes, drift adaptations, and test counts.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.