workflow-engine.md

Workflow engine

Maintainer notes on src/workflow/. The workflow file format and the driver contract are in the Workflows and Advanced workflows guide pages.

Each rule below spans more than one file. The rule names the code that owns it, so you can read the details there.

Rules that span files

• A run ends only in settleRun. The shared finishRun path calls it when settling reaches the top workflow's result, before it hands any lease back or out, and on a call that takes no lease too. reportAttempt first reconciles earlier reports, so an end cancels a later report before it can advance the revision. It records an accepted outcome and never ends the run. After a failed report, the next next skips the dependents and ends the run. Respond, cancel and retry in run-controls.ts, and next in engine.ts, share that path. Whole-run cancel settles directly as cancelled, without reconciling another round. To run code when a run ends, hook into finishRun. Do network I/O only after withRun releases the run lock. That path queues the end record, a local write, under the lock, and saves whether it did in the run's endRecordQueued field. When updateRun then fails to save the run file, the queue is one save ahead of the run file, and the next next settles the run again and overwrites the queued <run>.end.json. A later next reports a record state only for a run whose field is true (runRecordState in records.ts).

• Reconcile the latest report before applying another or handing out a lease. reconcileRun, shared by leaseNext and reportAttempt, settles every driverless transition to a fixpoint at the run's current revision, through settleScope and settleItems, including workflow starts, fan-out expansion, render failures and bound exhaustion. reportAttempt advances the revision only after it, so only the latest revision can be unreconciled. A reached end applies before a loop decision. Only after reconciliation does leaseNext hand back an omitted open lease or issue the first ready leaf in declaration order that this driver can take. A running workflow occupies its parent's position in that order. Settling returns every ready leaf in that order (ready), and leaseNext walks it. A ready item does not count against max_parallel until it is leased, and one next leases at most one step, so a held-back item leaves its room to the next one without exceeding the cap. canTakeLease: false still reconciles driverless transitions. Every changed run is saved, including a wait.

• A report is checked against the revision of its own lease. reportAttempt and retryStep increment revision; skips, if, end, and the creation of child steps inside next never do. A lease keeps the run revision it was handed out at, and reportAttempt compares a report with that, so the report of a parallel lease does not make it stale. An attempt that is not an open lease keeps the older answers: stale_revision when its revision is not the run's, otherwise wrong_attempt.

• An end or break stops what is still open under its workflow. stopWhere cancels every leased step under the workflow that reached the end, its sub-workflows included, and keeps the attempt on the step and in cancelledAttempts. It cancels running workflow steps there and skips pending and waiting ones. The walk takes a path predicate and failure. End and break construct their own end_reached failure; manual cancel and retry supply theirs. Step descendants use the same isUnder predicate as archiveUnder in loop.ts. Cancel records the ids this walk changed before reconciliation can settle or skip other steps. reportAttempt answers a cancelled attempt lease_cancelled before it checks the driver; cancelledAttempts is never trimmed. Every next answer lists in cancelled the attempts the driver says it runs that are no longer its open leases, so the driver learns of a cancellation from any answer, done included.

• The local driver overlaps steps that do not share a checkout. runLocalDriver runs up to workflows.max_parallel steps (local config, default 1) and sends the ones it has started as running. A command with no workspace, and a prompt in the current checkout, share that checkout, so only one of them runs there at a time. A prompt earlier in the queue for that checkout blocks a later command there. The driver starts the prompt when the checkout is free and fewer than max_parallel steps are in flight; commands in other worktrees keep running. A prompt with its own worktree does not take the checkout. A headless prompt takes one of the slots, and the driver keeps leasing steps for the others while it runs. An interactive prompt uses the terminal, starts only when no command is running, and no step is leased while it runs. On cancelled, it drops a held lease, and it stops a running command with SIGTERM, then SIGKILL after 5 s while any captured descendant or process in a detached group remains, even after its command exited. Attached commands snapshot their descendant tree and start times before stopping, then probe and send SIGKILL only to descendants whose start times still match. If the identity snapshot fails, only the shell is stopped; detached commands use their process group. It does not send a stopping command as running again, so next does not cancel it twice. Dying commands still take a slot, and a new attempt of their step waits until they are gone. The signal policy follows max_parallel. At 1, a command runs in teamai's process group, as env exec runs one (src/env-exec.ts): the terminal's Ctrl-C reaches it, and it can read /dev/tty. A signal env exec would pass on (SIGTERM or SIGHUP, and SIGINT or SIGQUIT when the terminal does not reach the command) makes the driver stop the command, wait for it, and exit with that signal. Above 1, each command runs detached in a process group of its own, with no terminal, and SIGINT, SIGQUIT, SIGTERM or SIGHUP makes the driver stop every group, wait for them, and exit with that signal. Windows uses taskkill /T /F in place of the group kill. Staging runs inside the attempt at every max_parallel (stageThenStart), so the driver keeps polling while it stages, and a cancel aborts staging. A step whose attempt was cancelled, or whose lease expired, while it staged never starts. stageLease calls runSetup (setup.ts) last, only in a workspace it just created, with setup's own keys, and removes the worktree when setup fails.

• A workspace is a git contract on the lease, and the driver is the only code that touches git. planWorkspace in workspace.ts chooses the mode. An explicit workspace wins. A command that omits it is read-only, even inside a copy; when it exits 0 and changed the checkout, the driver fails it read_only_dirty. That includes a commit that leaves the worktree clean, a switch to another branch or a detach at the same commit, a change that only stages an existing edit, another staged hunk that leaves the status mixed, a permission change on a file that was already dirty, an edit to a quoted non-ASCII path, a symlink whose target changes while the file it names stays the same, and an edit to the destination of a staged rename, and a change inside a submodule that was already dirty. An interactive prompt that sets workspace: read-only is checked the same way. An interactive prompt that does not may change the checkout, and those edits stay. Conflict paths, including agent files, are read without Git's quoting, so a non-ASCII name is still checked for leftover markers. A prompt that omits it continues the nearest enclosing new or continue workflow, and a for_each ancestor makes that new, so items do not share a worktree. leaseNext puts the contract on the lease for new and continue, and for a read-only step inside a copy (mode: 'read-only', merge: []). That reader's branch is the one its declared needs wrote or read (changes or read, through items and children, but not after items or inherited needs), or the enclosing copy's when they touched none; needs on several branches fail workspace_base. It needs continue in can.workspace and the copy's owner, as continue does. A read-only step outside any copy gets no contract and runs in the active checkout. The local driver stages a reader without merges, takes its snapshot over the base overlay, and afterwards restores the overlay; when the step changed the copy or left its branch, it keeps the state on refs/teamai/kept/<attempt> (git add -A, then write-tree and commit-tree on HEAD; not the stash list, which every worktree shares), and pushes each submodule commit that tree records and the read commit does not into the checkout's git dir as refs/teamai/kept-modules/<attempt>/<commit> (at any depth, without the user's push hooks or push.recurseSubmodules), since a gitlink does not keep its commit reachable once the copy is removed, then runs git checkout -f <branch>, git reset --hard to the commit it read, git clean -fd, and git submodule update --checkout --recursive on the submodules it changed, so a later writer never commits a reader's edits. It resets nothing when a repository among the changed paths has uncommitted or untracked files, or a nested submodule commit its parent does not record, which the keep cannot hold. The copy counts as put back only when HEAD is refs/heads/<branch> at that commit and git status --porcelain --untracked-files=all --ignore-submodules=none prints the same lines it printed when the reader was staged; ignored files are left in place. Otherwise the reader fails workspace_base. stageLease writes a teamai-unrestored marker, with that status, in the worktree's git dir before the reader's overlay, and the driver removes it once the copy is put back or left untouched, so a driver killed mid-reader still leaves it. stageLease refuses later leases in a marked copy until the copy is back, when it removes the marker. A status line does not show content, so a marker whose status was not empty stays until the person who puts the copy back deletes it. ensureWorktree also refuses an existing worktree whose HEAD is not on its branch or a directory that is not a worktree, and prunes and recreates one whose directory was deleted. The reader reports read (branch, base, commit) and never changes. report returns invalid_report for read on any other lease, and for changes or a success without read on a reader's lease. run.copies[].commit and fan-out branches follow changes only. cleanup comes from the lease's contract, so a reader that was the copy's last user removes it too. leaseNext holds a step back for this driver, and tries the next ready leaf, when the driver's can.workspace lacks that mode, or when a continue copy belongs to another driver (run.copies). It answers wait, with every held step's reason joined by ; , only when every ready leaf is held back. Driver capabilities stay out of reconcileRun, which reportAttempt and the run controls also call. It fails workspace_base when continue has no enclosing new, or when the run has no base. workflow start in a git checkout records the base (commit, and the branch unless HEAD is detached); otherwise the first next that sends base sets it, and a later base never changes it. The local driver fails a lease whose workspace it cannot read with workspace_base instead of running it in the active checkout. The branch is teamai/<runId>/<path-safe step path>/p<pass> (branchName); a [n] in the path becomes -n. The first branch in needs order is where the step starts, and the rest are merged in that order. An item's after branches come first. The local driver (local-driver.ts, worktree.ts) adds the worktree under <checkout>/.teamai/workflow-worktrees/, points core.hooksPath at an empty directory, and replaces AGENT_PATHS (AGENTS.md, CLAUDE.md, .agents, .githooks, and each default toolPaths entry's project directory, project MCP file and Copilot .github paths) with the run's base commit. An agent path under a symlinked parent fails base_config before anything is copied or removed, since both would follow the link. It then gives the worktree the inline pull a new worktree gets (prepareCheckout), which delivers the team's skills, rules, agents and MCP and records that checkout's team revision. The overlay is taken after that delivery, so those files are not the step's commit. A path the step edits is committed. A step that left HEAD off the leased branch fails workspace_base before anything is committed, reported as changes, or pushed, and the worktree stays as it left it. A path it leaves untouched is put back from the branch. A merge conflict stops the merge: a command fails merge_conflict, and a prompt is told the branches that remain. Snapshots and commits read submodules with --ignore-submodules=none, and after git add -A, which skips a submodule whose ignore is all, each submodule is staged by name, so submodule.<name>.ignore and diff.ignoreSubmodules hide no change from a reader's check, its keep, or a writer's commit. The driver commits as TeamAI <teamai@localhost> and reports changes (branch, base, from, to, commits). It pushes when origin exists and this step made commits, whether the step succeeded or failed, and also when a later attempt adds no commit but this branch has commits origin does not have, including a first push that never landed and an origin with no tracking refs yet. A failed push fails a step that succeeded with push_failed, and the worktree stays. report sets cleanup only when the step succeeded and no step that can still run uses that branch, judged on the run as the next next settles it: steps a loop runs again count, and a workflow or for_each step that has not started counts by the branch plannedBranch gives, the one around it unless it opens its own; only then does the driver run git worktree remove --force. It does not delete the branch, and it never resets, cleans, or switches the active checkout. A command that starts its own sandbox, such as docker, stays read-only. What it writes does not feed new, branches, or a later step. session: continue and session.from are step reads. The engine puts { tool, id } on the lease, with earlier, the sessions that one continued, newest first, and never copies a session. The driver resumes Claude and Codex by that id, copies another tool through AgentSession.copy, restores a copy from the team session archive when neither works, and tries each earlier session the same way before it starts fresh. OpenCode has no resume flag, so it starts fresh. A driver that continued the lease's session or an earlier one, by that id or in a copy, echoes the one it continued as resumed in its report. One that started a new session leaves resumed out. reportAttempt answers invalid_report for a resumed on a step that resumes no session, and for one that no pass of the run reported and no link continued (knowsSession). When the reported tool and session differ from resumed, reportAttempt appends { from: resumed, to } to run.sessionContinuations. sessionOf follows those links from the session a step would resume, taking the latest link from each session, so a later step resumes the copy that holds the last turns, whichever machine or tool made it. The snapshot publishes the links, so resumableSessions on this machine and the claim on the next one both see them. A step that resumed by the same id adds no link, so a run on one machine with one tool leases the same sessions as without links.

• A driver pushes the commits it reports, and fetches the commits it is leased. Every driver keeps these two git rules, so a run another driver or another machine continues finds each commit on origin. Whenever origin exists, a driver pushes the commits it reports in changes before it reports them, failed steps included; the local driver's finishWorkspace pushes for every outcome. A push that fails still reports the commits. A step that succeeded fails push_failed. A step that failed keeps its own failure, and the push sentence is added to its message: a retry must fix the step first, and a push_failed in its place would hide that cause from the person and from the run record. Before it starts from, merges, or applies a lease commit (start, each merge[] entry, configCommit), a driver checks it with git cat-file -e <c>^{commit}. For a missing one it fetches +refs/heads/teamai/<run>/*:refs/remotes/origin/teamai/<run>/*, and checks again, since a refspec fetch that matches no branch exits 0. A fetch by commit id is not used: a server can refuse it, and a fetched commit no ref holds can be collected later. A commit still missing fails workspace_base naming the commit and its branch, before anything merges or runs, so it is never a file-less merge_conflict or a prompt told to resolve one. The local driver does this in stageLease through fetchMissingCommits. workflowGitConformanceScenarios in conformance.ts checks both rules against a bare origin, and workflow-git-conformance.test.ts runs them against the local driver.

• A run changes machines only through handoff.ts. publishRun and the claim behind claimRun and resumeRun are the only writers of workflows/runs/<run>/state.json on teamai-reports, and they write it inside updateReports. A publish writes only while origin shows this install (installOwner, from <dataHome>/workflows/owner.json) and the state revision this machine last saw (acknowledged). An unchanged body writes nothing. A publish checks the business origin without a lock, then answers state_moved when, under the locks, the run's base, a copy's commit, or the sessions it archived differ from what it checked (checkedOutsideLocks). Anything else that moved meanwhile is published as the locks find it, so a run a driver is still advancing can be published. resumeRun continues a run locally when origin has no snapshot and this machine never saw one acknowledged (stateRevision 0), since no other install can have claimed it. A claim reads origin under the reports lock and releases it, decides outside the lock (definitionsOf can fetch and sync a shared clone, and checkOrigin runs ls-remote, only for a running run: a settled one leases nothing until a retry reopens it, and the local driver's reconcileTakenCopies waits for that too), then writes only while origin still holds the write id it read (replaceSnapshot). The compare key is the write id, not the revision, because two claims of one revision both write stateRevision + 1. A moved or missing snapshot answers state_moved. Every next that answers done, and every run control, calls publishRunEnd, which runs publishRun when the run was published and has ended, so origin never hands out a run its owner finished. A publish drains the run's records first, so none lands after its end. The drain (publishPendingRecords) drops the records of a run whose snapshot on origin names another install, so an install that ends its copy of a run it lost never replaces the owner's record. Once the end is on origin, a repeat writes nothing, so an end that a process saved but stopped before publishing reaches origin on the next request. A claim changes steps only through takeOver in step-state.ts: it releases a lease, and repends a lease in a new step so its next pass gets a new branch. handoff.ts never imports the engine. The engine calls claimRun for the claim field of next, and workflow publish --sessions passes the engine's resumableSessions in. What the snapshot carries is a rule table per type in snapshot.ts (publish, drop, or rewrite), so a new run or step field does not compile until it has a rule, and parseSnapshot refuses a field it would drop.

• The end reached first wins. A reported step's settledAt is the revision its report made. Every terminal transition, including a skip or cancellation, carries settledAt. A step the engine settles takes the run's current revision, which is that of the report that caused it, because every earlier revision was reconciled before that report was accepted. An open lease it fails takes the revision it was leased at, and a cancellation the winning end's revision. reportAttempt accepts a report only after reconcileRun has applied every end reached before it, so an end a later report reaches never competes with an earlier one. Every end settleScope has yet to apply was reached at the run's current revision, and it applies the one declared first. The winner therefore does not depend on when next runs between reports.

• A break is a nameless succeeded end. A succeeded step whose break holds competes with the steps that reached an end by the same rule. It ends its workflow, or that workflow's loop, as success. Its value cannot change after its step succeeds, because pull rejects break inside a step loop's range, where a previous-pass read could change.

• A workflow loop is decided when a round has nothing left. Once every step of a looping workflow has settled and no end or break was reached, decideRound in loop.ts evaluates the workflow's loop.while. When it is false, the workflow ends with the round's outcome and outputs. Otherwise every step of the workflow repends for the next round, unless the workflow has run max_iterations rounds: then it fails loop_exhausted (WorkflowResult.failure, and run.failure, shown in done, at the top). A failed step alone does not stop the loop. The round is kept on the step that runs the workflow (Scope.runner), so repend resets it with that step, or on the run at the top. Pull rejects a step loop in a looping workflow.

• A step waits for every step it reads. readsOf collects the steps that the step's expressions (stepExpressions), templates, and with read. For a step that runs an inline workflow, it also collects the outer steps that the inline steps read. skipUnreachable uses that set to decide readiness. If a new field reads steps, add it to stepExpressions or templatedTexts. Otherwise the step can read a value before it exists, or an inline workflow can wait forever. A step outside a loop that needs or reads a step inside it also waits until the loop owner is settled (waitsFor in loop.ts), and is neither skipped nor ready before.

• A read of a later step in the same loop is a previous-pass read, and never waits. previous in loop.ts classifies it: the target is in the reader's loop range and needs the reader, directly or not. For an inline step, the reader is the step that runs its workflow. Pull allows the read, the engine leaves it out of the reads a step waits for, and expressionSteps gives the reader the target's latest archived pass (previousPass), which holds no pass fields on pass 1. Classify a new kind of read through previous too, or pull and the engine disagree on it.

• A step loop is decided once per run of its owner. After checking for an end, settleScope calls decideStepLoop for each succeeded owner without loopFinal. A false while sets loopFinal. Otherwise the range runs again through repend, unless a while owner has run max_iterations times, which fails it loop_exhausted. A loop without while is bounded where the owner's action would start, after the skip rule and if, for a leaf and a workflow step alike (exhaustedAtStart). The owner counts its runs in iteration. Pull rejects ranges that share a step and loops that wait for each other (loopProblems), so the engine does not check them again.

• A pass id is never reused. repend moves a step's current pass into its passes and bumps pass. archiveUnder moves what a repended workflow or for_each step ran into run.archive, each pass stamped within with the step path and pass that archived it, and firstPass starts a new instance after every archived pass of its path. The full record of a run is run.steps, with each step's passes, plus run.archive; a run record, scorer, or budget reads both. steps.<id>.passes is a step's passes plus its current pass once it settled (passesOf). The expression projection keeps results for a for_each pass, including in previousPass. Pull checks the same outputs-or-results shape in step reads and workflow output aliases.

• An expression reads its absolute paths, wherever they are. expressionPaths returns every inputs and steps path, including those inside function arguments and predicates, and never a relative path such as .size, which reads the current element of a predicate. readsOf, skipUnreachable and the pull read checks use it, so a new kind of expression must return its paths there.

• Expression types are checked in two places, through typeProblems. parseExpression rejects what an expression shows by itself, such as x < 'a' or any(...) < 1. readDiagnostics checks paths against the types the workflow declares. It checks each path only in the workflow that owns it, so a predicate an inline step runs over an enclosing step is reported once.

• Pull and the engine share one list of templated fields, one list of step expressions, and one list of actions. The templated fields are in templatedTexts and the expressions, such as if and break, in stepExpressions, both in template.ts. HITL declarations normalize in parse-hitl.ts. The actions are in STEP_ACTIONS in types.ts. Add a new field or action there, not to a copy in parse.ts or engine.ts. Pull, the cache's prompt_file check, the lease render and readsOf all split a template with parseTemplate, which decides which {{ … }} is a template, and the render evaluates it with evaluateExpression, as for an if. parseReads in engine.ts and the step parse in parse.ts key a table by each expression, so a new one does not compile until both say how to read it.

• A new step status must fit stopped(), settled() and stopWhere(). All three are in step-state.ts. stopped() decides which dependents are skipped, which steps can recover, and the run outcome. settled() decides readiness. stopWhere() decides what an end or a break does to each step under it. Each switches over every status, so a status added to StepStateSchema does not compile until all three place it.

• Only the transitions in step-state.ts write step fields. waitForAnswer, answerHitl, resetPending, resetIteration, skip, lease, release, start, succeed, fail, cancel, repend, nextRound, endLoop and exhaust change a step in place and drop the fields its new status does not have. readRun parses every step through StepStateSchema, which gives a run file from an older teamai its defaults: settledAt 0, outcome from status, and no fields its status does not have.

• run.steps is replaced, never changed in place. Its type is readonly, so addItems, insertChildren and archiveUnder add or remove instances by assigning a new array. An instance's id is readonly too, so an instance cannot move to another scope's group. statesIn in scope.ts keeps the instances of every scope per array, in a WeakMap, so settling a scope reads its own instances without scanning every step. A new array starts a new index, except that insertChildren, which adds the instances of a workflow a step starts, carries the old index over.

• A pull check that changes what a file means carries a code. dropsKeptVersion in cache.ts removes the cached copy when a diagnostic has a code. An older definition then cannot keep running with the new meaning. A diagnostic without a code keeps the last valid version, but only after checkReferences checks that version again against what the workflows it runs settle to in the same sync.

• The workflow cache holds only what one sync settled. checkReferences checks each file's version against the versions the other files settle to, and dropUnsettled removes every other cache entry. startRun relies on this: it reads the workflows a run uses from the cache and does not check them against each other again. A new way for a file to keep or lose its version goes through checkReferences.

• The items of a for_each step settle like a workflow. When settling reaches a for_each step, advanceInstance evaluates its list and addItems puts an instance per item right after the step; the step is running until every item settled. settleRunning settles a running for_each step through settleItems, the way it settles a workflow step through settleScope. An item waits for the items its after names and is skipped when one of them stopped (afterStanding). It starts only while fewer than max_parallel items are leased or running (startedItems, counted once per scan). A failed item cancels nothing; the step fails once every item settled (itemsSettled). The rules that decide the items are in fan-out.ts, the settling in engine.ts. Pull rejects for_each with loop, end or break (loopProblems).

• item is bound per instance, in the engine and in pull alike. contextOf gives an item its own value and any other instance its workflow's (Scope.item), which childScope passes into the inline workflow an item runs. A for_each step's own if and for_each therefore read the enclosing item, never their own. Pull checks the same binding: ReadSite.item marks the templates of a for_each step, and ReadScope.item a workflow that runs per item. A template reads item only where it is bound: parseTemplate takes the binding, which pull decides from the step and the engine from whether the context holds an item, so elsewhere {{ item }} stays text in both. A new place that reads item must decide its binding in both.

• Step ids are paths. A step inside a sub-workflow has an id such as ship.check.classify, an item of a for_each step one such as build[2], and a step inside the workflow an item runs one such as build[2].verify. parseStepPath in step-path.ts reads them all; don't split ids by hand. A child step exists in the run only after the engine reaches it. To find a step's definition, use stepDefinitions in scope.ts, which reads the definitions alone, so it also finds the step of an archived pass. locateStep in engine.ts also gives the step's scope, and needs the instances along the path. Don't look a step up by top-level id.

• What a step requires is decided once per lease, in negotiate. offerTerms in engine.ts builds the offer for the ready loop and the hand-back alike, and negotiate in requirements.ts answers grant, hold or fail. ENFORCEMENT classifies every requirement as strict, which holds the step for a driver whose can declares it (wait.held), or best effort, which grants it and records the gap in unenforced on the lease, the step and the record. A copy is used once an accepted report shows its step ran there: a success, an exitCode, changes or read (run.copies[].used); a staging or setup failure leaves it unused. Every lease on a copy in a workflow that declares setup carries it, so can.setup holds the lease, and the driver decides whether setup runs: when it creates the copy, or finds its setup unfinished (stageLease's marker in the local driver). A held hand-back cancels the attempt and releases the step; on an unused copy the driver also gives the copy up, so a capable driver can take the step, unless the driver still holds another lease in it, while a used copy keeps its owner, whose work lives there. A granted hand-back keeps its driver as the copy's owner (rememberCopy). The member's workflows.allow holds before any capability. Env values never enter the run: the lease carries key names in envScope (and setup.envScope), and the driver resolves them. Deadlines and budgets settle in reconcileRun(run, now), which runs expire (limits.ts) first; the run deadline and the run's budget count from run.limitsFrom, which a retry sets when it reopens the run or finds it ended, or else from createdAt with nothing spent; a workflow step's budget counts from the limitsFrom its start records (startLimits); a for_each step spends its budget over all its items together (itemsSpent in settleItems), and each item keeps only the budget of the workflow file it runs. An attempt the engine stops goes through stopAttempt, which records it in run.stoppedAttempts with its step, driver and stop time, whatever its step becomes later; a late report from that driver adds its tokens once (countLateTokens), and a budget whose limitsFrom started at or after the stop takes them into its baseline, so it does not pay for them. Known secret values are masked at the three exits, updateRun, enqueueRecord and writeEnvelope, each of which takes the secret set as a required argument. The set holds the value of each key env/secrets.yaml declares, wherever this machine resolves it from, and each environment value whose name looks secret; masking follows the team's declarations, not what a run requires. A command resolves it once and passes it on: startRun, withRun and the run controls require it, so what the envelope masks and what the run file masks agree; the local driver also masks an attempt's result.json before it reports, as text when it is not JSON, and every attempt's result file when it stops driving the run, so an attempt it never reported keeps no secret either.

• A HITL request waits before any action or end. settleScope checks loop exhaustion first, renders a HITL request message, and moves an unanswered step to waiting. Waiting is neither stopped nor settled. An answer moves it to pending without a revision bump. Approve no skips, choose succeeds, approve yes continues. An end-only step with HITL reaches its end only after yes. The shared templatedTexts makes HITL reads participate in readiness. Only driver.can.hitl declares HITL. parseNextPayload reads driver.can into NextOptions.can; stored DriverIdentity and leases contain only kind, machine and workspace. Busy polls never answer waiting HITL requests through the no-HITL rules. Only a driver declaring HITL receives a waiting request; a busy poll without HITL receives wait. An interactive step is a requirement, not a gate: negotiate holds it for a driver without HITL, on new leases and hand-backs alike. The process-only autoApprove option is never read by the JSON payload parser. respond --json uses the same identity schema as next, reports unread fields through ignored, and records a driver UI's declared person answer with channel driver and agent false. The dashboard answers through the same respondToHitl with channel dashboard and agent false, always sending the pass it showed (answerFromDashboard in src/dashboard/workflow-runs.ts). Answer driver identity stays local; records publish only choice, channel and agent.

• Retry validates every container before resetting any. retryContainers in run-controls.ts climbs from the target through scopes and fan-out items until a running runner or the top. Each dependency cone includes needs, reads and loop waits, or item after dependencies. An independent standing end or break rejects the retry. The apply pass archives replaced passes and descendants, reopens settled runners with start to keep inputs and round, then bumps revision and reconciles. Only the targeted loop owner's iteration resets; the top-level round does not. An answered refused approve or succeeded choose is retryable too, so a changed answer resets its dependent branches. A HITL request in the cone that is waiting, or answered and not yet run, starts a new pass too, so its answer is asked again in the new context and an answer to the old pass is stale. Unanswered skips in every affected container reset so skips caused by cancellation can run again. Retry's prelude reconciles without finishing a provisional outcome. Reopening clears the run's end, failure and endRecordQueued flag and removes its queued end file through records.ts. These three pieces stay consistent: run state, the pending end record, and published stats. The next end upserts the same record path; stats replace the origin end's outcome and duration contribution with the new end's, without adding another run. Replaying the same endedAt contributes nothing.

• The local driver keeps tools and questions interruptible. A single question joins command and prompt exits in the driver's race. An interactive tool owns the terminal alone: no question opens while it runs, and starting one aborts and closes an open question. A poll closes the question when its own step is leased or a different step or pass waits; an unrelated lease leaves it open. It polls next every second while asking, every two seconds while tools run, and aborts a replaced HITL request or cancelled prompt. src/tool-launch/process-stop.ts shares process stopping between commands and adapters. Headless tools detach and stop with SIGTERM; interactive tools stay attached and stop with their captured descendant tree. Foreground interactive signals reuse env-exec's terminal detection and signal policy: when the terminal reaches the child, teamai ignores SIGINT and SIGQUIT, while SIGTERM and SIGHUP are forwarded and stop the driver. Interactive adapters inherit stdio and accept an optional result file. Repository edits are kept for interactive steps; only headless prompts run the read-only check. A nonzero interactive exit without a result reports its exit status. leaseNext records the identity of the latest driver that called next in RunState.driver, without capabilities. Resume uses this machine's open local lease workspace, then run.driver.workspace for a local driver on this machine, then cwd. Controls pass run.driver to finishRun so end records retain the driver kind.

• One local driver owns a run on this machine. withDriverLock in local-driver.ts takes driverLockPath once with acquireLock and releases it in finally. runLocalDriver drives inside it, and workflow resume also claims inside it, so a claim never lands under a driver already running the run. A live holder returns driver_running, including its pid, before anything is claimed or driven; the existing lock machinery reclaims dead owners. The driver lock spans tool execution and questions, while withRun holds a different run lock only during one engine operation. Neither lock's scope is extended by the other. Every HITL decision without a HITL driver logs once when the engine changes the step. HITL replies carry one literal respond command template, which drivers display unchanged.

• Judge work never comes from workflow next. scoring.ts computes open judge work on every teamai score --next from facts already on disk: the run files, the scorers startRun pinned in run.scorers, the installId that reportAttempt stamps on each pass it settles, the within stamp on archived passes, and the labels, attempts and on-demand requests (teamai score --run) under <dataHome>/scores/. Nothing else is queued. The engine has no scoring code path, and scoring never writes a run file, so a run's counts, tokens, failure and endedAt cannot move and it settles without waiting. A judge lease settles only through teamai score: --lease <ref> --submit, or --lease <ref> and --run, which run the local judge and settle through the same submit path. workflow report answers judge_lease. A driver never lists a judge ref in workflow next's running, which holds step attempts only: judge refs go in score --next's running, which renews them.

Tests

Test parsing through parseWorkflow and syncWorkflowDefinitions. Test the engine through startRun, leaseNext, and reportAttempt. Every driver must pass workflowConformanceScenarios in src/workflow/conformance.ts, which runConformanceScenario in src/workflow/conformance-interpreter.ts runs, and a driver that takes workspace leases the git scenarios in conformance.ts too. That one list takes in the scenarios a concern keeps in its own file, such as limits-conformance.ts, so a runner cannot miss one. If a driver can see a new behavior, add a scenario to the file for its concern; a new file joins the list in conformance.ts.