feat(temporal): durable pause. A run stops at a node and waits for a human verdict - #120
Merged
piotrblaszczyk merged 15 commits intoSep 8, 2026
Merged
Conversation
WB-497 step 1: a parked run reports 'waiting' while a gate holds it. Deliberately absent from TERMINAL_EXECUTION_STATUSES.
…t chain EventEmitterPort, Activities, ExecutionStore and the worker database now take ExecutionEventType / ExecutionStatus instead of bare strings, so a typo like 'canceled' fails to compile instead of silently never closing a run. EventEmitterPort is restated in the temporal package's sandbox-safe core-contract (package-name imports survive into the bundled d.ts), with the drift pin extended in core-contract.test.ts. WB-497 step 1b.
…tResolution port NodeExecutionResult is now a union: CompletedNodeExecution or WaitingNodeExecution, discriminated by the waiting marker, so reading .output without checking no longer compiles while existing executors stay source- compatible. ActivityRunnerPort gains optional awaitResolution(nodeId) resolving to the completion the verdict carries. A waiting result on an adapter without the port is a runner-level abort routed around errorPolicy: under 'continue' it would close the run as completed with the gate silently skipped. WB-497 step 2.
A waiting result now parks its wave slot: node_waiting is emitted, the run status goes 'waiting' on the first park and back to 'running' after the last resolution (parked is a counter, so concurrent gates share one transition), and the verdict's completion rejoins the normal path — node_completed, output, nextPort. The missing-port abort carries code 'waiting_unsupported'. WB-497 step 3.
defineUpdate('resolveNode') carries { nodeId, resolution } where resolution is
the CompletedNodeExecution the parked node finishes with, passed through
untouched. The handler and the resolutions map live inside the workflow
function (per-instance), first-write-wins rejects a duplicate verdict
synchronously with 'verdict_already_delivered', and awaitResolution is a
deadline-free condition() so the gateless history is untouched. Decisions
recorded in durable-pause.decision-log.md.
WB-497 step 4.
…g for durable pause Three harness scenarios: a parked run survives a worker restart and the resolveNode update resumes it with downstream running exactly once; two gates in one wave park concurrently, take verdicts independently and reject a duplicate synchronously; cancelling a parked run closes it as cancelled with execution_cancelled after node_waiting and no node_failed for the gate. WB-497 step 5.
Adds the 'Pausing a run for a human' section: the waiting result, node_waiting and the waiting/running status pair, delivering a verdict through the resolveNode update, first-write-wins, cancel behaviour, and the three deliberate wave-barrier limitations. The duplicate-verdict test now pins the verdict_already_delivered failure type the README promises, and an unused WaitingNodeExecution re-export flagged by knip is dropped. WB-497 step 6.
piotrblaszczyk
requested review from
librowski,
lukasz-jazwa and
szymon-t-sc
as code owners
September 4, 2026 12:42
A non-TemporalFailure thrown from an update handler fails the workflow task, which retries and redelivers forever: one malformed verdict wedged a parked run for good. The validator runs before acceptance instead, so a rejection writes nothing to history and cannot fail the task. It guards engine integrity only: envelope shape (verdict_malformed, including the reserved errorRoute port), the node exists (verdict_for_unknown_node), first-write-wins (verdict_already_delivered, moved out of the handler) and the node is actually waiting (node_not_waiting, retryable when a verdict races the parking activation). Gate lifecycle now lives in one per-run wait-state map: idle (absent) -> waiting -> resolved. WB-497 step 7, after external review.
… verdict The waiting/running writes are derived state: their failure is now swallowed (setAdvisoryStatus), so an exhausted DB activity no longer turns a parking gate into node_failed or evicts a delivered verdict from the finally block. The parked counter increments inside the try, closing a leak when the waiting write threw before it. updateStatus now chains behind the sequenced emitter's event tail, so a 'waiting' from one gate can never race past a 'running' from another. Terminal statuses stay loud on purpose: nothing after them would heal a silently missed write. WB-497 step 8, review round 2.
…nning stamps it Durable pause made the gate un-park the first writer of 'running', which armed the previously dead started_at CASE: a verdict was stamping (and every later resume re-stamping) the start timestamp, visible in the API. The IS NULL guard stamps only the first transition to 'running'. Writing 'running' at actual run start stays a separate fix (follow-up: running-status-at-start). WB-497, review round 3.
The committed v0-parked-gate.json carries the accepted resolveNode update, node_waiting, the waiting/running status activities and the resume, so a command moved on the parked path fails CI even while the gateless baseline stays green (verified: an extra emit turns only this pin red, with a DeterminismViolationError). A full park-and-resume history stands in for every prefix, so it also guards runs still waiting when a deploy lands. Shared test helpers move to fixtures/helpers.ts. WB-497, review round 4.
…opped The default payload converter is JSON, so resolution.output: undefined reaches the workflow with no output key at all. The validator required the key and rejected such a verdict as verdict_malformed, leaving the run parked. A missing output now reads as undefined; arrays are still rejected, and the key, nextPort and wait-state checks are unchanged. Verified through the real converter in the unit test and against a local Temporal in the durable-pause suite, on a port-routed fixture so nextPort has an edge to reach.
The branch collects the human-in-the-loop feature PRs before they reach main, so they need the same gate. Nothing else in the workflow was tied to main: the changeset guard already diffs against github.base_ref.
…istory WorkflowUpdateFailedError with the expected type would also come back from an accepted handler that threw later, so the rejection test now gives the malformed updates explicit ids and checks that history holds exactly one accepted update, neither of them. The cancel test checks the failure cause is a CancelledFailure and that the run's last event is WorkflowExecutionCanceled, not just that result() rejected.
…test helper Deleting the current element while iterating a Set skips nothing, so the reverse index loop with splice is no longer needed. Notification order flips to registration order; it was never a contract, since notify only resolves a promise and the continuations run after the loop.
librowski
approved these changes
Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A run can now stop at a chosen node for an unbounded time (days, weeks), survive
worker restarts while suspended, and resume exactly at that node when a verdict
arrives. This is the seam every other HITL capability builds on.
Best reviewed commit by commit: each step is isolated and lands green.
How it works
{ waiting: true }instead of a completion.NodeExecutionResultis now a union (CompletedNodeExecution | WaitingNodeExecution),so reading
.outputwithout checking does not compile, while existing executorsstay source-compatible.
node_waitingand awaits the new optionalActivityRunnerPort.awaitResolution(nodeId)in the same wave slot. The wave barrieris untouched: the pause is nothing more than an unresolved
Promise.allslot.waitingon the first park, back torunningafter the lastresolution (counter, not a flag, so concurrent gates share one transition).
waitingis a new non-terminalExecutionStatus.resolveNode)plus a deadline-free
condition(). The update carries{ nodeId, resolution },where
resolutionis the completion the node finishes with (output,nextPort),passed through untouched. First write wins; a duplicate verdict is rejected
synchronously (
verdict_already_delivered).waiting_unsupported, deliberatelybypassing
errorPolicy: a configuration error must not close a run as completedwith the gate silently skipped.
Design rationale lives in
packages/temporal/src/workflow/durable-pause.decision-log.md.Compatibility
A gateless history recorded before this change replays unchanged (pinned by the
committed-history replay test). Registering the update handler writes nothing to
Event History and the deadline-free condition creates no timer command, so the
gateless path produces the identical command sequence as before.
Tests
nextPort, two gates in onewave, a fatal sibling holding the run until the verdict, missing-port fail-fast.
verdicts plus duplicate rejection, cancel-while-waiting (
execution_cancelledafter
node_waiting, nonode_failedfor the gate).GET /api/executions/:idshowswaiting,worker kill and restart, verdict via Temporal CLI, run completes with the verdict
output visible downstream.
Review pointers
graph-runner.ts: the park path and the abort-vs-errorPolicy routing.run-workflow.ts: per-instance handler state (must not live in the factoryclosure) and the first-write-wins map.
non-goals, including that a fatal failure in the same wave cannot close the run
until the verdict arrives.