From a8d1efab14a7d399e7a514c29d414f153122edbb Mon Sep 17 00:00:00 2001 From: Douglas Ezra Morrison Date: Fri, 31 Jul 2026 22:19:19 -0700 Subject: [PATCH 1/2] ums: a background watcher reports failure as silence by default fail-fast covers a by-hand check whose failure and pass paths print the same thing. A watcher is worse: its output channel is a notification, so its silence is what you asked for, and a bounded poll loop that prints only when the condition is met exits quietly when it never is. Adds the terminal-else and emit-on-every-terminal-state fixes, and records that reading the Monitor tool's own coverage guidance did not prevent writing the bug twice in one hour. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01HMmB6db4xpbuqcqaQKArZb --- shared/principles/fail-fast.md | 57 ++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/shared/principles/fail-fast.md b/shared/principles/fail-fast.md index f91e4a50b..cfe1a58f3 100644 --- a/shared/principles/fail-fast.md +++ b/shared/principles/fail-fast.md @@ -233,6 +233,63 @@ classifier denied minutes earlier. A later run of the fixed script reported 910 of 947, which is how the rate-limit truncation above was found.) +### A background watcher reports failure as silence by default + +The two cases above are checks you read the output of. +A watcher is one you deliberately stop reading, which is its whole purpose --- +so its output channel is a *notification*, and the absence of one is +indistinguishable from the thing still running. + +That inverts the usual economics of this bug. +A silent `|| echo "none"` at least sits in front of you. +A watcher's silence is what you asked for: quiet means nothing to report, +which is exactly what a healthy long-running job looks like. +So the failure is not merely unnoticed, it is *reassuring*. + +The shape is a poll loop that emits only on the happy path: + +```sh +for i in $(seq 1 25); do + pending=$(...) + if [ "$pending" = 0 ]; then echo "settled: ..."; break; fi + sleep 60 +done # <- falls out silently when it never settles +``` + +Every iteration finds work still pending, the loop exhausts its range, and the +script exits 0 having printed nothing. +Nothing failed, so nothing is reported, and the watcher's silence gets read as +"not finished yet" indefinitely. + +Two fixes, and take both. +Give the loop a **terminal else**, so exhausting the range says so out loud and +names what it was waiting for. +And emit on **every** state you would act on, not just the one you hope for --- +a failed check, a blocking verdict, a job that vanished. + +Note the second is the same instruction the Monitor tool's own documentation +gives ("if this process crashed right now, would my filter emit anything?"), +which is worth saying because reading that guidance is evidently not sufficient +to follow it. + +- **Do:** end a bounded poll loop with an explicit timeout message naming the + condition that never arrived. +- **Do:** widen the filter to every terminal state, then confirm by asking what + the watcher would have printed had the job died at the start. +- **Don't:** read a watcher's quiet as evidence the work is still in flight. +- **Don't:** treat "I read the tool's guidance about coverage" as having applied + it. + +(2026-08-01, a `UCD-SERG/ucd-serg.github.io` session: two successive monitors +watching a PR's checks exited silently after 25 minutes, both written to print +only when zero checks were pending. +The first hid a red `validate`; the second hid nothing but was equally +uninformative. +Both were caught by querying the PR directly rather than by anything the +watchers did, and the second was armed *after* writing a status note about the +first --- so knowing the failure mode did not prevent repeating it within the +hour.) + #### A zero-shaped summary can be sound, and the scope line is what decides it The rule above has a false-positive direction, and it lands on exactly the From bb5628e97402de51ae8a213d0984f4a1b572a5f2 Mon Sep 17 00:00:00 2001 From: Douglas Ezra Morrison Date: Fri, 31 Jul 2026 22:38:12 -0700 Subject: [PATCH 2/2] review: move the watcher section below the zero-shaped-summary caveat The insertion sat between the fan-out section and its own H4 caveat, breaking cross-references that depend on that adjacency and re-parenting the H4 under the new H3 -- implying the scan-count caveat was about polling loops. Also drops a count from the opening line ("the two cases above" would now read as three) and cross-links the pipe-stage route to the same silence already recorded in memories/claude-code.md, which my dupe-check grep missed. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01HMmB6db4xpbuqcqaQKArZb --- shared/principles/fail-fast.md | 105 ++++++++++++++++++--------------- 1 file changed, 57 insertions(+), 48 deletions(-) diff --git a/shared/principles/fail-fast.md b/shared/principles/fail-fast.md index cfe1a58f3..0bd94cd2d 100644 --- a/shared/principles/fail-fast.md +++ b/shared/principles/fail-fast.md @@ -233,9 +233,56 @@ classifier denied minutes earlier. A later run of the fixed script reported 910 of 947, which is how the rate-limit truncation above was found.) +#### A zero-shaped summary can be sound, and the scope line is what decides it + +The rule above has a false-positive direction, and it lands on exactly the +tools that already comply with it. + +A well-behaved instrument prints its scope --- which is the remedy this +section asks for --- but it prints it on a **different line** from its +summary, and the summary can be phrased so that it reads as the vacuous-scan +signature: + +``` +Linting: 439 files +Summary: 0 issues in 0 files +``` + +That is `markdownlint-cli2`. +`0 files` counts **files with issues**, not files scanned. +So the line that looks like "this examined nothing" is the line reporting +that nothing was wrong, and the evidence against that reading is sitting two +lines up. + +The failure this produces is not a swallowed error but a needless +retraction: you report your own check as having verified nothing, withdraw a +true claim, and spend a round re-running an instrument that was fine. +That is the same cost the fragment warns about elsewhere --- a check nobody +trusts stops being run --- arriving from over-application rather than from +under-application. + +So read for the scope line before concluding a zero is vacuous, and quote it +alongside the result rather than quoting the summary alone. +Where a tool prints no scope at all, the original rule stands unchanged: that +zero is not yet evidence. + +- **Do:** look for a scanned/examined count on its own line before calling a + zero-hit result vacuous. +- **Do:** report the scope and the finding together --- "439 files linted, 0 + issues" cannot be misread in either direction. +- **Don't:** read a summary's "0 files" as the number examined without + checking what that tool counts. +- **Don't:** retract a check as vacuous on the strength of one line of its + output. + +(Morrison-Lab/ai-config#974, 2026-07-31: a `markdownlint-cli2` result already +published in a PR body as `0 issues in 0 files` was about to be re-reported as +a check that examined nothing. +Re-running it printed `Linting: 439 files` above the same summary.) + ### A background watcher reports failure as silence by default -The two cases above are checks you read the output of. +The cases above are all checks you read the output of. A watcher is one you deliberately stop reading, which is its whole purpose --- so its output channel is a *notification*, and the absence of one is indistinguishable from the thing still running. @@ -280,6 +327,15 @@ to follow it. - **Don't:** treat "I read the tool's guidance about coverage" as having applied it. +A second route to the same silence, with a different cause, is recorded in +[`memories/claude-code.md`](../../memories/claude-code.md): a pipe stage that +consumes the content a later stage was meant to read (`grep -q`, `-l`, or `-c` +upstream of something that greps stdout) starves the loop of anything to emit. +That one is about what reaches the filter and this one is about what the filter +is written to match, so the fixes differ --- but the symptom is identical, and +in both cases the discrepancy surfaced only by running the underlying query by +hand. + (2026-08-01, a `UCD-SERG/ucd-serg.github.io` session: two successive monitors watching a PR's checks exited silently after 25 minutes, both written to print only when zero checks were pending. @@ -290,53 +346,6 @@ watchers did, and the second was armed *after* writing a status note about the first --- so knowing the failure mode did not prevent repeating it within the hour.) -#### A zero-shaped summary can be sound, and the scope line is what decides it - -The rule above has a false-positive direction, and it lands on exactly the -tools that already comply with it. - -A well-behaved instrument prints its scope --- which is the remedy this -section asks for --- but it prints it on a **different line** from its -summary, and the summary can be phrased so that it reads as the vacuous-scan -signature: - -``` -Linting: 439 files -Summary: 0 issues in 0 files -``` - -That is `markdownlint-cli2`. -`0 files` counts **files with issues**, not files scanned. -So the line that looks like "this examined nothing" is the line reporting -that nothing was wrong, and the evidence against that reading is sitting two -lines up. - -The failure this produces is not a swallowed error but a needless -retraction: you report your own check as having verified nothing, withdraw a -true claim, and spend a round re-running an instrument that was fine. -That is the same cost the fragment warns about elsewhere --- a check nobody -trusts stops being run --- arriving from over-application rather than from -under-application. - -So read for the scope line before concluding a zero is vacuous, and quote it -alongside the result rather than quoting the summary alone. -Where a tool prints no scope at all, the original rule stands unchanged: that -zero is not yet evidence. - -- **Do:** look for a scanned/examined count on its own line before calling a - zero-hit result vacuous. -- **Do:** report the scope and the finding together --- "439 files linted, 0 - issues" cannot be misread in either direction. -- **Don't:** read a summary's "0 files" as the number examined without - checking what that tool counts. -- **Don't:** retract a check as vacuous on the strength of one line of its - output. - -(Morrison-Lab/ai-config#974, 2026-07-31: a `markdownlint-cli2` result already -published in a PR body as `0 issues in 0 files` was about to be re-reported as -a check that examined nothing. -Re-running it printed `Linting: 439 files` above the same summary.) - ### The pattern itself is the other half, and it fails without erroring Everything above is about a check that *cannot report* its own failure.