Status: draft, pending review by @havardjohansen13 before circulation. No assignees, no broader @-mentions yet.
Context
Three consecutive Night Shift runs on the merkur-frontend + distriktssenteret-frontend sister sites have shown the same five patterns. Documented per-run in /Users/havardjohansen/Dev/night-shift/IMPROVEMENTS.md entries 2026-05-07, 2026-05-08, 2026-05-11 (private workspace file — happy to copy excerpts here on request).
Pattern 1 — Stale-base detection
Night Shift's per-task PR is correct relative to the branch checkout at run-start, but it doesn't see what the previous run already shipped. Result: DS produced word-for-word regenerations of fixes already on develop three days running.
Concrete examples from 2026-05-11:
Proposed fix: before opening a PR for a task, diff against not just develop but the most recent merged night-shift/* bundle (or the last N bundles if cadence is unclear). If the proposed change is structurally already present, skip the task with a brief log line.
Pattern 2 — DO-NOT-CHANGE policy blocks ignored
The 2026-05-11 DS #67 PR would have reverted <html lang=\"no\"> → lang=\"nb\" and og:locale: nn_NO → nb_NO. Both are explicitly forbidden by a block titled ## Language policy — DO NOT CHANGE near the top of the repo's CLAUDE.md:
Do not "fix" lang=\"no\" to lang=\"nn\" or lang=\"nb\" — that would misrepresent the rendered page.
This is the first time we've seen Night Shift actively regress a documented policy.
Proposed fix: when reading CLAUDE.md / AGENTS.md, parse ## (or deeper) headings whose text contains DO NOT CHANGE (or similar policy markers) and treat the contents as hard constraints. SEO task in particular should never modify lang/locale attributes if such a block exists.
Pattern 3 — Test PRs open with red CI
DS #64 (2026-05-11) and DS #48 (2026-05-08) both opened with red Cypress E2E. The new specs referenced components/tokens that exist on merkur but not yet on DS ([data-cy=\"ai-search-block\"], .u-head-sm, <h2.u-head-lg>).
Proposed fix: in the test-add task, after opening the PR, poll gh pr checks for N minutes (or gh run watch). If E2E fails, automatically close the PR with a comment explaining the failure — don't leave it for a human to triage. Better still: run the new tests locally against the project's dev server before opening the PR.
Pattern 4 — Docs/changelog/ADR/suggestions cadence
Three days, 12+ closed PRs across this category, zero merged. Pattern is consistent: README refresh, CHANGELOG init, ADR-0001 (shared codebase decision), "3 suggestions in docs/SUGGESTIONS.md". Reviewers don't merge them because the content doesn't change between runs — Night Shift keeps regenerating the same suggestions.
Proposed fix: move docs/changelog/ADR/suggestions tasks to a weekly (not nightly) cadence, OR make them opt-in per repo, OR have them edit existing files instead of always proposing new ones. Currently they're pure noise on the nightly review surface.
Pattern 5 — Security task gating on already-fixed targets
Security task fires nightly. It correctly identifies open-redirect / unauthenticated-route patterns, but its sense of "what's already fixed" is wrong (see pattern 1). The "Risk: Medium" tag on each PR has been the only thing keeping reviewers from accidentally merging regressions of already-applied fixes. Without that tag, the cost of mishandling these PRs would be high.
Proposed fix: either (a) skip the security task entirely if pattern 1 (stale-base detection) lands and would catch these, or (b) move security to a less frequent cadence (weekly) with assign-for-review rather than "Risk: Medium" auto-PRs, until pattern 1 ships.
Suggested triage
- High priority: pattern 1 (stale-base detection) — root cause for patterns 5 and roughly half of pattern 3.
- Medium: pattern 2 (DO-NOT-CHANGE parser).
- Medium: pattern 4 (cadence reduction for docs/suggestions).
- Low (but cheap): pattern 3 (self-CI-check on test PRs).
Pattern 5 dissolves once pattern 1 lands.
Cross-reference
Sister sites: frontkom/merkur-frontend, frontkom/distriktssenteret-frontend. Today's bundled output:
- merkur PR #142 (3 mergeable items extracted from 6 generated)
- DS PR #72 (1 mergeable item extracted from 8 generated — that's a 12.5% signal rate)
The cost of bundling is currently ~30–45 min of human review per repo per day. If patterns 1, 4, and 5 land, that should drop to ~10 min.
🤖 Drafted by Claude Code; awaiting Håvard's review before circulation.
Context
Three consecutive Night Shift runs on the
merkur-frontend+distriktssenteret-frontendsister sites have shown the same five patterns. Documented per-run in/Users/havardjohansen/Dev/night-shift/IMPROVEMENTS.mdentries2026-05-07,2026-05-08,2026-05-11(private workspace file — happy to copy excerpts here on request).Pattern 1 — Stale-base detection
Night Shift's per-task PR is correct relative to the branch checkout at run-start, but it doesn't see what the previous run already shipped. Result: DS produced word-for-word regenerations of fixes already on
developthree days running.Concrete examples from 2026-05-11:
developalready hasSANITY_PREVIEW_SECRETvalidation +isRelativeUrl()insrc/app/api/draft-mode/enable/route.ts. The PR would have regressed toSANITY_REVALIDATE_SECRET(wrong env var name).sizesprop and the iframeloading=\"lazy\"already on develop from the 2026-05-08 bundle.latin-extRubik subset.Proposed fix: before opening a PR for a task, diff against not just
developbut the most recent mergednight-shift/*bundle (or the last N bundles if cadence is unclear). If the proposed change is structurally already present, skip the task with a brief log line.Pattern 2 — DO-NOT-CHANGE policy blocks ignored
The 2026-05-11 DS #67 PR would have reverted
<html lang=\"no\">→lang=\"nb\"andog:locale: nn_NO→nb_NO. Both are explicitly forbidden by a block titled## Language policy — DO NOT CHANGEnear the top of the repo'sCLAUDE.md:This is the first time we've seen Night Shift actively regress a documented policy.
Proposed fix: when reading
CLAUDE.md/AGENTS.md, parse##(or deeper) headings whose text containsDO NOT CHANGE(or similar policy markers) and treat the contents as hard constraints. SEO task in particular should never modifylang/localeattributes if such a block exists.Pattern 3 — Test PRs open with red CI
DS #64 (2026-05-11) and DS #48 (2026-05-08) both opened with red Cypress E2E. The new specs referenced components/tokens that exist on merkur but not yet on DS (
[data-cy=\"ai-search-block\"],.u-head-sm,<h2.u-head-lg>).Proposed fix: in the test-add task, after opening the PR, poll
gh pr checksfor N minutes (orgh run watch). If E2E fails, automatically close the PR with a comment explaining the failure — don't leave it for a human to triage. Better still: run the new tests locally against the project's dev server before opening the PR.Pattern 4 — Docs/changelog/ADR/suggestions cadence
Three days, 12+ closed PRs across this category, zero merged. Pattern is consistent: README refresh, CHANGELOG init, ADR-0001 (shared codebase decision), "3 suggestions in
docs/SUGGESTIONS.md". Reviewers don't merge them because the content doesn't change between runs — Night Shift keeps regenerating the same suggestions.Proposed fix: move docs/changelog/ADR/suggestions tasks to a weekly (not nightly) cadence, OR make them opt-in per repo, OR have them edit existing files instead of always proposing new ones. Currently they're pure noise on the nightly review surface.
Pattern 5 — Security task gating on already-fixed targets
Security task fires nightly. It correctly identifies open-redirect / unauthenticated-route patterns, but its sense of "what's already fixed" is wrong (see pattern 1). The "Risk: Medium" tag on each PR has been the only thing keeping reviewers from accidentally merging regressions of already-applied fixes. Without that tag, the cost of mishandling these PRs would be high.
Proposed fix: either (a) skip the security task entirely if pattern 1 (stale-base detection) lands and would catch these, or (b) move security to a less frequent cadence (weekly) with
assign-for-reviewrather than "Risk: Medium" auto-PRs, until pattern 1 ships.Suggested triage
Pattern 5 dissolves once pattern 1 lands.
Cross-reference
Sister sites:
frontkom/merkur-frontend,frontkom/distriktssenteret-frontend. Today's bundled output:The cost of bundling is currently ~30–45 min of human review per repo per day. If patterns 1, 4, and 5 land, that should drop to ~10 min.
🤖 Drafted by Claude Code; awaiting Håvard's review before circulation.