Skip to content

fix(runner): clean claude runner checkouts after every session - #244

Merged
rfay merged 2 commits into
mainfrom
20261111_rfay_runner_checkout_cleanup
Nov 11, 2026
Merged

rfay merged 2 commits into
mainfrom
20261111_rfay_runner_checkout_cleanup

Conversation

@rfay

@rfay rfay commented Nov 11, 2026

Copy link
Copy Markdown
Member

Short Summary (TL;DR)

Runner checkouts are now cleaned after every session and before every runner start. A session's leftovers are kept as stashes or rescue directories, so a new session no longer starts in a mess it can't clean.

The Issue

A runner keeps its clones in ~/workspace/runner-<n> from one session to the next. Nothing cleaned them, so sessions left uncommitted changes, untracked files, worktrees and extra clones behind. The runner's --release-idle-session-min 30 releases a session after 30 idle minutes, dirty or not. Coder stopping the workspace kills the runner mid-session.

The next session found those files and was told not to delete them without the user's approval. A web session often can't get that approval, and the skill suggested a worktree for a second checkout. So sessions created worktrees and clones to get around the leftovers, which made the next checkout worse.

The SessionStart hook added in #234 also ran on compact and clear. At that point the dirty files are the session's own work.

How This PR Solves The Issue

  • New claude-runner-sweep <runner dir> cleans one runner's directory, and loses nothing a commit or remote doesn't already hold.
    • In the runner's own clones, it stashes uncommitted changes, including untracked files, as runner-rescue stashes.
    • It stashes changes in worktrees the same way, then removes the worktrees.
    • It deletes other clones a session made if everything in them is pushed. Otherwise it moves them, and broken worktrees, to ~/.local/state/claude-runner-rescue/<runner>/<time>/.
    • It logs each action, with the session ID, to ~/.local/state/claude-runner-rescue/log. Rescues older than 30 days are removed.
  • The sweep runs as each runner's post-session lifecycle hook (--hooks-dir). The loop also runs it before every runner start, which covers runners killed before the hook ran. Each runner has --capacity 1, so no session uses the directory during a sweep.
  • The SessionStart hook now runs only on startup and resume. It tells Claude to stash any leftovers rather than work around them. A resumed session whose changes were saved is told where they went.
  • The coder-shared-workspace skill no longer suggests worktrees. It forbids worktrees or clones made to avoid leftovers. It tells Claude to commit before waiting on the user, because an idle release can come without warning.

Manual Testing Instructions

Push the template and use a freeform workspace with the runner turned on:

make push-template-freeform
coder update <workspace>
ssh <workspace>.coder cat ~/.claude-runner/hooks/runner-1/post-session
# Expect: exec "$HOME/.local/bin/claude-runner-sweep" /home/coder/workspace/runner-1

In a claude.ai/code session for that environment, ask Claude to create an untracked file and a worktree, then end or archive the session.

ssh <workspace>.coder cat ~/.local/state/claude-runner-rescue/log
# Expect "stashed changes ..." and "removed worktree ..." lines with the session ID
ssh <workspace>.coder git -C /home/coder/workspace/runner-1/<owner>/<repo> stash list
# Expect a "runner-rescue <time> session_... checkout on <branch>" entry
grep 'post-session hook' /tmp/claude-runner-1.log
# Run on the workspace. Expect "post-session hook starting ... via .../hooks/runner-1/post-session"

Start a new session in the same environment. Its opening context must not report a dirty checkout.

Automated Testing Overview

  • freeform/tests/validate.tftest.hcl checks that the startup script installs claude-runner-sweep and gives each runner --hooks-dir. It also checks that the SessionStart hook matches only startup|resume.
  • Claude Code ran make validate and make test-templates on macOS. All 22 tests passed.
  • Claude Code ran claude-runner-sweep in an ubuntu:24.04 container against a fake runner directory. The directory held a dirty runner clone, a dirty worktree, and a worktree with a merge conflict. It also held an extra clone with an unpushed commit, a pushed extra clone, a broken worktree, and a repository under _sessions/. Each one was stashed, removed, deleted or moved as described above. The _sessions/ repository was left alone, and a second run changed nothing. A separate run checked that expired runner-rescue stashes are dropped and other stashes are kept.
  • Nobody has run this in a live workspace with real runner sessions. The post-session hook's file name and environment come from the claude self-hosted-runner binary, Claude Code 2.1.319.
  • Nobody has checked that the SessionStart hook's session_id equals the runner's CLAUDE_RUNNER_SESSION_UUID. If they differ, a resumed session is not told where its saved changes went, and nothing else changes.

Second commit: ~/.claude-runner README

When the runner setting is on, the workspace now creates ~/.claude-runner with mode 700 before its other checks. It writes a README.md there on every start. The README covers saving the secret, the hooks/ directory, logs, claude-sessions, checkouts, rescued work, and how to stop the runner. The "runner NOT started" message points to it, and the docs setup steps no longer need mkdir. A test checks that the README is written and names ~/.claude-runner/environment-secret.

Release/Deployment Notes

  • Existing runner workspaces get the sweep on their next start after coder update. The first sweep cleans everything earlier sessions left. Unpushed clones are moved to ~/.local/state/claude-runner-rescue/, so look there for anything missing.
  • An unfinished merge, rebase, cherry-pick or revert in a checkout is aborted before stashing. Its commits stay on their branch.

🤖 Generated with Claude Code

rfay and others added 2 commits November 11, 2026 10:25
## Short Summary (TL;DR)

Runner checkouts are now cleaned after every session and before every runner start. A session's leftovers are kept as stashes or rescue directories, so a new session no longer starts in a mess it can't clean.

## The Issue

A runner keeps its clones in `~/workspace/runner-<n>` from one session to the next. Nothing cleaned them, so sessions left uncommitted changes, untracked files, worktrees and extra clones behind. The runner's `--release-idle-session-min 30` releases a session after 30 idle minutes, dirty or not. Coder stopping the workspace kills the runner mid-session.

The next session found those files and was told not to delete them without the user's approval. A web session often can't get that approval, and the skill suggested a worktree for a second checkout. So sessions created worktrees and clones to get around the leftovers, which made the next checkout worse.

The SessionStart hook added in #234 also ran on `compact` and `clear`. At that point the dirty files are the session's own work.

## How This PR Solves The Issue

- New `claude-runner-sweep <runner dir>` cleans one runner's directory, and loses nothing a commit or remote doesn't already hold.
  - In the runner's own clones, it stashes uncommitted changes, including untracked files, as `runner-rescue` stashes.
  - It stashes changes in worktrees the same way, then removes the worktrees.
  - It deletes other clones a session made if everything in them is pushed. Otherwise it moves them, and broken worktrees, to `~/.local/state/claude-runner-rescue/<runner>/<time>/`.
  - It logs each action, with the session ID, to `~/.local/state/claude-runner-rescue/log`. Rescues older than 30 days are removed.
- The sweep runs as each runner's `post-session` lifecycle hook (`--hooks-dir`). The loop also runs it before every runner start, which covers runners killed before the hook ran. Each runner has `--capacity 1`, so no session uses the directory during a sweep.
- The SessionStart hook now runs only on `startup` and `resume`. It tells Claude to stash any leftovers rather than work around them. A resumed session whose changes were saved is told where they went.
- The `coder-shared-workspace` skill no longer suggests worktrees. It forbids worktrees or clones made to avoid leftovers. It tells Claude to commit before waiting on the user, because an idle release can come without warning.

## Manual Testing Instructions

Push the template and use a freeform workspace with the runner turned on:

```bash
make push-template-freeform
coder update <workspace>
ssh <workspace>.coder cat ~/.claude-runner/hooks/runner-1/post-session
# Expect: exec "$HOME/.local/bin/claude-runner-sweep" /home/coder/workspace/runner-1
```

In a claude.ai/code session for that environment, ask Claude to create an untracked file and a worktree, then end or archive the session.

```bash
ssh <workspace>.coder cat ~/.local/state/claude-runner-rescue/log
# Expect "stashed changes ..." and "removed worktree ..." lines with the session ID
ssh <workspace>.coder git -C /home/coder/workspace/runner-1/<owner>/<repo> stash list
# Expect a "runner-rescue <time> session_... checkout on <branch>" entry
grep 'post-session hook' /tmp/claude-runner-1.log
# Run on the workspace. Expect "post-session hook starting ... via .../hooks/runner-1/post-session"
```

Start a new session in the same environment. Its opening context must not report a dirty checkout.

## Automated Testing Overview

- `freeform/tests/validate.tftest.hcl` checks that the startup script installs `claude-runner-sweep` and gives each runner `--hooks-dir`. It also checks that the SessionStart hook matches only `startup|resume`.
- Claude Code ran `make validate` and `make test-templates` on macOS. All 22 tests passed.
- Claude Code ran `claude-runner-sweep` in an `ubuntu:24.04` container against a fake runner directory. The directory held a dirty runner clone, a dirty worktree, and a worktree with a merge conflict. It also held an extra clone with an unpushed commit, a pushed extra clone, a broken worktree, and a repository under `_sessions/`. Each one was stashed, removed, deleted or moved as described above. The `_sessions/` repository was left alone, and a second run changed nothing. A separate run checked that expired `runner-rescue` stashes are dropped and other stashes are kept.
- Nobody has run this in a live workspace with real runner sessions. The `post-session` hook's file name and environment come from the `claude self-hosted-runner` binary, Claude Code 2.1.319.
- Nobody has checked that the SessionStart hook's `session_id` equals the runner's `CLAUDE_RUNNER_SESSION_UUID`. If they differ, a resumed session is not told where its saved changes went, and nothing else changes.

## Release/Deployment Notes

- Existing runner workspaces get the sweep on their next start after `coder update`. The first sweep cleans everything earlier sessions left. Unpushed clones are moved to `~/.local/state/claude-runner-rescue/`, so look there for anything missing.
- An unfinished merge, rebase, cherry-pick or revert in a checkout is aborted before stashing. Its commits stay on their branch.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
## Short Summary (TL;DR)

With the runner setting on, the workspace now creates `~/.claude-runner` with a README. The README says where to save the secret and where the runner keeps its logs, checkouts and rescued work.

## The Issue

Turning the runner on did nothing visible until the user created `~/.claude-runner/environment-secret` by hand. The directory didn't exist, and nothing in the workspace said what belonged in it.

## How This PR Solves The Issue

- When the runner is on, the startup script creates `~/.claude-runner` with mode 700 before its other checks. It writes `README.md` there on every start.
- The README covers saving the secret, the `hooks/` directory, the runner logs, `claude-sessions`, and the checkouts. It also says where `runner-rescue` stashes and rescued clones go, and how to stop the runner.
- The "runner NOT started" startup message points to the README.
- The setup steps in `docs/content/workspaces/ai-tools.md` no longer need `mkdir`.

## Manual Testing Instructions

```bash
make push-template-freeform
coder update <workspace>
ssh <workspace>.coder ls -ld .claude-runner
# Expect drwx------ even before a secret is saved
ssh <workspace>.coder cat .claude-runner/README.md
```

Without a secret, expect the startup log to say "NOT started" and name `~/.claude-runner/README.md`.

## Automated Testing Overview

- `freeform/tests/validate.tftest.hcl` checks that the startup script writes the README, and that the README names `~/.claude-runner/environment-secret`.
- Claude Code ran `make validate` and `make test-templates` on macOS. All tests passed. Claude Code also ran markdownlint on the README and the changed docs page.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rfay rfay changed the title fix(runner): clean runner checkouts after every session fix(runner): clean claude runner checkouts after every session Nov 11, 2026
@github-actions

Copy link
Copy Markdown

Previews

Site Preview
docs.coder.ddev.com (docs/content, built by Read the Docs) https://coder-ddev--244.org.readthedocs.build/
start.coder.ddev.com (docs/*.html, deployed to gh-pages) https://start.coder.ddev.com/pr-preview/pr-244/

The landing-page preview creates workspaces on staging-coder.ddev.com, and its Documentation links open the docs preview. Read the Docs posts its build status and changed pages in its own comment; the docs preview may take a minute or two after a push to update.

Updated for b36a403

@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 coder-ddev | 🛠️ Build #35818769 | 📁 Comparing b36a403 against latest (44a1dd2)

  🔍 Preview build  

1 file changed
± workspaces/ai-tools/index.html

@rfay
rfay merged commit d0b7220 into main Nov 11, 2026
21 checks passed
@rfay
rfay deleted the 20261111_rfay_runner_checkout_cleanup branch November 11, 2026 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant