Merge pull request blooop#577 from blooop/fix/575-reclaim-the-launch-locks

fix: a per-workspace launch lock was never reclaimed by anything but --purge

Author: blooop

CHANGELOG.md

@@ -69,6 +69,61 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   now say what it does -- recreate the container and its volumes -- and
   `docs/cli.md` and `docs/workspaces.md` gained the sentence that `rm` is the only
   verb which refreshes git state.
+
+### Fixed
+
+- **`dl --prune` reclaims the per-workspace launch locks**, which nothing short
+  of `--purge` removing the whole cache directory had ever reached. Two `dl` runs
+  launching one workspace serialize on an empty file under
+  `<cache>/devlaunch/launch-locks/<workspace-id>.lock`; every workspace ever
+  launched left one and no removal path took it away. Measured on one host: 18 of
+  26 entries named no workspace `devpod list` still returns, against 8 live
+  workspaces, the oldest three weeks old.
+
+  Zero bytes of disk, and still the thing worth fixing. A directory that only
+  ever grows is one nobody can read as a description of anything, and "nothing
+  devlaunch causes to exist accumulates unreclaimed" is the rule the rest of the
+  cleanup contract is written to.
+
+  The precondition is the volumes' precondition, asked of the same listing: no
+  workspace `devpod list` returns carries that id, re-asked under the acting
+  pass's own second listing so the approved set can shrink between the plan and
+  the act and can never grow.
+
+- **Unlinking a lock file is safe now, which it was not before.** Removing an
+  `flock`'d file is the self-defeating move: a process holding the old inode
+  excludes nobody, while new arrivals lock a fresh file at the same path and walk
+  past it, and two `devpod up`s of one workspace run against each other. The rule
+  used to be that no lock file is ever deleted, which is what made the leak
+  above permanent.
+
+  What replaced the rule is one question asked after every `flock`, on both
+  sides: is the path still naming the inode I just locked? An acquisition that
+  has lost its file queues again against the live one rather than believing
+  itself alone, and a sweep that has lost its file re-opens rather than unlinking
+  whatever the path names by then. Both halves are needed. A sweep that only took
+  the lock would still have unlinked a live launch's file, because an inode
+  somebody else already unlinked has no holders and locking one always succeeds.
+
+  The unlink is then performed while holding the lock, and never queues for it: a
+  lock a launch is holding is left standing, reported as held, and reclaimed by
+  the next prune. That is not counted as a failure and does not change the exit
+  code, because it is the guard working.
+
+  **`--purge` is not the only other unlinker, it is the other unlinker that holds
+  no lock.** It removes the cache directory entire, launch locks with it, which is
+  what makes "somebody replaced this path" an ordinary event here rather than a
+  devlaunch bug, and why the question above is asked every time rather than as a
+  belt over a brace.
+
+- **devpod's own per-workspace lock is left alone, deliberately**, and the
+  cleanup contract now says so in a table row rather than by omission. The same
+  measurement found 70 of 78 stale entries under
+  `~/.devpod/contexts/default/locks`. They stay: it is another program's lock,
+  taken by processes devlaunch does not run, and the revalidation above is a
+  promise devlaunch can only make about a file only devlaunch opens. It is the
+  same file `dl <ws> kill` has always been careful never to unlink.
+
 ## [0.31.0] - 2026-09-04
 
 ### Added

README.md

@@ -253,7 +253,7 @@ instead. [docs/cli.md](docs/cli.md) has the full `--rm` contract, including whic
 | `dl --ls` | List every workspace |
 | `dl --ls --json` | The same, machine-readable, with what each workspace would lose if deleted |
 | `dl --ls --size` | Add what deleting each one would free. Opt-in: it walks every file |
-| `dl --prune` | Remove the clone directories no workspace opens any more, the agent git worktrees inside the clones it keeps, and the volumes of workspaces devpod has forgotten |
+| `dl --prune` | Remove the clone directories no workspace opens any more, the agent git worktrees inside the clones it keeps, and the volumes and launch locks of workspaces devpod has forgotten |
 | `dl --reconcile` | Re-point workspaces whose recorded source folder went missing. Deletes nothing |
 | `dl --purge` | Remove devlaunch's own workspaces and caches |
 | `dl --install` | Install shell completions, and the `dl-herdr-shell` name a herdr pane opens through |
@@ -405,7 +405,7 @@ different jobs:
 
 | Command | Takes | Leaves |
 |---|---|---|
-| `dl --prune` | Clone directories no workspace opens, collectable agent git worktrees inside the ones it keeps, and the volumes of workspaces devpod no longer lists | Every workspace, container and image |
+| `dl --prune` | Clone directories no workspace opens, collectable agent git worktrees inside the ones it keeps, and the volumes and launch locks of workspaces devpod no longer lists | Every workspace, container and image |
 | `dl --purge` | The workspaces devlaunch created, and its caches | Workspaces it did not create, each named with its source before it asks |
 | `dl --reconcile` | Nothing | Repairs records that stopped matching the disk |
 

docs/cleanup.md

@@ -304,8 +304,9 @@ until now nothing removed them: measured on one host, **52 clone directories for
 all-or-nothing: the only way to get the 4 GB back was to destroy the 7.86 GB
 too, and every bare cache with it.
 
-`dl --prune` removes exactly the clone directories no live workspace opens, and
-reclaims the Docker volumes of workspaces devpod no longer lists. It never
+`dl --prune` removes exactly the clone directories no live workspace opens,
+reclaims the Docker volumes of workspaces devpod no longer lists, and reclaims
+the launch locks of workspaces devpod no longer lists. It never
 deletes a devpod workspace, a container or an image, never
 touches a repo's `.bare` cache (0.08 GB for seven repos, and it is what makes
 the next clone of a repo fast), and never looks outside
@@ -637,6 +638,57 @@ walk of a 12000-file environment, and the listing is a read-only command people
 run casually; a derivative is never a reason a site stands, so the listing loses
 no answer by not asking.
 
+#### The per-workspace launch locks
+
+Two `dl` runs launching one workspace serialize on a lock file under
+`<cache>/devlaunch/launch-locks/<workspace-id>.lock`. It is empty: the only thing
+it carries is the `flock` the kernel holds on it while a launch is inside its
+critical section, and the kernel drops that when the process ends, however it
+ends.
+
+Until [#575](https://github.com/blooop/devlaunch/issues/575) nothing removed one.
+Every workspace ever launched left an entry and no removal path took it away, so
+the directory only ever grew: measured on one host, **18 of 26 entries named no
+workspace `devpod list` still returns**, against 8 live workspaces, the oldest
+three weeks old. `dl <ws> rm` did not reach them, `--prune` did not, and
+`--purge` only did because it removes the whole cache directory. That is zero
+bytes of disk and it is still the thing worth fixing: a directory that only grows
+is one nobody can read as a description of anything.
+
+`--prune` reclaims them now, on the same plan and the same `y/N` as everything
+above, and the precondition is the volumes' precondition: no workspace `devpod
+list` returns carries that id.
+
+| artifact | who reclaims it | what makes that safe |
+| --- | --- | --- |
+| `<cache>/devlaunch/launch-locks/<id>.lock` | `--prune`, and `--purge` with the rest of the cache | no workspace devpod lists carries that id, re-asked under the acting pass's own listing, **and** nothing holds the lock at the moment of the unlink |
+| `~/.devpod/contexts/<ctx>/locks/<id>.workspace.lock` | nobody, deliberately | it is devpod's lock, taken by processes devlaunch does not run. See below |
+
+**Why unlinking a lock file needed a change to how they are taken.** Removing an
+`flock`'d file is the classic self-defeating move: a process that opened the old
+inode still holds a lock nobody else can see, while new arrivals lock a fresh
+file at the same path and walk straight past it, and two `devpod up`s of one
+workspace then run against each other. That hazard is the reason the rule used to
+be that no lock file is ever deleted. What replaced the rule is a check at the
+other end: an acquisition now hands back a guard only for the inode the path
+*still names*, so a run whose file went out from under it queues again against
+the live one instead of believing itself alone. The unlink itself is performed
+while holding the lock, and never blocks for it, so a launch inside its critical
+section keeps its file and the sweep says so rather than waiting.
+
+A lock a launch is holding is left standing and is not counted as a failure. It
+is the guard working, the run still ends 0, and the next `--prune` finds it free.
+
+**devpod's own lock is not devlaunch's to reclaim.** The same measurement found
+70 of 78 stale entries under `~/.devpod/contexts/default/locks`, and they stay.
+It is the same file `dl <ws> kill` is careful never to unlink, for the same
+reason: the revalidation above is a promise devlaunch makes to itself about a
+file only devlaunch opens, and devpod makes no such promise about this one. A
+`dl` that removed it would be deciding that another program's mutual exclusion is
+devlaunch's to break. The file devpod does leave stale, and which `kill` does
+remove, is the busy marker beside it, which is a plain file and not a lock at
+all.
+
 #### The disk neither command frees
 
 Both commands end on the same line, in the same words: