dl --prune, dl --purge and dl --reconcile in full, plus the disk
accounting behind dl --ls --size and the JSON a cleanup tool reads.
The three global commands that touch what is already on disk, in full. --prune takes clone
directories and no workspaces, --purge takes both but only what devlaunch made, and --reconcile
removes nothing; it repairs records that stopped matching the disk. Each one prints its plan and
asks before acting, and -y is what skips the question.
One exception, and it is not a released build: a binary compiled from somebody's
working tree appends -dev (dl <version>-dev). That comes from the dev-build
cargo feature, which ./dev.sh builds with and nothing that ships enables, and it
is what tells dl-next apart from dl when both are on PATH. See "Two installs"
in AGENTS.md.
Removing a workspace removes three things: the devpod workspace, the local clone
(unless it holds work that exists nowhere else, see cleaning up
workspaces), and the named Docker volumes that
workspace's devcontainer created. Every path that removes a workspace does all three:
dl <ws> rm, dl <ws> rme, dl <ws> --rm, and --purge.
Two volumes per workspace, both named from what devpod recorded substituting into the devcontainer:
| volume | declared by |
|---|---|
<workspace-folder-basename>-pixi |
a mounts entry in the devcontainer, for the .pixi cache. This repo's own devcontainer has one |
dind-var-lib-docker-<devcontainerId> |
the docker-in-docker feature, for the nested daemon's /var/lib/docker |
They used to be left behind, and that was measured rather than assumed: on one
development machine, 39 orphaned volumes holding 37.28 GB, not one of them
with a surviving workspace in devpod list (devlaunch#324). devpod delete
removes the container and never a volume, and Docker never garbage-collects a
named volume, so nothing in the picture was ever going to reclaim them.
Three things about how the removal behaves, all of them so that a delete cannot be made worse by it:
- The names are read from devpod's own record, never guessed from a pattern.
devpod writes down what it substituted, and that is the only thing consulted. A
workspace devpod never finished creating has no such record, so nothing is named
and no
dockercommand runs at all, rather than one carrying a made-up name, which would be somebody else's disk. - It is best-effort, and cannot fail a delete. The workspace is gone either
way; reporting failure would send you looking for a workspace that is not there.
A volume Docker will not release, one another container still holds, say, is a
line on stderr and nothing more. A machine with no
dockerat all says nothing, because a machine with no Docker never made these volumes. - Images are still yours. See the disk neither command frees. That boundary is about images now, and deliberately stays there.
The read above happens at delete time, immediately before devpod delete takes
devpod's record away with the workspace. That closes the leak for every delete
that goes through dl. It does nothing for a workspace deleted some other way: a
bare devpod delete, or a devpod home that was cleared out. devpod removes its
own record and leaves the volumes, and after that there is nothing on the machine
that names them.
So dl keeps its own copy. At the end of every up that completed, it reads the
same two fields out of the same workspace_result.json and writes them to a small
per workspace file under its cache, beside the tool verdict markers:
~/.cache/devlaunch/workspace-copies/<workspace>.json. dl --prune reclaims from
those copies, and the whole of what it asks per copy is: does any workspace
devpod list returns carry that id? Where none does, the workspace is gone and
its volumes are leftovers.
Nothing is invented at either read. Every name dl hands docker volume rm still
came out of a substitution devpod performed and wrote down, which is the rule
above and is unchanged. The alternative, matching <basename>-pixi and
dind-var-lib-docker-<id> against docker volume ls, stays refused: the
docker-in-docker feature writes that second name in every devcontainer tool that
runs it, so the candidates dl cannot attribute are exactly the ones that belong
to somebody else, and a volume is not an image. The wrong answer there is data
loss, not a rebuild.
A copy can still be wrong, and it can be wrong in exactly two ways. It can name a
volume that is already gone, which docker volume rm --force treats as a success
and says nothing about. It can name a volume something else now holds, which
Docker refuses: volume is in use, reported, nothing removed, and the copy kept so
the retry is still there. Neither is caught by trusting the file. A copy is dropped
once, when a removal came back removed for a workspace devpod does not list, which
is the one moment it is provably pointless.
Two consequences worth knowing. A run pointed at a scratch cache
(XDG_CACHE_HOME=...) finds no copies at all, so it names no volume and removes
none, which is what makes the scratch convention safe here by construction. And the
39 orphaned volumes measured on the reference host are out of scope: their
records died before any of this existed, no route reaches them that is not the
pattern above, and they stay. See the disk neither command
frees.
dl <ws> rm names the workspace going in and again once it has gone, both on
stderr, with whatever happened to the clone in between:
Removing workspace devlaunch-main-3j1t...
Removed workspace clone: ~/.cache/devlaunch/repos/blooop/devlaunch/devlaunch-main-3j1t
Removed local clone for devlaunch-main-3j1t
Removed workspace devlaunch-main-3j1t.
The first line comes after the unsaved-work guard has had its say and before devpod is asked, so a teardown that takes a while has a name attached to it and a refusal is never announced as a removal. The last line closes the block, which is what makes a batch legible: one block per workspace, each ending in the name of the workspace it was.
rme adds one line after the last of them, saying what became of the shell:
the pid it hung up, or why it hung up nothing. It is the last thing written to a
terminal that is about to close, and the only thing that explains a run where
nothing closed. See rme.
With --force that last line reads Workspace <id> is gone. instead, and the
difference is what the exit code proved rather than a change of tone. Without the
flag, devpod fails on a workspace it does not have, so a delete that succeeded is one
devpod had and let go. --force passes devpod's own --ignore-not-found, which is
the flag asking for absence rather than for a removal, and it makes "there was
nothing there" succeed with nothing in the answer to tell it from a real delete. A
path is resolved without asking devpod anything at all, so dl ./wrong-directory rm --force gets as far as the delete and comes back successful. Saying Removed there
would be dl affirming a delete that never happened.
The two middle lines belong to the clone, and a workspace with no clone recorded
under it prints neither of them. That is the case the outer two exist for: before
them, the only thing on any stream naming what had been deleted was devpod delete's own line on stdout, which is devpod's wording rather than dl's.
It matters most for dl rm with no workspace named. The picker draws its rows as
<owner> | <repo> | <branch> and then takes its screen away, so a pick names the
row it took beside the id it resolved to, before the first workspace is touched:
Picked blooop | devlaunch | main -> devlaunch-main-3j1t
Both halves, because neither stands in for the other. An id is
<repo-slug>-<ref-slug>-<suffix> and carries no owner, so a fork and its upstream are
one id apart only in the hashed suffix, which the picker deliberately never draws:
reading it is no part of choosing a workspace. The row is what was on screen while the
choice was being made, and the id is what devpod is addressed by and what every line
below this names. A batch takes a heading and one line each:
Picked 3 workspaces for rm:
blooop | devlaunch | main -> devlaunch-main-3j1t
myfork | devlaunch | main -> devlaunch-main-7q0x
- | someones-project -> someones-project
devpod's workspace list is shared. A workspace you made with devpod up, or that
another tool made, sits in the same list as the ones dl made, and dl --purge
has no business destroying it. So it deletes only the workspaces devlaunch
created, meaning the clones it made under its own cache directory ($XDG_CACHE_HOME or
~/.cache, then devlaunch/repos/<owner>/<repo>/<id>), which is exactly the
directory the purge is about to remove anyway. Everything else keeps working
afterwards, because nothing a purge touches backs it.
Anything it is leaving is named before it asks, by where it came from and not only by its id:
$ dl --purge
This will remove all devlaunch data:
- 4 DevPod workspace(s)
- /home/you/.cache/devlaunch/ (workspace clones, repo caches, the shared pixi cache, completions)
Leaving 2 workspace(s) devlaunch did not create:
- pythontemplate: https://github.com/blooop/pythontemplate
- my-hand-made-workspace: /home/you/projects/thing
Removing the cache also drops what dl recorded about them, the copy of their volume names included. They keep working, and `dl <workspace> rm` still removes one and its volumes while devpod still lists it.
A clone an older dl placed outside the cache is named only by a record in there, though, so remove such a workspace now if the clone should go with it.
Are you sure? [y/N]
The source is the half you can decide on. An id is what devpod addresses a
workspace by and says nothing about where it came from, so pythontemplate reads
the same whether it is a dl <git-url> of yours, a dl ./project, or something
somebody else's tool made. It is the same string dl --ls shows in its SOURCE
column, read the same way, so the two never describe one workspace differently.
A source dl cannot read at all, which is devpod's own object rather than a path
or a URL, is printed as that object and said to be one.
Three things dl does create can land in that second list rather than the first.
dl ./some/path and dl <git-url> open a source dl did not clone, so it
cannot tell them from a workspace you made by hand. The third is older: a
workspace an earlier dl placed outside the cache, under the worktree.repos_dir
key that version still read. This one no longer does, so such a clone reads as
someone else's here and --prune does not reach it either. Delete any of the
three with dl <workspace> rm.
Erring this way is deliberate: a purge that skips one of your own workspaces
costs you a command, and the other kind of mistake costs you work you cannot get
back.
That third one is what the two sentences under the list are about. The workspace
stays, and the record naming its clone was in the cache that has just gone, so a
dl <workspace> rm afterwards deletes the workspace and leaves the directory
standing with nothing on the machine pointing at it. Removing it before the purge
is what takes the clone too. If config.toml still sets worktree.repos_dir,
--purge now says so as well, before the plan: where no workspace opens such a
clone any more there is no line in any list for it, and that notice is the only
mention that tree will get.
Under -y the second sentence is the same fact in the tense that run has earned.
"Remove such a workspace now" is an action only somebody who still has the
question in front of them can take, and printing it into a run that deletes the
records three lines later would be advice arriving after the door shut, so what
-y says instead is what will be true of dl <workspace> rm from then on.
The volume names are part of that loss, and this is the one place it shows.
Deleting a survivor with dl <workspace> rm still takes its volumes: that read
happens at delete time, out of devpod's own record under DEVPOD_HOME, which a
purge does not touch. What goes is the copy dl
keeps, which is what
--prune reclaims from once devpod has forgotten a workspace. So a survivor
deleted with a bare devpod delete after a purge leaves both its volumes with
nothing on the machine naming them, where before the purge dl --prune would
have reclaimed them. Deleting such a workspace through dl is what avoids that,
and it is the same advice the sentence above gives for its clone.
A container writes into its clone as its own user, vscode at uid 1000 in the
standard devcontainer base image. Where your host user is uid 1000 too, nothing
here comes up. Where it is not, on CI, a shared machine, a container running as
root, or devlaunch developed inside its own devcontainer, the directories the
container made cannot be emptied by you, and the purge cannot remove them.
It removes everything else anyway, and names what is left:
$ dl --purge -y
Removed what was permitted under /home/you/.cache/devlaunch. These refused:
- /home/you/.cache/devlaunch/repos/blooop/bencher/bencher-main-ii41: Permission denied
Usually this means a container wrote them as a different user, and:
sudo rm -rf '/home/you/.cache/devlaunch'
clears them. Check the reasons above first -- it does not fix all of them.
devlaunch does not manage Docker images: the images these workspaces built may still hold disk, and `docker system df` shows what Docker is holding.
That last line ends every purge, including one that found nothing to purge and
one you answered n to. dl --prune ends on the same one, in the same words.
See the disk neither command frees.
Exit status is 1, because a clone you were told would go is still on disk. It
used to be 1 with the whole cache still standing: the first refusal stopped
the purge, so the completion caches, metadata.json and every other clone
survived on account of one directory.
When none of it goes, meaning nothing under the cache came away at all, which is what a symlinked cache root gives you, or one that cannot even be looked at, or one whose every entry refused, the headline says that instead of claiming a partial success:
$ dl --purge -y
Removed nothing under /home/you/.cache/devlaunch. These refused:
- /home/you/.cache/devlaunch: Permission denied
The report underneath is the same one, and so is the exit status: 0 means the
cache is gone and nothing else does, which is the only distinction a script can
act on. Removed everything, removed what it was permitted to and removed nothing
are three outcomes rather than two, and the sentence is where the third one
lives, because it is the one that decides whether you still have clones to go
and look for.
What is listed is the directory, once, not the hundreds of files inside it. Unlinking needs write permission on the directory rather than on the file, so every entry in that clone refuses separately and they are all the same fact. Two separately unwritable directories on one path are two lines, though, because clearing the inner one would leave the outer one just as stuck.
Each line carries what the system actually said. A container running as another
user is the common cause, but a read-only mount, chattr +i and a busy
mountpoint all land here too, and sudo rm -rf does not fix those, which is
why the report offers the cause rather than asserting it.
If you have moved your cache by making ~/.cache/devlaunch a symlink, a
purge refuses it and names the target rather than following it. Remove the real
directory yourself if you meant to: following the link would empty a directory
you never named, and removing just the link would report a clean sweep while
your clones sat on the other volume.
A workspace per branch means clone directories accumulate under the cache, and
until now nothing removed them: measured on one host, 52 clone directories for
17 live devpod workspaces, 37 of them attached to nothing, 4.00 GB, against
7.86 GB still in use. --purge is the wrong tool for that, being
all-or-nothing: the only way to get the 4 GB back was to destroy the 7.86 GB
too, and every bare cache with it.
dl --prune removes exactly the clone directories no live workspace opens,
reclaims the Docker volumes of workspaces devpod no longer lists, and reclaims
the launch locks of workspaces devpod no longer lists. It never
deletes a devpod workspace, a container or an image, never
touches a repo's .bare cache (0.08 GB for seven repos, and it is what makes
the next clone of a repo fast), and never looks outside
<cache>/devlaunch/repos. Every directory it finds is one of three things:
- a live workspace opens it. Kept, and named with the workspace that has it. "Opens" means at or under: a workspace opened on a subdirectory of a clone still needs the clone.
- nothing opens it. Removed, unless it holds work that exists nowhere else,
or
gitwould not say what it holds. A clone a container wrote as another user is unreadable rather than empty, and "cannot tell" is kept, not removed. dl's records and devpod's disagree about it. Kept, always. This is #88's shape. On that ticket's host, 36 devpod workspaces out of 39 recorded a source folder that was gone or was a config-only stub, while the real checkout sat beside it under a newer naming scheme, so a perfectly healthy clone was opened by nobody, and the stub was the only thing anything pointed at.--prunewill not guess which clone such a workspace needs: it keeps every clone of that repository and names the record to go and fix.--forcedoes not move any of them.dl --reconcileis what fixes them.
Note that every directory two levels under <cache>/devlaunch/repos is a
candidate, so a stray directory somebody left there is looked at like any other.
The cache is dl's to manage; things that are not clones do not belong in it.
But git cannot say what a directory that is not a repository holds, and
"cannot say" is kept rather than removed, so clearing junk out of the cache
takes --force. That is the same refusal a clone with a half-written .git
gets, and deliberately so: telling the two apart would mean --prune forming
its own opinion about a directory dl <workspace> rm already refuses on.
$ dl --prune
Clone directories under /home/you/.cache/devlaunch/repos:
Removing 2 that nothing references -- 1.4 GiB:
- /home/you/.cache/devlaunch/repos/blooop/bencher/bencher-test1-mxvm (1.1 GiB)
- /home/you/.cache/devlaunch/repos/blooop/devlaunch/devlaunch-t1-d7bw (317.0 MiB)
Leaving 3:
- /home/you/.cache/devlaunch/repos/blooop/devlaunch/devlaunch-main-3j1t: workspace devlaunch-main-3j1t still opens it
- /home/you/.cache/devlaunch/repos/blooop/wayfinder/wayfinder-devlaunch-6pl5: holds 2 unpushed commit(s) -- add --force to remove it anyway
- /home/you/.cache/devlaunch/repos/blooop/rockerc/rockerc-main-brdr: devpod lists workspace rockerc-main-brdr and sources it at /home/you/.cache/devlaunch/repos/blooop/rockerc/main; see devlaunch#88
Dropping 12 record(s) of directories already gone.
Are you sure? [y/N]
-y skips the question. A clone holding uncommitted or unpushed work is kept
and named, in the same words dl <workspace> rm
refuses in. 13 of those 37 stale clones did, two of them with real unpushed
commits, so this is load-bearing rather than a formality. --force promotes
that one case and nothing else. Erring this way costs you a flag; erring the
other way costs work that cannot be recovered.
The sizes are the same exclusive bytes dl --ls --size reports, and they mean
the same thing: what removing that directory
would actually free, not what du would print. Where a walk could not read
something the figure reads ≥ and so does the total, because a floor printed as
a total is a cleanup tool telling you a directory is small when it is not.
Directories that will not come away are named the same way a purge names
them, the rest still go, and the exit
status is 1.
Nothing here runs on its own. A full scan measured 1017 ms on that host,
about two warm launches, and it gets slower exactly as the cache gets fuller,
so it is never on a launch path and never folded into dl --ls. Answering n
is the read-only view; there is no separate flag for it. It costs one
devpod list to build the plan and no devpod status at all, because whether a
workspace is running has no bearing on whether a directory is opened by one. A
run you say yes to pays a second devpod list before it removes anything, and
classifies every directory again: a launch that finishes while the report is on
screen registers a workspace for one of the directories in the plan, and that is
the one thing the plan cannot be re-checked against from disk. The set you
approved can shrink between the report and the act. It can never grow.
It also drops the metadata.json records of directories that are already gone.
That file was append-only in practice, 49 records for 17 live workspaces on the
same host, and this is the first thing that prunes it.
An agent harness working inside a workspace makes its own git worktrees under
<clone>/.claude/worktrees/<name>/, one per task, and nothing ever collected
them. Measured on one host: 72 of them, 104.5 GB, 18 carrying a whole
.pixi/envs/default, about 82% of everything under repos/. One clone held
55 GB on its own. Every one of them was inside a clone belonging to a live
workspace, so the rule above not only missed them, it must never fire on them:
firing would delete a live workspace's checkout. So this is a second rule, and
it runs only on the clones the first one is keeping. A clone that is going
already accounts for everything inside it.
The word "worktree" is git's here, not dl's. These are real registered
worktrees, made from inside the container, so the path git holds for one is
/workspaces/<id>/.claude/worktrees/<name>, which does not resolve on the host
at all. That non-resolution is what makes the metadata operation work rather
than what stops it: git worktree remove <the path git printed> drops exactly
that one registration when the path does not resolve, and refuses when it
resolves to something unrelated, which --force cannot argue it out of.
A site is a place inside the clone of the shape
.claude/worktrees/<leaf>, nested as deep as the harness nested it. A site is
collectable only when it and every site nested inside it are, decided
bottom-up. That is the whole of what makes a nested worktree safe, and it is
structural rather than a check: the removal is a subtree removal, so a unit
narrower than the operation would leave the difference unprotected, which is how
a nested worktree holding an afternoon of work, an unpushed commit, or a lock
used to be deleted with its parent with no flag typed.
Containment comes from the filesystem walk, never from comparing the paths git recorded. On a host every recorded path is a string about another machine, and a worktree of a different repository has no entry in this clone's listing at all. Recorded paths are used for exactly two things, neither of which resolves them: matching a registration to a place, and naming the registration to forget.
There is no git worktree prune here, and its absence is the design rather than
an omission. That command's domain is a directory read at the moment it runs,
so a registration created after the plan was printed is inside its blast radius
and no plan can name it. Three registration states it reaches never appear in any
listing either. An operation whose domain dl cannot enumerate is one it cannot
fail towards keeping with, so the sweep drops registrations one at a time, by
name, and every name came out of a listing it read. Those unlisted states are
reclaimed by git gc's own worktree prune --expire, on gc.worktreePruneExpire,
three months by default: a named third party rather than a straggler.
A site's verdict is collectable, with a proof, or standing, with at least one reason. A reason is either work that was found or a question that could not be put, and reasons accumulate up the subtree, so a site that is both dirty and locked reports both and a parent's line names the child that caused it. There is no third value and no way to reach the collectable arm by nothing having objected: the proof is a witness that only a probe which actually answered can mint, so "nothing objected" and "nothing was asked" are different answers.
Four questions, and their scopes are not the same. What is at the site at all is
per site. Whether the working tree holds anything that exists nowhere else is
per working tree, which is why the clone's own git status cannot answer for
what is nested in it: .claude/worktrees/ is ordinarily gitignored, and a nested
worktree has an index of its own. Whether the commits exist somewhere else is
per repository, and
it is asked of the sibling .bare cache first: a workspace clone is cut from the
bare and then repointed at the forge with no fetch of its own, so its
refs/remotes/origin/* is as of clone time and asking it alone reports
pushed-and-merged branches as unpushed. Whether a third party claims the site is
the lock, and a lock is an unproved, never a loss: git documents it as saying
nothing about whether anybody is working in there, so reporting it as work would
be inventing work that may not exist.
One limit, stated because it is a limit and not an oversight: gitignored
content is not weighed. git worktree remove deletes a worktree whose only
content is gitignored, exit 0 and silent, and so does the removal here. It is
left that way because a clone's own ignored bytes have never been weighed either
dl <workspace> rmand the orphan rule above bothrm -rfpast them - and one conjunction wants one definition of what makes a tree dirty rather than two that disagree about the same bytes.
Weighing it at the site alone was tried and taken back out, and the cost is worth
recording. An installed .pixi/envs/default is ignored content; it is 18 of the
72 directories on the reference host and the difference between 104 GB and about
10. Weighing it put every one of them behind --force-worktrees, which is also
the flag that carries past a lock and past another repository's worktree, so
getting the disk back would have meant typing the flag that switches off every
protection described here. Whether ignored bytes should be weighed is a real
question and it is one question for both scopes, not a special case for this one.
Stashes need no probe. A git stash push from inside a linked worktree writes
the clone's own refs/stash, survives the directory, and is reached by
rev-list --all; nothing here removes the shared ref store.
--force-worktrees is the one flag that carries a site past any of this, and it
is deliberately not --force: --force is a word people already type at
--prune, and widening it would turn it into permission to remove a worktree
somebody may be working in.
A directory in the worktrees place whose registration is not in this clone's
listing is not dl's to remove. That covers a worktree of a different
repository, a plain directory, an unreadable gitfile and a symbolic link. Each
stands, is named, and pins everything above it, and none of them is ever probed.
Ownership is the registration join and not the .git gitfile: a gitfile tail
says a directory is a worktree of some repository, and reading it as this one's
is how a live worktree of another repository, holding uncommitted work, was once
offered for removal unopposed under the printed reason "git has already forgotten
it". That was false: the repository that registered it had forgotten nothing. git contributes nothing to that case, so nothing here leans on it: its
one unforceable refusal fires on a recorded path handed to git worktree remove,
an invocation dl never makes for a foreign worktree, and it says nothing at all
about a directory removal.
dl will not reclaim the worktree itself, and says so with the owning
repository named. --force-worktrees is what removes one, and the honest thing
to do first is usually to take it back from the repository that owns it.
A regenerable subtree inside one is a separate question and gets a separate line. Whose repository an installed environment belongs to was never part of the argument for reclaiming it: the tag and the lockfile beside it say the same thing either way, and being another repository's is an answer about git's account of the content rather than a claim by somebody over the directory. So the worktree stands and is named, and a tagged environment inside it can still be reclaimed on the same plan. The two lines are about two different units, which is why neither sentence claims the other's scope.
Nothing here detects containers and no arm exists to protect one. Two properties
do that instead. --prune's domain is enumerated from the cache directory alone,
and dl never mounts a cache clone into a container at a path inside that
container's own cache, so a container's own clone is never in the domain. And the
directory goes before the forget, with nothing forgotten on a partial removal, so
the recorded path does not resolve at the moment the forget runs even where it
resolved a moment earlier.
$ dl --prune
Clone directories under /home/you/.cache/devlaunch/repos:
Leaving 1:
- /home/you/.cache/devlaunch/repos/blooop/devlaunch/devlaunch-main-zovo: workspace devlaunch-main-zovo still opens it
Agent git worktrees inside the clones above -- 6.0 GiB in worktrees that go, and 2.3 GiB in regenerable subtrees inside the ones that stay:
/home/you/.cache/devlaunch/repos/blooop/devlaunch/devlaunch-main-zovo:
- removing .../.claude/worktrees/agent-a49a (5.8 GiB), and dropping its 1 registration(s)
- removing .../.claude/worktrees/agent-a8da (204.0 MiB), and dropping its 2 registration(s)
- leaving .../.claude/worktrees/agent-b120: git is holding it locked (claude session) -- add --force-worktrees to remove it anyway
- leaving .../.claude/worktrees/agent-c771: holds 3 uncommitted change(s) -- add --force-worktrees to remove it anyway
- reclaiming .../.claude/worktrees/agent-c771/.pixi/envs/default (2.3 GiB): a pixi environment, re-derived by `pixi install --frozen -e default` from .claude/worktrees/agent-c771/pixi.lock
Whether a worktree's commits are anywhere else is as of the last fetch into the repository cache; --prune does not fetch.
A regenerable subtree is one whose creator wrote a CACHEDIR.TAG into it and whose lockfile and manifest are still beside it; putting one back is one command and no network beyond the shared package cache.
Are you sure? [y/N]
The plan states two byte figures and they are two different claims: what removing
the clone directories would free, and what the worktrees inside the clones it is
keeping would free. Folding the second into the first made the headline number
describe directories that are not going, and then said the same bytes twice. The
bytes are also attributed in dl --ls --size, as a part of the clone's figure and
never an addition, because the worktrees are inside it. They were invisible there
on the host above, which is how it reached 100%.
For an orphan clone that has agent worktrees in it, reclaiming takes two runs. The clone's own verdict conjoins every site inside it, correctly, because removing the clone destroys whatever they hold. So run one keeps the clone and sweeps the worktrees; run two finds the clone empty of them and reclaims it with no flag.
The 18 duplicated .pixi/envs/default copies are the reason the figure is 104 GB
rather than about 10, and they cannot be pointed at the shared package cache: only
the pixi download cache is shared, because installed environments bake absolute
paths (see "The shared pixi package cache" in
workspace-tools.md). Removing the worktree is one way those
bytes come back: an env is gitignored content, and by the limit above it does not
by itself keep a finished worktree standing. The next section is the other way,
and it is the one that reaches the worktrees that have to stay.
| artifact | who reclaims it | what makes that safe |
|---|---|---|
a directory carrying a CACHEDIR.TAG inside an agent worktree that is staying |
--prune, on the same plan and the same y/N as everything above it |
the tag plus the lockfile: the program that made the directory declared it regenerable, and a lockfile still inside the worktree names the environment it re-derives |
Most of the 104.5 GB is in worktrees that have to stand. 18 of the 72 carried a
whole .pixi/envs/default, and a site that is dirty, unpushed or unprovable
keeps every byte of it. About 94.5 GB of the measured total is in there, which is
roughly 90%.
Reaching into a standing worktree to delete part of it sounds like exactly the
thing the rules above exist to stop, and the answer is what the standing verdict
is a statement about. Every reason a site stands, except a claimant's, is an
answer about git's account of that site's content: what git status says through
its admin directory, what a ref reaches, or the fact that neither could be
obtained. Nothing under .pixi/envs/ has ever been in that account, because pixi
writes a .pixi/.gitignore of * and !config.toml and puts it outside every
index, every status and every commit, in every clone, forever. The bytes the
verdict is uncertain about and the bytes under the tag are disjoint sets, and a
file the installer wrote is what separates them. The alternative was a report
that says "this holds work that exists nowhere else, 0 bytes" and refuses to give
back 5 GB in the same breath.
What names it is a declaration, and it never reads a directory's name. The
gate is the Cache Directory Tagging Specification:
a file called CACHEDIR.TAG whose first 43 bytes are the published signature,
written by the program that created the directory to say the contents are
regenerable. Measured: rattler, cargo, uv and pytest write one; python -m venv
writes none and npm writes none anywhere beneath node_modules. So a .venv is
taken or left depending on which program made it, which no rule keyed on the name
could express, and .pixi appears nowhere in the predicate. The walk stops at
the outermost tag rather than descending, so a target/ inside an environment is
never a second line for the same bytes.
A tag says regenerable; it does not say by what. So a tagged directory is
reclaimed only when a reader on this side answers with the thing that re-derives
it, and one reader is implemented. It reads pixi's own conda-meta/pixi for the
environment name, then walks up inside the worktree for a pixi.lock a pixi
project owns whose environments: map names it. Measured: a lock that
names the environment restores 5507 of 5507 files in 0.52 s with no network, and
does it with every proxy variable pointed at a dead port; a stale lock still
restores what was there, because the environment on disk came from that lock; a
lock that is absent restores nothing; and an environment the lock no longer
names is reproducible from nothing on disk. The last two stand, are named with
their bytes, and the second gets pixi clean -e <name> as the pointer. A tag no
reader recognises stands the same way.
The command the plan prints is the command every measurement was taken with,
pixi install --frozen -e <name>. Without --frozen, plain pixi install
compares the lock's hash to the manifest and re-solves, which is network, and the
stale-lock case above is exactly the one this reclaims. And a lockfile no pixi
project owns is not treated as a recipe at all, because the command would not
run. Two shapes reach that, and the question asked is pixi's own rather than a
file name: pixi install in a directory holding a lockfile alone exits with
could not find pixi.toml or pyproject.toml with tool.pixi, and beside a
pyproject.toml carrying no [tool.pixi] table it exits with found pyproject.toml without tool.pixi section. The second is the shape a repository
leaves behind when it migrates off pixi, which is the abandoned environment this
whole path goes looking for. A pointer somebody reads after the bytes are gone
has to be a command that works. Whether that table is there is the whole of
what is read out of the manifest, so a stale manifest changes nothing:
derivability is the lock's answer.
A claim reaches the subtree; an account of content does not. A
git worktree lock, or a repository lock dl could not take, is somebody
asserting a claim over the directory and drawing no line between its parts, and a
lock may mean running right now. Those pin the tagged subtree along with
everything else. Dirty, unpushed and unprovable do not, because they were never
statements about these bytes. A worktree of another repository is in the
second group and not the first: whose repository an environment belongs to was
never part of the argument.
No new flag, and it does not ride on --force-worktrees. This is a removal
with a proof rather than a force, and --force-worktrees also carries past a
lock and past another repository's worktree, so hanging this on it would make one
flag carry two consents. It rides --prune's own question: the plan names each
directory and its size before the y/N, and the acting pass reads the tag, the
record and the lockfile again under the lock before anything goes.
What goes is the tagged directory alone. Never .pixi, which carries no tag of
its own and holds config.toml, the one file .pixi/.gitignore un-ignores, and
that is structural rather than lucky: nothing in dl spells .pixi to achieve
it. A lockfile re-derives the one directory pixi install --frozen -e <name>
beside it would write, which is <the lockfile's own directory>/.pixi/envs/ <name>, so the reader claims a tagged directory only when that path and this one
are the same.
A tag planted at the top of a .pixi with an environment's record under it fails
that comparison, because .pixi is not .pixi/envs/default, and stands. No
directory name is matched anywhere: what is compared is where pixi would put the
environment the record names against where this directory actually is.
One thing the tag does not promise, said out loud. pixi does not defend its
own declaration: a file planted by hand inside an environment survives
pixi install --frozen unmentioned, so the tag is a claim about what the
directory is for and not a proof about what is in it now. The case rests on the
disjoint sets above, and anything somebody put inside an environment goes with
it.
That includes anything written into it after the plan was printed, and the
asymmetry with the worktrees above is deliberate. A worktree that gains a
nested site between the plan and the y is withheld whole, because the thing it
gained is a unit of its own that nobody consented to. A tagged environment that
gains a file gains nothing of the kind: by the paragraph above, the plan already
said out loud that everything under the tag was going. So the re-read asks again
what makes the removal legitimate, which is the tag, the record, the lockfile and
the claimant fold, and it does not ask whether the contents moved. Asking that
would mean a full walk of a 12000 file environment on both sides of one question,
to protect bytes the plan already named. The window is the one --prune holds
the repository lock across, so the only writer that can reach it is one that is
not a participant in that lock, which is a container.
dl --ls does not cost these. Weighing one is a full walk of a worktree plus a
walk of a 12000-file environment, and the listing is a read-only command people
run casually; a derivative is never a reason a site stands, so the listing loses
no answer by not asking.
Two dl runs launching one workspace serialize on a lock file under
<cache>/devlaunch/launch-locks/<workspace-id>.lock. It is empty: the only thing
it carries is the flock the kernel holds on it while a launch is inside its
critical section, and the kernel drops that when the process ends, however it
ends.
Until #575 nothing removed one.
Every workspace ever launched left an entry and no removal path took it away, so
the directory only ever grew: measured on one host, 18 of 26 entries named no
workspace devpod list still returns, against 8 live workspaces, the oldest
three weeks old. dl <ws> rm did not reach them, --prune did not, and
--purge only did because it removes the whole cache directory. That is zero
bytes of disk and it is still the thing worth fixing: a directory that only grows
is one nobody can read as a description of anything.
--prune reclaims them now, on the same plan and the same y/N as everything
above, and the precondition is the volumes' precondition: no workspace devpod list returns carries that id.
| artifact | who reclaims it | what makes that safe |
|---|---|---|
<cache>/devlaunch/launch-locks/<id>.lock |
--prune, and --purge with the rest of the cache |
no workspace devpod lists carries that id, re-asked under the acting pass's own listing, and nothing holds the lock at the moment of the unlink |
~/.devpod/contexts/<ctx>/locks/<id>.workspace.lock |
nobody, deliberately | it is devpod's lock, taken by processes devlaunch does not run. See below |
Why unlinking a lock file needed a change to how they are taken. Removing an
flock'd file is the classic self-defeating move: a process that opened the old
inode still holds a lock nobody else can see, while new arrivals lock a fresh
file at the same path and walk straight past it, and two devpod ups of one
workspace then run against each other. That hazard is the reason the rule used to
be that no lock file is ever deleted. What replaced the rule is a check at the
other end: an acquisition now hands back a guard only for the inode the path
still names, so a run whose file went out from under it queues again against
the live one instead of believing itself alone. The unlink itself is performed
while holding the lock, and never blocks for it, so a launch inside its critical
section keeps its file and the sweep says so rather than waiting.
A lock a launch is holding is left standing and is not counted as a failure. It
is the guard working, the run still ends 0, and the next --prune finds it free.
devpod's own lock is not devlaunch's to reclaim. The same measurement found
70 of 78 stale entries under ~/.devpod/contexts/default/locks, and they stay.
It is the same file dl <ws> kill is careful never to unlink, for the same
reason: the revalidation above is a promise devlaunch makes to itself about a
file only devlaunch opens, and devpod makes no such promise about this one. A
dl that removed it would be deciding that another program's mutual exclusion is
devlaunch's to break. The file devpod does leave stale, and which kill does
remove, is the busy marker beside it, which is a plain file and not a lock at
all.
Both commands end on the same line, in the same words:
$ dl --prune -y
...
Removed 2 clone director(ies) -- 1.4 GiB.
devlaunch does not manage Docker images: the images these workspaces built may still hold disk, and `docker system df` shows what Docker is holding.
The gigabytes a cleanup reports are usually not the ones you are looking for. On
the host this was measured, --prune had 4.00 GB of stale clones to give back
while docker system df read 86.5 GB of reclaimable images, 43.18 GB of
volumes and 13.88 GB of build cache. An order of magnitude more, sitting
behind a command that had just said "Removed". Saying nothing is what makes a
freed figure read as all of it, so both commands say this instead, whether they
removed 40 clones, found nothing to remove, or were answered n at the
confirmation. The report you get for saying n is a reason to print it, not an
exception: that is where somebody is deciding what is worth deleting.
The sentence used to say "images or volumes", and volumes came off it
(devlaunch#325). Deleting a workspace now removes the named volumes its
devcontainer created, see what a delete takes with
it, so a disclaimer that still covered them would
be describing a leak that has been fixed. The --prune half of the pair frees
volumes too, and its reason changed rather than disappeared: it still never
deletes a workspace, it reclaims the volumes of one devpod has already forgotten,
from the copy dl keeps.
The 39 orphans above are not among them. Nothing reaches those but a pattern.
It is a sentence, not a measurement. dl runs no docker command to print
it, so there is nothing to be slow and nothing to fail where Docker is absent,
stopped, or reachable only as another user. The figures above are this page's,
from the host it was measured on, not from your machine. docker system df is
where yours are.
And it points rather than offers. There is deliberately no dl flag that
removes an image, and no list of image ids here to paste into docker image rm.
Images devpod builds carry no devlaunch or devpod label, so any list dl printed
would be a guess at which of them belong to these workspaces, and docker image prune -a is not scoped to devlaunch at all: it would take images built by
everything else on the machine. Deleting them is a decision with your own
containers on the other side of it, and docker system df is the tool that shows
you what it costs.
| artifact | who reclaims it | what makes that safe |
|---|---|---|
loose ref files under a bare cache's refs/ |
the background freshness sweep, with one git pack-refs --all after each fetch that succeeded |
packing changes how a ref is stored and not which refs exist, so there is nothing to prove and nothing to consent to |
Every ref a fetch updates is written as a file, and a file costs a whole
filesystem block: about 4096 bytes each, against the 81 or so the same ref takes
as a line in packed-refs. Nothing used to collapse them. pack-refs --auto is a
documented no-op on git's files ref backend, and dl runs no gc on a bare, so
gc.auto never gets the chance either.
The cost is one block per ref, which means it tracks how many branches and
tags a remote leaves open and has almost nothing to do with how big the
repository is. Measured across ten real remotes with git ls-remote,
torvalds/linux carries 1887 refs against microsoft/vscode's 5342 and
rust-lang/rust's 334, with a median around 370. So a whole cache of 20 to 40
repositories holds something like 30 MB to 60 MB of loose refs, a couple of
percent of one bare's own size. Disk is not the reason this is here.
What carries it is placement. The broad sweep is the only thing in dl that
fetches every head and tag, so it is the only thing that makes loose refs in
quantity, and it already holds the repository's lock while it does. Packing there
costs one more bounded git call in a scope that just spent its whole network
budget, and it happens only on a pass that actually fetched. Measured on git
2.51.1 over a bare of 551 refs, 301 of them loose, those files held 1204 KiB of
blocks against a 30 KiB packed-refs for all 551, and the pack itself took 23 ms.
There is a second payment, and it is banked rather than collected. A guard that walks every ref on the bare to decide whether a clone is safe to remove reads one file instead of thousands once the refs are packed, and such a probe measured 2.8 ms against 5.3 ms on that same bare. No shipped code collects that yet. The bare-side reachability guard is decided and not built, and what ships today asks the clone instead. So the saving is a reason to keep this once that guard arrives, and it is not a reason this is here now.
A pack that refuses is not a fetch that failed. The fetch is the point of the sweep and the packing is the optional half, so a refusal becomes a notice carrying the repository and git's own words, the record's freshness stamp still lands, and the next sweep tries again. Withholding the stamp would make every later pass re-fetch the whole repository forever on account of a representation change that did not come off.
The notice itself still reaches nobody, and it never will: the sweep runs
detached with its output discarded, so every line it raises goes to a null
descriptor. What reaches somebody is the record. A pass that refuses a pack
writes the repository and git's own words into metadata.json beside that
repository's freshness stamp, and dl --ls prints them under its table:
Last cache sweep of blooop/devlaunch: could not pack the refs it fetched: fatal: unable to create 'packed-refs.lock': Permission denied
dl --ls --json carries the same thing as a lastSweep object on every row
whose repository has one. Three things bound what that costs. The note is
overwritten on every pass that acts on a repository, so a cache holds at most one
per repository and there is no rotation and no second file. A pass that goes
cleanly clears it, so a machine whose permissions have been fixed stops
complaining on its own. And a pass that attempted nothing, because the interval
had not elapsed or another run held the lock, leaves the last note standing
rather than reporting a clean sweep it never ran.
What a refusal costs while nobody has yet acted on it is bounded too: loose refs are one file per ref rewritten in place rather than appended, so a pack that keeps failing holds the ref count flat at what one sweep writes instead of growing it.
Packing does not change what a later prune may delete. A ref the remote
retracts is removed whether it was loose or packed: git rewrites packed-refs
through the same ref transaction that unlinks a loose file, and a ref that was
loose over a stale packed line loses both, so nothing comes back at an old sha.
The only difference is cost, and it falls on the prune rather than here, since
removing a packed ref rewrites the whole file where removing a loose one unlinks a
single path.
dl keeps its own record of every workspace, and devpod keeps one too. They
agree until the naming that connects them moves, and it has moved twice: first when
workspace ids and clone-directory names gained a hashed suffix, and again when that
suffix was cut from eight characters to four. Both times dl's records were migrated
to the new naming and devpod's were not, because nothing knew to
touch them. On the host that reported the first, 36 of 39 devpod workspaces recorded
a source folder that was missing, or was a stub with no .git in it, while the
real checkout sat next to it under the new name. Nothing was deleted and nothing
was corrupted: dl was simply asking devpod about workspaces devpod had never
been given, and devpod was answering correctly that there were none.
Two things fix that, and they are different jobs. dl now writes the devpod
workspace id down when it creates a workspace, so the naming can move again
without taking anything with it. That is automatic and needs no command. It
does nothing for the records that already disagree, because they were written
before there was a field to write it in. dl --reconcile is for those:
$ dl --reconcile
devpod workspaces sourced under /home/you/.cache/devlaunch/repos at something that is not a clone:
Re-pointing 2:
- devlaunch-main: .../blooop/devlaunch/main -> .../blooop/devlaunch/devlaunch-main-3j1t
- bencher-test1: .../blooop/bencher/test1 -> .../blooop/bencher/bencher-test1-mxvm
Each of these needs `dl <workspace> recreate` afterwards: the container
still has the old source bind-mounted, and no record change moves a mount.
Leaving 1, which dl will not guess at:
- rockerc-main (.../blooop/rockerc/main): no clone of that repository answers to this name
Nothing here is deleted. `dl <workspace> rm` is how one goes, if it should.
It matches the two sides by path, never by id. The id is the thing that
changed, so it connects nothing, while the source folder devpod kept still names
the owner and the repository exactly, and its last component still names the
branch in one of the three ways dl has named a clone directory. Where that
match is not unique it is refused rather than guessed: a clone a live workspace
already opens, at it or anywhere under it, is never taken from it, a clone two
dead records both match is claimed by neither, and a name that two clones answer
to (the old flattened spelling turned feature/auth, feature auth and
feature:auth all into feature-auth) adopts neither of them. If a live
workspace's source cannot be followed at all, the whole command stops the way
dl --prune does, because such a workspace could be holding any of the clones on
offer. Nothing is ever deleted. A workspace dl cannot match
is named and left exactly where it is, because whether a workspace is finished
with is not something dl can know, and the two mistakes are not the same size.
Run it as often as you like. A repaired workspace is no longer sourced at a non-checkout, so a second run finds nothing to do.
A re-pointed workspace still needs rebuilding. Its container was built with
the dead path bind-mounted into it, and changing a record does not move a mount.
dl <workspace> recreate is what finishes the repair, and it is the step that
needs Docker.
Do not point an old dl at a reconciled cache. A dl from before the naming
changed derives the old directory name, does not find it, and treats the launch
as a cold one: it clones a second directory under the old name, registers a
second devpod workspace, and rewrites that branch's record with the old naming
and an empty workspace id. That undoes the repair for that one workspace, and
leaves you two clones of the branch. It is not destructive and the next
dl --reconcile sorts it out, but a machine that runs both builds against one
cache will keep re-breaking. Upgrade the old one, or give it its own
XDG_CACHE_HOME.
One workspace per branch means workspaces accumulate, and --purge is the wrong
tool for tidying: it is all-or-nothing and takes the caches with it.
devlaunch does not decide which workspaces are finished. Whether a piece of
work is over is a fact about a ticket, a review, or somebody's intent, and dl
knows about clones and containers. Inferring it from the branch, merged into
the default or deleted from the remote, was tried and dropped: it reads like a
git fact but is a guess at intent, and it cannot tell a squash-merged branch
from an abandoned one. So dl supplies the two halves a tool that does know
needs, and that tool drives the cleanup:
dl --ls --json # what exists, and what each workspace holds
dl --ls --json --size # ...and what removing each one would free
dl <workspace> rm # remove oneThe JSON reports, per workspace: id, devlaunch (did dl create it),
repo, branch (what the workspace was made for), checkedOut (what its clone
is on now, which can differ), path, state, lastUsed, and the field a
cleanup tool must not ignore, unsaved:
{
"id": "devlaunch-wayfinder-devlaunch-80-myzf",
"devlaunch": true,
"repo": "blooop/devlaunch",
"branch": "wayfinder/devlaunch-80",
"state": "Stopped",
"unsaved": {
"wouldLose": "2 uncommitted change(s) (pixi.lock, notes.md) and 1 unpushed commit(s)"
}
}unsaved is an object with exactly one key, and the key says which of three
answers it is:
unsaved |
Meaning |
|---|---|
{"nothingToLose": true} |
Everything in the clone exists on a remote too. Deleting it costs nothing. |
{"wouldLose": "<what>"} |
Uncommitted changes (untracked files included), commits no remote has, or both. |
{"couldNotTell": "<why>"} |
git could not read the clone as a repository: a half-removed .git, an interrupted delete. The files are still there and nothing has established that they exist anywhere else. |
The commits half of that answer is asked of every ref in the clone, not of the
branch it happens to have checked out. Commit on wip, switch back to main, and
the work on wip is still the only copy of itself; a question scoped to the
checked-out branch could not see it, and reported the clone as free to delete. So
the answer covers every local branch, every worktree's HEAD including detached
ones, and the stash, which is one ref per clone and holds work that exists nowhere
else either.
Tags are the one ref kind the answer has to think about, and both directions of
getting it wrong have a ticket. A tag your remote carries, but which no remote
branch reaches any more, must not read as unpushed: a repository that tags
releases on branches it then deletes has those by the hundred, and one does, at
265 commits reachable only from its tags. That was what stood between six of the
eight workspaces on a host and being deleted, at 265 to 269 unpushed commits
apiece and none of it real (#485). A guard in that state is not a clone kept for
the price of some disk. It is a guard that has to be --forced past to delete
anything, until --force is what you type without reading, over the clone that did
hold an hour of work as readily as over this one. But a tag you typed here and
never pushed is the opposite case, and the backup habit reaches it in two commands:
tag before a rewrite, move the branch off the tag, and that commit exists in one
place on earth (#487).
A clone cannot tell the two apart on its own, because no remote-tracking ref
carries a tag: nothing in refs/tags/ says which name arrived in a fetch. What
knows is the bare mirror under repos/<owner>/<repo>/.bare, which dl fetches
tags into and clones the workspace from. So the rule is a comparison, and it costs
no network:
| The tag in your clone | Counted as work at risk? |
|---|---|
| The mirror has it, at the same object | No. It came off the remote. |
| The mirror has not got it | Yes. Nothing but this clone has ever seen it. |
| The mirror has the name, at another object | Yes. It was moved or retyped here, and what it used to reach may be nowhere else. |
| There is no mirror to ask | Yes, every tag. |
The last row is the same principle the whole guard is built on: a check that
cannot establish safety fails towards keeping the clone. It is reached by a clone
dl has no record for and by a cache directory that has been deleted out from
under a workspace, and in both the answer is a clone kept, which costs disk and
nothing else.
Where a tag is the reason, the refusal says so and names it, because the sentence a refusal ends with has to be one you can act on:
$ dl blooop/repo@feature rm
error: devlaunch-repo-feature-xyz holds 1 unpushed commit(s), 1 reachable only
from local tag(s) (backup-before-rebase).
Push or commit it, or run: dl blooop/repo@feature rm --force
Both counts, because they answer different questions: how much would be lost, and
how much of it pushing cannot clear. A commit under a tag is already committed, so
"push or commit it" is advice you have already taken, and the tag's name is what
tells you which case you are in. backup-before-rebase is the work being saved.
v0.26.0 is a release you did push, whose tag has not reached the mirror yet,
which is what a dl --refresh fixes and what nothing else on the machine would
have told you.
The count that is not attributed to a tag is left alone: a commit on a branch
needs no explaining, so a clone holding one of each reads 2 unpushed commit(s), 1 reachable only from local tag(s) (backup), and a tag sitting on a commit some
branch also holds is named nowhere, because it explains nothing about why the
clone is being kept.
The changed paths are named, not just counted, and that matters more than it
looks: a devcontainer that runs a package install in its postCreateCommand can
leave a tracked lockfile modified in every workspace it builds. This repo's
own did, until that install became pixi install --frozen, and as a bare count
that is indistinguishable from an hour of unsaved work. A cleanup tool believing the count would then never clean anything. Named,
it is judgeable. A workspace dl did not create reports devlaunch: false and
no unsaved. unsaved is null exactly where devlaunch is false, and
nowhere else: there is no clone of dl's to protect, and it has no business
inspecting your checkout. (repo and branch are a weaker test and not the
same set: they come from dl's metadata record, and a clone dl owns can have
lost its record while the clone and the work in it are still on disk. That clone
is inspected and reported like any other.)
dl <workspace> rm refuses to delete a clone it would lose work from, both when
the recorded clone holds unsaved work and when it cannot tell what that clone
holds, so a caller that forgets to read the field is still caught. (Recorded,
because that is the directory the guard reads; the case with no record is
neither, and is described below.)
$ dl blooop/repo@feature rm
error: devlaunch-repo-feature-xyz holds 1 unpushed commit(s).
Push or commit it, or run: dl blooop/repo@feature rm --force
$ dl blooop/repo@feature rm
error: devlaunch-repo-feature-xyz: git could not read /home/…/repo/feature:
fatal: not a git repository. devlaunch will not delete a clone it cannot
check. Look at it, or run: dl blooop/repo@feature rm --force
That refusal is the only judgement dl makes here, and it is not about finished
work. It is dl declining to destroy the only copy of something, including
when it cannot prove there is another copy. Say --force if you mean it.
--force changes one more answer: an already-absent workspace counts as
deleted, like rm -f. Unforced, rm reports devpod's refusal to delete a
workspace it does not have; forced, the contract is the state afterwards, not
that a delete happened, which is what lets the cold benchmark's per-run
reset run before the first launch, when there is
nothing to remove yet.
The guard reads dl's metadata record, so the recorded directory is the one it
asks about. (The delete does not always remove that same directory: when the
recorded path is not on disk it falls back to a derived one. That divergence is
older than this guard and is tracked as devlaunch#174.) One case is therefore
neither a refusal nor a delete: a clone under dl's cache that has no record, from a metadata write
that failed, a record pruned, or a cache restored without one. The listing still
reports what that clone holds, so unsaved is the field to read; but rm
removes the devpod workspace, exits 0 without asking for --force, and leaves
the clone on disk, because there is no recorded directory for it to remove
either. Nothing is destroyed, and nothing then points at the clone: it is yours
to keep or to rm -rf by hand.
wf is the caller this was built for. It
names its branches after its tickets, so it knows which workspaces belong to
finished work and removes those.
dl --ls --size adds a SIZE column, and dl --ls --json --size adds a disk
object beside the other per-workspace facts:
$ dl --ls --size
WORKSPACE TYPE SOURCE SIZE LAST USED
kinisi-ros-main-lubadaha local /home/…/repos/kinisi-robotics/kinisi_ros/main 64.9 MiB 2026-08-08 11:43:27
my-own-checkout local /home/…/projects/scratch - 2026-08-01 09:12:04
The number is what deleting that workspace would give back, not what du
prints. Those differ, and the gap is the point of the design. A repo is cloned
once into a bare cache and every workspace clone hardlinks its git objects out
of that one copy, so the objects exist once on disk however many workspaces
share them. A size that walked each workspace on its own, which is what du
does when you point it at one directory, counting the blocks every file in it
occupies, bills each workspace for the whole shared pool.
The measurement the row above comes from, taken with the shipped code on one
machine (Ubuntu 24.04, ext4, warm page cache) on a real clone of that repo made
by git clone from the bare in dl's own cache:
| bytes | |
|---|---|
du -s --block-size=1 on the clone alone |
353,230,848 |
what dl --ls --size reports for it |
68,050,944 |
what dl --ls --size reports for the bare it clones from |
651,264 |
du -sc --block-size=1 over both together |
353,882,112 |
du bills that workspace 5.2x what deleting it would actually free. The
difference is a single 270,823,424-byte pack file with one link in the clone and
one in the bare, so removing either end frees none of it.
That sharing is a promise, not a coincidence, and a test holds it to that.
git clone <path> <path> hardlinks pack files by default, and the default is
all that was ever keeping it true. A file:// URL, an intermediate copy, or an
explicit --no-hardlinks would each forfeit it with nothing failing and no
warning printed. Measured on this repo, with du -sc over the cache and each
clone's .git, ext4, git 2.55.0, that is 2400 KB for the cache plus one
workspace against 4472 KB unshared, and 196 KB rather than 2268 KB of .git for
every workspace after the first. So an integration test asserts the pack files are
the cache's, same inode and more than one link, and that assertion goes red on
all three. No clone flag is used to guard it: --local is already the default
and does not even reject a file:// source, and --shared/--reference were
measured to leave a workspace that fails git fsck once the cache has fetched
and gc'd, for a 2 KB saving.
Sharing does erode, in one measured way that is a safety property rather than a
fault. When the cache repacks, an existing workspace's pack loses its second
link and becomes that workspace's own complete copy, still passing git fsck.
The workspace stops being cheap and never stops being valid, which is the trade
--shared and --reference get wrong, and the reason they are not used.
Large files are shared the same way, but nothing about git clone does it for
you. git-lfs objects are not git objects: the clone does not carry them at
all, so a workspace of an LFS repo used to download the entire payload from the
forge and keep a private copy of it in .git/lfs/objects, every workspace,
every time, on top of the worktree copy. dl now makes the bare cache the
repo's LFS store as well: the payload is fetched once into <repo>/.bare/lfs
for the branch being launched, and each workspace materializes out of that,
which git-lfs does by hardlinking. Measured with git-lfs 3.7.1 on ext4: the
workspace's object file is the same (st_dev, st_ino) as the cache's, so its
store costs nothing, and the materialization succeeds with the remote deleted
from disk. The second workspace of an LFS repo touches the network for its
large files not at all. What remains per workspace is the worktree copy, which
is real bytes and cannot be shared: a container build has to be able to read
them. If the cache cannot supply an object, on a first launch offline or for a
payload the branch alone introduces, the old download from origin still runs, and a
workspace left holding pointer files is retried on the next launch rather than
written off.
Nothing about that is written into the workspace's .git/config, and that
restraint is load-bearing rather than tidy: dl bind-mounts the clone
directory into the devcontainer and .bare is a sibling that is not mounted, so
an lfs.storage entry or an added remote naming a host path would break every
git checkout of an LFS repo inside the container while working perfectly on
the host. A test asserts the clone keeps exactly one remote, still pointing at
the forge, and no lfs.storage at all.
So dl counts a file only when every one of its hardlinks lies inside the
workspace being measured. Two consequences, both deliberate:
- The sizes do not add up to the size of the cache. Bytes shared between workspaces belong to none of them, because deleting any one frees none of them. They become the last workspace's the moment it is the last one, which is exactly when deleting it would free them. In the table above that is the last two rows read against each other: 68,702,208 reported bytes against 353,882,112 held.
- A workspace's size can change without the workspace changing, when a sibling that was sharing with it goes away. That is the truth about shared storage.
A workspace dl did not create reads - (null in JSON): there is no clone of
dl's there to measure, and walking your own project directory is not dl's to
do. The table and the JSON decide that from the same rule, is the clone one
dl put in its own cache, the same question --purge deletes by, so the two
always name the same set of workspaces as measurable. Where a walk hits a
directory it cannot read, and a container writing into its clone as its own user
makes that happen, the answer is a floor rather than a
total: ≥2.0 MiB in the table, and {"atLeastBytes": …, "unreadable": 1} in
JSON instead of {"exclusiveBytes": …}. A partial measurement never comes back
looking like a complete one.
It is opt-in because it walks the whole clone. Plain dl --ls is one devpod
round-trip and no filesystem work at all, and the walk is O(files) with no
ceiling. Measured with the shipped code on one machine, on Ubuntu 24.04 and ext4
with a warm page cache, five runs after a warm-up and the machine otherwise busy:
a real 8,309-entry clone walked in 24 to 28 ms, this repo's own tree with its built
environment inside it (9,124 entries) in 17 to 21 ms, and a 114,817-entry tree in
232 to 239 ms. No cold-cache figure is quoted because none was taken: dropping the
page cache needs root on that machine. Those are one machine's numbers on warm
cache and yours will differ, but the shape is the point. It grows with the file
count, and a devcontainer that builds its environment inside the clone (this
repo's own does) is most of that count. That is not a bill a listing should
present unasked.
Docker images and named volumes are not counted: dl did not create the layer
store, and a volume is not a directory it can walk. docker system df is the
tool that knows, the same boundary --prune and --purge
name when they finish. Not counting a volume is
a different thing from not removing it: a workspace's volumes go when the
workspace does, see what a delete takes with
it. What is missing here is only the figure.