Honor a container image's USER and WORKDIR by dims · Pull Request #1918 · agent-substrate/substrate

Davanum Srinivas (dims) · 2026-09-26T11:40:09Z

USER resolves as Docker does, against the image's own /etc/passwd and /etc/group: uid:gid as is, a bare uid with its login group (gid 0 without an entry), names looked up or Run/Restore fails, group memberships as supplementary groups. No USER means root. atelet resolves it into the worker's ContainerSpec; the pause container follows the same rule through WorkloadSpec.pause_user. The process starts in the image's WORKDIR (default /), created in the actor's upper when the image lacks it.

Behavior change: images with a USER stop running as root; images with a WORKDIR stop starting in /. On gVisor no snapshot from before this restores: runsc restore checks Process.User and Process.Cwd of every container, and the pause image sets USER 65535. Micro-VM snapshots resume as they were.

Merge after #2392 (image layer owners): without it a non-root process cannot write the files its image gives it. On the micro-VM class, images that serve on a privileged port also need #1904.

Live run on kind, both classes, main + #1906 + #2392 + this PR: a node image (USER 1000, WORKDIR /app, durable dir) and the jupyter demo run as their user, write to their workdir and durable dir, and resume on demand after a suspend. On micro-VM uid 1000 cannot bind port 80 (#1904); gVisor allows it.

Tests pass: go test -race on Linux for atelet, imagecache, ocispec, ateworker-gvisor and kata.
Appropriate changes to documentation are included in the PR

Davanum Srinivas (dims) · 2026-09-27T11:53:58Z

/assign Maya Wang (@mayawang) Benjamin Elder (@BenTheElder)

… note the rootfs root dependency of kagent-upstream#125 (agent-substrate#1918)

Maya Wang (mayawang)

The USER and WORKDIR resolution looks right to me, and moving to moby/sys/user covers Ben's gid point. My main ask is a live run before this merges, since it changes who real images run as. It also needs a rebase: internal/ocispec/parity_test.go conflicts with main.

… root dependency of kagent-upstream#125 (#94) docs(fork): the egress default's upstream exit is agent-substrate#1558; note the rootfs root dependency of kagent-upstream#125 (agent-substrate#1918)

…#1918

Maya Wang (mayawang) · 2026-10-02T20:31:32Z

Two gVisor issues from Preston Holmes (@ptone) that every image this change moves to non-root will hit: #2035 (the actor rootfs / is mode 0700, so a non-root user can't resolve any path) and #2036 (files the image gives to a non-root user show up as owned by uid 0). There's a fork fix for #2036 in giantswarm#115. Neither is covered by this stack, and I think we need a plan for both before this lands, otherwise images with a USER go from running as root to not starting. Davanum Srinivas (@dims) Benjamin Elder (@BenTheElder) let me know what you think.

Benjamin Elder (BenTheElder) · 2026-10-07T01:40:56Z

I agree that we should fix #2036 and probably need to fix it first.

Davanum Srinivas (dims) · 2026-10-15T19:01:58Z

Maya Wang (@mayawang) Plan is roughly spreadh between #1906 / #2392 (fixes #2036) and this PR.

#2035 is fixed in this PR's first commit (merged root 0755, layerless case included); #2430 fixes the upper only. Whichever lands first, I drop the other.

Ran the three together on kind, both classes: USER images start, write their own files and resume after suspend. Details in the live-run thread above. Privileged ports on microVM need #1904.

…nt-substrate#1906) Durable-dir volumes were created `0700`, so a container running as a non-root user could not write its own volume. Now `0777`, as Kubernetes creates an emptyDir: one actor's containers can run as different users, and the `0700` actor directory above keeps other actors out. A restore keeps the mode the snapshot recorded, as emptyDir keeps a pod's own chmod. Volumes created earlier stay `0700`, fine while every container runs as root. No effect until an image's `USER` is honored (agent-substrate#1918). Cleanup of files a non-root container leaves behind landed in agent-substrate#1910. Tests: `TestPrepareDurableDirVolume`. Exercised on kind with the micro-VM class: uid 65532 writes a durable dir, then suspend and resume. - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR

Benjamin Elder (BenTheElder) · 2026-10-16T05:39:57Z

#1906 is in.
#2392 is close.

rootfs and upper were 0700. On gVisor the container's / takes that mode, so a non-root process could not search its own root. 0755 is what containerd and every image root use; the 0700 actor directory above keeps other actors out. The micro-VM guest already sees 0755.

atelet writes the OCI spec before the worker mounts the rootfs, so it cannot read the image's /etc/passwd from a mount. Image.ReadFile reads a file from the cached layers top-down, honoring recorded whiteouts and opaque dirs. Capped at 1 MiB.

Resolve USER as Docker does, against the image's own /etc/passwd and /etc/group: uid:gid as is, a bare uid with its login group (gid 0 without an entry), names looked up or Run/Restore fails, group memberships as supplementary groups. No USER means root. atelet resolves it into the worker's ContainerSpec; the pause container follows the same rule through WorkloadSpec.pause_user. On gVisor every snapshot taken before this needs re-taking, whatever the image: runsc restore checks Process.User, and the pause image sets USER 65535.

Start the process in the image's WORKDIR instead of "/", and create the directory in the actor's upper when the image lacks it. atelet resolves it into the worker's ContainerSpec. runsc restore checks Process.Cwd as well as Process.User, so gVisor snapshots from before this need re-taking.

Substrate makes no compatibility promise before v1.0.0, so a note about one release's snapshots is noise in the upgrade procedure. The API guide describes how the image's USER and WORKDIR apply.

This was referenced Sep 26, 2026

agentgateway e2e lane fails on main since #1809: ateom-for-actor certificates carry no ActorIdentity extension #1922

Closed

imagecache: make the merged rootfs root searchable by non-root #1905

Closed

Davanum Srinivas (dims) force-pushed the pr/image-user branch from 7321a72 to 4ab3b53 Compare September 26, 2026 23:34

Davanum Srinivas (dims) mentioned this pull request Sep 27, 2026

atelet: make durable-dir volumes writable by non-root containers #1906

Merged

2 tasks done

Davanum Srinivas (dims) force-pushed the pr/image-user branch from 4ab3b53 to 58e3ad2 Compare September 27, 2026 01:30

Davanum Srinivas (dims) marked this pull request as ready for review September 27, 2026 02:25

Quentin Bisson (QuentinBisson) mentioned this pull request Sep 28, 2026

feat(harness): run Claude Code as the image's unprivileged user when the harness runs as root giantswarm/kagent-upstream#125

Merged

Benjamin Elder (BenTheElder) reviewed Sep 28, 2026

View reviewed changes

Comment thread cmd/atelet/oci.go Outdated

Comment thread internal/ocispec/ocispec.go Outdated

Comment thread cmd/atelet/oci_test.go

Davanum Srinivas (dims) force-pushed the pr/image-user branch from 58e3ad2 to 9ed0a39 Compare September 29, 2026 00:58

Davanum Srinivas (dims) marked this pull request as draft September 29, 2026 01:05

Davanum Srinivas (dims) marked this pull request as ready for review September 29, 2026 01:28

This was referenced Sep 29, 2026

docs(fork): egress default exits through #1558; rootfs root dependency of kagent-upstream#125 giantswarm/substrate#94

Merged

docs(fork): align the ledger with giantswarm/giantswarm#37742 after the 2026-09-30 audit giantswarm/substrate#95

Merged

Quentin Bisson (QuentinBisson) added a commit to giantswarm/substrate that referenced this pull request Sep 29, 2026

docs(fork): the egress default's upstream exit is agent-substrate#1558;…

fd5e65d

… note the rootfs root dependency of kagent-upstream#125 (agent-substrate#1918)

Bowei Du (bowei) added area/node area/security Security related issue/pr kind/feature An enhancement / feature request or implementation labels Sep 30, 2026

Maya Wang (mayawang) reviewed Sep 30, 2026

View reviewed changes

Comment thread cmd/atelet/oci.go Outdated

Comment thread docs/api-guide.md

Comment thread internal/imagecache/imagefile.go Outdated

Davanum Srinivas (dims) force-pushed the pr/image-user branch from 9ed0a39 to 0ca0710 Compare October 1, 2026 11:51

Davanum Srinivas (dims) changed the title ~~atelet: honor a container image's own USER~~ atelet: honor a container image's USER and WORKDIR Oct 1, 2026

Davanum Srinivas (dims) force-pushed the pr/image-user branch from 0ca0710 to d41a2fc Compare October 1, 2026 12:04

Quentin Bisson (QuentinBisson) mentioned this pull request Oct 1, 2026

fix(imagecache): carry the rootfs root fix of agent-substrate/substrate#1918 giantswarm/substrate#101

Merged

Quentin Bisson (QuentinBisson) added a commit to giantswarm/substrate that referenced this pull request Oct 1, 2026

docs(fork): carry the rootfs root fix of agent-substrate#1918

750f3ea

This was referenced Oct 1, 2026

docs(fork): name the rootfs root commits by their SHAs on the line giantswarm/substrate#102

Merged

feat(atelet): carry the image USER and WORKDIR stack of agent-substrate/substrate#1918 giantswarm/substrate#103

Merged

Quentin Bisson (QuentinBisson) added a commit to giantswarm/substrate that referenced this pull request Oct 1, 2026

docs(fork): carry the image USER and WORKDIR stack of agent-substrate…

3deee38

…#1918

Quentin Bisson (QuentinBisson) mentioned this pull request Oct 2, 2026

fix(imagecache): keep the owner an image layer gives its files giantswarm/substrate#115

Merged

2 tasks done

Benjamin Elder (BenTheElder) reviewed Oct 7, 2026

View reviewed changes

Comment thread docs/upgrade.md Outdated

Benjamin Elder (BenTheElder) mentioned this pull request Oct 7, 2026

Build the OCI spec in ateom instead of atelet #2293

Merged

2 tasks done

Anish Ramasekar (aramase) mentioned this pull request Oct 9, 2026

Keep the file owners an image's layers record inside the actor #2392

Open

1 of 2 tasks

Quentin Bisson (QuentinBisson) mentioned this pull request Oct 9, 2026

docs(fork): name agent-substrate/substrate#2392 as the exit of the layer-owner patch giantswarm/substrate#221

Merged

Davanum Srinivas (dims) force-pushed the pr/image-user branch from d41a2fc to 9ac6eae Compare October 15, 2026 13:35

Davanum Srinivas (dims) changed the title ~~atelet: honor a container image's USER and WORKDIR~~ Honor a container image's USER and WORKDIR Oct 15, 2026

Davanum Srinivas (dims) force-pushed the pr/image-user branch 3 times, most recently from 4d5bcaf to 6361a58 Compare October 17, 2026 01:48

Davanum Srinivas (dims) added 5 commits October 18, 2026 17:04

docs: drop the upgrade note on the USER and WORKDIR change

41969a1

Substrate makes no compatibility promise before v1.0.0, so a note about one release's snapshots is noise in the upgrade procedure. The API guide describes how the image's USER and WORKDIR apply.

Davanum Srinivas (dims) force-pushed the pr/image-user branch from 6361a58 to 41969a1 Compare October 18, 2026 21:13

Navigation Menu

Repository navigation

Honor a container image's USER and WORKDIR - #1918

Honor a container image's USER and WORKDIR#1918
Davanum Srinivas (dims) wants to merge 5 commits into
agent-substrate:mainfrom
dims:pr/image-user

Davanum Srinivas (dims) commented Sep 26, 2026 •

edited

Loading

Uh oh!

Davanum Srinivas (dims) commented Sep 27, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Maya Wang (mayawang) left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Maya Wang (mayawang) commented Oct 2, 2026

Uh oh!

Benjamin Elder (BenTheElder) commented Oct 7, 2026

Uh oh!

Uh oh!

Davanum Srinivas (dims) commented Oct 15, 2026

Uh oh!

Benjamin Elder (BenTheElder) commented Oct 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Davanum Srinivas (dims) commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Davanum Srinivas (dims) commented Sep 27, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Maya Wang (mayawang) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Maya Wang (mayawang) commented Oct 2, 2026

Uh oh!

Benjamin Elder (BenTheElder) commented Oct 7, 2026

Uh oh!

Uh oh!

Davanum Srinivas (dims) commented Oct 15, 2026

Uh oh!

Benjamin Elder (BenTheElder) commented Oct 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Davanum Srinivas (dims) commented Sep 26, 2026 •

edited

Loading