Skip to content

Make the actor container RLIMIT_NOFILE configurable #2064

Description

@ygao-g

What happened

atelet writes RLIMIT_NOFILE 1024, soft and hard, into the OCI spec of every actor
container (internal/ocispec/ocispec.go). There is no way to raise it.

Proposal

An atelet flag, --actor-nofile-limit, that sets the soft and hard limit for actor
containers on the node. Default stays 1024, so nothing changes unless it is set.

I prototyped a fix and verified it: with the flag at 65536 a probe actor opened 1600 sockets on gVisor.

Links

cc Benjamin Elder (@BenTheElder) Jing (@milantracy) Julian Gutierrez Oschmann (@juli4n)

Suggested labels for a maintainer to add: kind/feature, area/node.

Activity

  1. BenTheElder commented on Oct 1, 2026

    @BenTheElder
    Collaborator

    I think we need to stop having atelet control the OCI spec at all versus leaving it to the ateoms. cc Haven Xia (@HavenXia) re: decoupling ateom/atelet for ugprade. There's a weird layering here that doesn't make sense anymore.

  2. BenTheElder commented on Oct 1, 2026

    @BenTheElder
    Collaborator

    We should really look at how kubernetes pods let you influence this and/or what their defaults are. This might be an actor option, a worker option, or just a better default.

  3. BenTheElder commented on Oct 1, 2026

    @BenTheElder
    Collaborator

    Yuan Gao (@ygao-g) can you please investigate those before we add this atelet flag? I'm not sure this is the right abstraction and we're going to commit to some stability soon. It will be faster to review with that context. Otherwise I will do the same investigation before considering the code.

  4. HavenXia commented on Oct 1, 2026

    @HavenXia
    Member

    +1 for a better default.

    Atelet configurations are write in the atelet.yaml which doesn't has entry point for a --actor-nofile-limit flag. So introducing such flag doesn't give either us or customer the ability to adjust.

  5. ygao-g commented on Oct 2, 2026

    @ygao-g
    CollaboratorAuthor

    Benjamin Elder (@BenTheElder) Haven Xia (@HavenXia)

    Looked into how Kubernetes handles this.

    Pods cannot configure it. The Pod spec has no rlimit field (kubernetes/kubernetes#3595). A container inherits RLIMIT_NOFILE from the container runtime's process, so in practice it is a node-level setting.

    What pods actually get here, read from /proc/<pid>/limits:

    Where Runtime Soft Hard
    GKE, plain busybox pod containerd 2.2.5 on COS 1048576 1048576
    GKE node: containerd, every shim, atelet, ateom-gvisor, runsc sandbox same 1048576 1048576
    kind, plain busybox pod containerd 2.3.4 in Docker 1073741816 1073741816
    Substrate actor today pinned in internal/ocispec 1024 1024

    So on GKE the whole chain down to the runsc sandbox process sits at 1048576/1048576.

    Two more data points: runsc's own default when the spec carries no Rlimits is 1048576/1048576, capped to the host runsc process.

    For reference, Commands and raw output are available in https://gist.github.com/ygao-g/2cc30e2805de6a6c2c6eecf88d386213

  6. ygao-g commented on Oct 5, 2026

    @ygao-g
    CollaboratorAuthor

    Benjamin Elder (@BenTheElder) Haven Xia (@HavenXia) WDYT? Okay to directly modify the current default 1024 to a larger value like 1048576?

  7. HavenXia commented on Oct 5, 2026

    @HavenXia
    Member

    [Benjamin Elder (Benjamin Elder (@BenTheElder))](https://github.com/BenTheElder) [Haven Xia (Haven Xia (@HavenXia))](https://github.com/HavenXia) WDYT? Okay to directly modify the current default 1024 to a larger value like 1048576?

    That SGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area/nodekind/featureAn enhancement / feature request or implementation

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions