Skip to content

fix: bound atenet-egress's ephemeral storage, oauth2-proxy from gsoci - #912

Merged
teemow merged 2 commits into
mainfrom
fix/880-kyverno-audit
Oct 10, 2026
Merged

teemow merged 2 commits into
mainfrom
fix/880-kyverno-audit

Conversation

@teemow

@teemow teemow commented Oct 10, 2026

Copy link
Copy Markdown
Member

Closes #880.

Problem

A 4.x installation with Kyverno reports two Audit findings of the platform's own workloads:

  • require-emptydir-requests-and-limits on Deployment ate-system/atenet-egress: the Substrate chart renders the drain-signal emptyDir without a sizeLimit and both containers (agentgateway, ext-proc) without resources.
  • restrict-image-registries on Deployment kagent/kagent-oauth2-proxy: the kagent chart's oauth2-proxy subchart defaults to quay.io/oauth2-proxy/oauth2-proxy:v7.15.5.

Change

The egress gateway's ephemeral storage. substrate.atenetEgress carries ephemeral-storage requests 16Mi and limits 256Mi for both containers (they write nothing but their logs and the drain marker; CPU and memory stay the installation's to size) and a 16Mi sizeLimit for the emptyDir. Two halves, and why:

  • How the chart renders Substrate's egress today: the substrate: block travels verbatim into the substrate HelmRelease's values, the Substrate chart carries no values schema (a key it does not know is pruned in silence), and components.substrate.versionRange confines one minor (agent-platform.substrate.validateRange). The Substrate line now takes the three keys (atenetEgress.resources, atenetEgress.extProc.resources, atenetEgress.drainSignal.sizeLimit, fix(chart): bound the egress gateway's ephemeral storage substrate#255: upstream-shaped, with defaults that make the chart's render compliant on its own), from its release 1.8.0 on. That is the half that survives every Substrate upgrade: values are the chart's contract.
  • The 4.x range on main is >=1.7.0 <1.8.0, and the line's giantswarm branch is already past 1.7.0, so no release the range admits can carry the keys before a re-pin. Below the release agent-platform.substrate.egressStorageFloor names (1.8.0) the meta chart therefore applies the same bound as one kustomize strategic-merge patch on the substrate HelmRelease (a Flux postRenderer, agent-platform.substrate.egressStoragePatch, built from the same values): the containers agentgateway and ext-proc merged by name, the volume drain-signal likewise. Those names are upstream's canonical ate-system render, unchanged from 1.6 through the line's 1.8 candidates, and the range confines one minor, so a Substrate patch within it keeps them; the re-pin that moves the floor past 1.8.0 retires the patch and the chart renders the bound from the forwarded keys. A patch without the floor would have had no end of life and no guard against a chart that renders the bound itself; the keys without the patch would have fixed nothing on today's range. A null key bounds nothing (the chart's own default from the floor on, no patch for it below).

oauth2-proxy from gsoci. kagent.oauth2-proxy.image pins gsoci.azurecr.io/giantswarm/oauth2-proxy:v7.15.5, retagger's mirror of the subchart's default at the same digest (sha256:8498b0d0… on both registries). The pin holds the tag of the subchart the kagent release at the range's floor bundles (kagent 1.6.0: oauth2-proxy chart 10.7.1, appVersion 7.15.5), so a kagent re-pin that bumps the subchart moves it.

Checks. make verify-substrate-egress-ephemeral-storage: the substrate chart at the range's floor, rendered with the forwarded values, fails the rule by itself below the floor (the floor is not stale) and passes with the HelmRelease's one patch merged by name (every container and volume the patch names rendered by the release); at or above the floor no patch renders and the chart passes from the forwarded keys; the three knobs null render no patch; the forwarded block is verbatim. make verify-kagent-oauth2-proxy-image: the pin reaches the kagent release verbatim, is the gsoci copy of the subchart the kagent floor bundles, and is published. Goldens regenerated (tests/golden/meta-*.yaml: the forwarded keys, the patch on the engine-off shape), README rows by helm-docs, CHANGELOG.

Proof

  • Render (helm template with the chart's defaults and the CI inputs): the substrate HelmRelease carries
    postRenderers:
      - kustomize:
          patches:
          - patch: |-
              apiVersion: apps/v1
              kind: Deployment
              metadata:
                name: atenet-egress
              spec:
                template:
                  spec:
                    containers:
                    - name: agentgateway
                      resources: {limits: {ephemeral-storage: 256Mi}, requests: {ephemeral-storage: 16Mi}}
                    - name: ext-proc
                      resources: {limits: {ephemeral-storage: 256Mi}, requests: {ephemeral-storage: 16Mi}}
                    volumes:
                    - {name: drain-signal, emptyDir: {sizeLimit: 16Mi}}
            target: {kind: Deployment, name: atenet-egress}
    and the kagent HelmRelease's values oauth2-proxy.image: {registry: gsoci.azurecr.io, repository: giantswarm/oauth2-proxy, tag: v7.15.5}.
  • The patch through the real tool: the substrate chart 1.6.2 and 1.7.0 pulled from gsoci, rendered with the forwarded values and put through kubectl kustomize with the patch — on both, agentgateway and ext-proc carry requests.ephemeral-storage: 16Mi, limits.ephemeral-storage: 256Mi, drain-signal is emptyDir: {sizeLimit: 16Mi}, images unchanged.
  • make verify-substrate-egress-ephemeral-storage: "the substrate chart 1.7.0 (below 1.8.0) fails the rule by itself (ext-proc mounts an unbounded emptyDir and has no resources.requests.ephemeral-storage; … limits …) and passes with the HelmRelease's patch merged by name"; "the three knobs null render no patch". make verify-kagent-oauth2-proxy-image: "kagent 1.6.0 bundles oauth2-proxy chart 10.7.1 (appVersion 7.15.5); the pin gsoci.azurecr.io/giantswarm/oauth2-proxy:v7.15.5 is its gsoci copy and is published". verify-meta, verify-target, verify-substrate-images, verify-images, verify-workerpool green.
  • Live, a test installation on 4.x with Kyverno (read-only, before): kubectl get policyreports -n ate-system shows Deployment atenet-egress with FAIL 1 — require-emptydir-requests-and-limits/autogen-check-emptydir-requests-limits … failed at path /resources/limits/; -n kagent shows Deployment kagent-oauth2-proxy and its ReplicaSets with FAIL 1 — restrict-image-registries/autogen-validate-registries … failed at path /spec/template/spec/containers/0/image/; the live Deployment runs quay.io/oauth2-proxy/oauth2-proxy:v7.15.5. The installation follows the release candidates of this chart through its GitOps range; the policyreports after the candidate rolls there are recorded on fix: atenet-egress emptyDir limits and oauth2-proxy registry on 4.x #880.

Written by an agent.

@teemow
teemow requested a review from a team as a code owner October 10, 2026 10:10
@circleci-architect

circleci-architect Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Published Helm chart agent-platform

4.124.1-rca6c70c7t20261010101132h88d601a
Chart agent-platform
Version 4.124.1-rca6c70c7t20261010101132h88d601a
OCI reference oci://gsoci.azurecr.io/charts/giantswarm/agent-platform:4.124.1-rca6c70c7t20261010101132h88d601a
Digest sha256:d526f4417d9ed28b82bf4899d47310b932b431dfa44a866d0b3d77650aa0e05f
Registry public — gsoci.azurecr.io
Git catalog giantswarm-test-catalog (index)
Pull this chart
helm pull oci://gsoci.azurecr.io/charts/giantswarm/agent-platform --version 4.124.1-rca6c70c7t20261010101132h88d601a

Posted by architect-orb · build 20992 · commit 88d601a · updated in place on every push

@circleci-architect

circleci-architect Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Published Helm chart agent-platform-connectivity

4.124.1-rca6c70c7t20261010101132h88d601a
Chart agent-platform-connectivity
Version 4.124.1-rca6c70c7t20261010101132h88d601a
OCI reference oci://gsoci.azurecr.io/charts/giantswarm/agent-platform-connectivity:4.124.1-rca6c70c7t20261010101132h88d601a
Digest sha256:16906b1f0bea86e9a1b9f19216cb11e5ba7b9336b8f207362d372abb3283f04e
Registry public — gsoci.azurecr.io
Git catalog giantswarm-test-catalog (index)
Pull this chart
helm pull oci://gsoci.azurecr.io/charts/giantswarm/agent-platform-connectivity --version 4.124.1-rca6c70c7t20261010101132h88d601a

Posted by architect-orb · build 20995 · commit 88d601a · updated in place on every push

@teemow
teemow merged commit b70de28 into main Oct 10, 2026
14 checks passed
@teemow
teemow deleted the fix/880-kyverno-audit branch October 10, 2026 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: atenet-egress emptyDir limits and oauth2-proxy registry on 4.x

1 participant