diff --git a/FORK.md b/FORK.md index 6f474050fc..1717500cf3 100644 --- a/FORK.md +++ b/FORK.md @@ -108,6 +108,7 @@ Everything on `giantswarm` that is not in the pin (`git log v0.4.0-alpha1..giant | A sandbox's `/etc/hosts` names its own hostname: both ateoms write `127.0.0.1 localhost`, `::1 localhost ip6-localhost ip6-loopback` and `127.0.1.1 actor` (`dns.WriteRootfsHosts`, `ocispec.Hostname`) into each application container's bundle rootfs on run and restore — the micro-VM ateom beside the guest's resolv.conf, the gVisor ateom into the bundle's private upper rather than as a bind like resolv.conf, because runsc restore refuses a mount set that differs from the checkpointed one (`--restore-spec-validation` enforces by default) and a new mount would fail every existing snapshot. Tests: `TestWriteRootfsHosts`, `TestWriteRootfsHostsDoesNotFollowAPlantedSymlink`; the identity e2e asserts the entry, that the hostname resolves to 127.0.1.1 after a golden restore and a suspend/resume, and that neither actor's worker logged a DNS relay warning for `actor.` | the hostname `actor` was in no sandbox's `/etc/hosts`, so every lookup of it (`hostname -f`, a runtime resolving itself at start) left the sandbox through the DNS relay; cluster DNS forwarded the bare name upstream, which answered SERVFAIL, and each e2e lane run logged 96 relay warnings for `actor.` ([#220](https://github.com/giantswarm/substrate/issues/220)) | `giantswarm`: [#243](https://github.com/giantswarm/substrate/pull/243) | to file: agent-substrate/substrate `main` (`fd7a9cde`, 2026-10-10) writes no `/etc/hosts` into the sandbox; queued in the upstream engagement list | | The egress gateway takes further credential providers (`credentialProvider.additionalProviders`: `uriAuthority`, a lowercase DNS name, and `host`, `..svc:`, rendered on both egress listeners beside the bundled `k8s.io` provider, dialed with the gateway's pod identity and verified against the `servicedns.podcert.ate.dev` trust bundle; an entry cannot replace `k8s.io` or repeat an authority; `charts/substrate/tests/atenet_egress_credential_providers_test.yaml`) | kagent injects the caller of the current turn at egress through its own provider, so the caller's bearer never enters the actor: agentgateway already selects a provider by URI authority and fetches the actor's egress policy on every request, but the chart rendered one provider only (giantswarm/giantswarm#38054) | [#248](https://github.com/giantswarm/substrate/pull/248) (carries [#104](https://github.com/giantswarm/substrate/pull/104) onto the v0.4.0-alpha1 line) | to file for kagent-dev/substrate (chart only); giantswarm/giantswarm#37742 row to add | | The Kubernetes credential provider mints Google access tokens (`ate-secret://google-access-token.k8s.io/default///`: the entry must hold a service account key; the provider signs its JWT assertion, exchanges it at the key's https `token_uri` for a `cloud-platform` token, caches the token per key while at least fifteen minutes remain, so the gateway's five-minute cache never serves an expired one, and returns the token, never the key; the chart and the kustomize component route both authorities, bundled names of `substrate.egressCredentialProviders` that `additionalProviders` cannot replace, on the HTTP and HTTPS egress listeners to the one provider Deployment) | Vertex AI takes OAuth 2.0 access tokens only, and minting one means signing with the key, which the egress gateway cannot do and an actor must not hold: no kagent agent with a Vertex `ModelConfig` (Gemini or Anthropic on Vertex AI) ran on Substrate ([#142](https://github.com/giantswarm/substrate/issues/142); giantswarm/kagent-upstream#178; [#37742](https://github.com/giantswarm/giantswarm/issues/37742) row 119) | [#250](https://github.com/giantswarm/substrate/pull/250): `git cherry-pick -x` of the upstream-ready branch's commit, which is kagent-dev/substrate's `5ac16e7c` adapted to the `k8s.io/default` URI grammar (upstream's commit predates it and names the authority `google-access-token.kubernetes.io`) | [kagent-dev/substrate#47](https://github.com/kagent-dev/substrate/pull/47) (open); the adapted commit on the pin is branch [`upstream/google-access-token-provider`](https://github.com/giantswarm/substrate/tree/upstream/google-access-token-provider) (`4453a6f4`), offered to #47 | +| The egress gateway's ephemeral storage is bounded (`atenetEgress.resources` for the `agentgateway` container and `atenetEgress.extProc.resources`, ephemeral-storage requests `16Mi` and limits `256Mi` by default, CPU and memory left to the operator; `atenetEgress.drainSignal.sizeLimit`, `16Mi` on the `drain-signal` emptyDir the ext-proc writes its drain marker to; `null` renders the field as before; `charts/substrate/tests/atenet_egress_ephemeral_storage_test.yaml`; README rows) | the chart rendered both containers without `resources` and the emptyDir without a `sizeLimit`, so a cluster policy that requires bounded ephemeral storage for a container mounting an emptyDir (Kyverno's `require-emptydir-requests-and-limits`) reported the egress gateway on every install ([giantswarm/agent-platform#880](https://github.com/giantswarm/agent-platform/issues/880)) | [#255](https://github.com/giantswarm/substrate/pull/255) (`fix(chart): bound the egress gateway's ephemeral storage`) | to file: upstream's `atenet-egress.yaml` renders the same unbounded shape and no upstream issue covers it; queued in the upstream engagement list | | Require `golang.org/x/net` v0.61.0 in every module that had it below (`go.mod`, `hack/tools/{code-generator,controller-gen,go-licenses,ko}`, `internal/plugins/gcp-secret-manager`, `tools/apitool`; `go get golang.org/x/net@v0.61.0 && go mod tidy` in each, then `go mod vendor` for the root module the CircleCI builds compile with `-mod=vendor`; `x/sync`, `x/sys`, `x/term`, `x/text` and `x/tools` move along, and tidy drops the stale indirect requirements `code-generator`'s module file still listed) | CVE-2026-97032 and CVE-2026-78663, two HTTP/2 vulnerabilities in `x/net` below v0.61.0; the platform's Go repositories move together, their CircleCI nancy step refusing the older module | [#252](https://github.com/giantswarm/substrate/pull/252) (`fix(deps): bump golang.org/x/net to v0.61.0 for two HTTP/2 CVEs`) | not for upstream as such: changes versions only and falls away at the re-pin onto the first upstream commit that requires `x/net` ≥ v0.61.0 (upstream's dependency automation moves it) | Twenty-three patches change Substrate ahead of upstream — egress for an actor while it resumes, without which no diff --git a/charts/substrate/README.md b/charts/substrate/README.md index e419f5b0eb..6f5f24a8a3 100644 --- a/charts/substrate/README.md +++ b/charts/substrate/README.md @@ -74,3 +74,6 @@ See `values.yaml` for the full set; the important keys: | `otel.metrics.endpoint` | `""` | OTLP endpoint for metrics, overriding `otel.endpoint` | | `otel.logs.enabled` | `true` | Set to `false` to export no logs. Gates both OTLP log sources: ateapi's actor lifecycle events and the router access log | | `otel.logs.endpoint` | `""` | OTLP endpoint for logs, overriding `otel.endpoint` | +| `atenetEgress.resources` | ephemeral-storage requests `16Mi`, limits `256Mi` | Resources of the egress gateway's `agentgateway` container, as the pod spec takes them; the default bounds the ephemeral storage only (the container writes nothing but its logs), CPU and memory are the operator's to size; `null` renders none | +| `atenetEgress.extProc.resources` | ephemeral-storage requests `16Mi`, limits `256Mi` | Resources of the egress gateway's `ext-proc` container, the same way | +| `atenetEgress.drainSignal.sizeLimit` | `16Mi` | `sizeLimit` of the `drain-signal` emptyDir the ext-proc writes its drain marker to; `null` renders an unbounded emptyDir, which a cluster policy that requires bounded ephemeral storage then reports | diff --git a/charts/substrate/templates/atenet-egress.yaml b/charts/substrate/templates/atenet-egress.yaml index 7be456ca75..1924807f74 100644 --- a/charts/substrate/templates/atenet-egress.yaml +++ b/charts/substrate/templates/atenet-egress.yaml @@ -177,6 +177,10 @@ spec: path: /healthz/ready port: readiness periodSeconds: 1 + {{- with .Values.atenetEgress.resources }} + resources: + {{- toYaml . | nindent 10 }} + {{- end }} volumeMounts: - name: egress-mitm mountPath: /run/egress-mitm @@ -245,6 +249,10 @@ spec: initialDelaySeconds: 5 periodSeconds: 2 failureThreshold: 3 + {{- with .Values.atenetEgress.extProc.resources }} + resources: + {{- toYaml . | nindent 10 }} + {{- end }} volumeMounts: - name: servicedns mountPath: /run/servicedns.podcert.ate.dev @@ -275,7 +283,12 @@ spec: configMap: name: {{ include "substrate.fullname" (list "atenet-egress-agentgateway-config" .) }} - name: drain-signal + {{- with .Values.atenetEgress.drainSignal.sizeLimit }} + emptyDir: + sizeLimit: {{ . }} + {{- else }} emptyDir: {} + {{- end }} - name: servicedns projected: sources: diff --git a/charts/substrate/tests/atenet_egress_ephemeral_storage_test.yaml b/charts/substrate/tests/atenet_egress_ephemeral_storage_test.yaml new file mode 100644 index 0000000000..dbf54cdaa1 --- /dev/null +++ b/charts/substrate/tests/atenet_egress_ephemeral_storage_test.yaml @@ -0,0 +1,109 @@ +# Copyright 2026 The Agent Substrate Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +suite: atenet egress ephemeral storage +templates: +- atenet-egress.yaml +tests: +- it: bounds the ephemeral storage of both containers and the drain-signal emptyDir by default + documentSelector: + path: kind + value: Deployment + asserts: + - equal: + path: spec.template.spec.containers[0].name + value: agentgateway + - equal: + path: spec.template.spec.containers[0].resources + value: + requests: + ephemeral-storage: 16Mi + limits: + ephemeral-storage: 256Mi + - equal: + path: spec.template.spec.containers[1].name + value: ext-proc + - equal: + path: spec.template.spec.containers[1].resources + value: + requests: + ephemeral-storage: 16Mi + limits: + ephemeral-storage: 256Mi + - contains: + path: spec.template.spec.volumes + content: + name: drain-signal + emptyDir: + sizeLimit: 16Mi + +- it: forwards an operator's resources, merged over the defaults + set: + atenetEgress.resources: + requests: + cpu: 100m + memory: 128Mi + ephemeral-storage: 32Mi + limits: + memory: 512Mi + ephemeral-storage: 1Gi + atenetEgress.extProc.resources.requests.cpu: 50m + atenetEgress.drainSignal.sizeLimit: 1Mi + documentSelector: + path: kind + value: Deployment + asserts: + - equal: + path: spec.template.spec.containers[0].resources + value: + requests: + cpu: 100m + memory: 128Mi + ephemeral-storage: 32Mi + limits: + memory: 512Mi + ephemeral-storage: 1Gi + - equal: + path: spec.template.spec.containers[1].resources + value: + requests: + cpu: 50m + ephemeral-storage: 16Mi + limits: + ephemeral-storage: 256Mi + - contains: + path: spec.template.spec.volumes + content: + name: drain-signal + emptyDir: + sizeLimit: 1Mi + +- it: renders no resources and an unbounded emptyDir when the three knobs are null + set: + atenetEgress.resources: null + atenetEgress.extProc.resources: null + atenetEgress.drainSignal.sizeLimit: null + documentSelector: + path: kind + value: Deployment + asserts: + - notExists: + path: spec.template.spec.containers[0].resources + - notExists: + path: spec.template.spec.containers[1].resources + - contains: + path: spec.template.spec.volumes + content: + name: drain-signal + emptyDir: {} diff --git a/charts/substrate/values.yaml b/charts/substrate/values.yaml index 841d671b35..6902ae2110 100644 --- a/charts/substrate/values.yaml +++ b/charts/substrate/values.yaml @@ -172,6 +172,29 @@ atenetEgress: # nothing here. A change rolls the gateway: the roots are read at start. upstreamTrust: caBundle: "" + # The gateway container's resources (requests and limits, as the pod spec + # takes them). Only the ephemeral storage is bounded by default: the + # container writes nothing but its logs. CPU and memory are the operator's + # to size. Null renders no resources. + resources: + requests: + ephemeral-storage: 16Mi + limits: + ephemeral-storage: 256Mi + extProc: + # The ext-proc container's resources, the same way. It writes the drain + # marker (below) and nothing else. + resources: + requests: + ephemeral-storage: 16Mi + limits: + ephemeral-storage: 256Mi + # The emptyDir the ext-proc writes its drain marker to (/var/run/atenet): + # one small file, so its sizeLimit is tight. Null renders an unbounded + # emptyDir, which a cluster policy requiring bounded ephemeral storage + # (Kyverno's require-emptydir-requests-and-limits) then reports. + drainSignal: + sizeLimit: 16Mi # ate-api-server deployment overrides. Its snapshot-store client reaches the # same object store as atelet's: the ServiceAccount annotations carry the pod