Increase Envoy dataplane efficiency by adding egress policy cache by yanavlasov · Pull Request #2200 · agent-substrate/substrate

yanavlasov · 2026-10-05T17:30:02Z

This PR implements egress policy cache in Envoy dataplane at CONNECT termination. It reduces number of callouts for policy retrievals from the same actor to the same destination port.

Cache for egress policy enforcement on the inner connection is in the followup PR.

Tests pass
Appropriate changes to documentation are included in the PR

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Lior Lieberman (LiorLieberman)

thx yan!

Lior Lieberman (LiorLieberman) · 2026-10-06T16:25:01Z

+An Envoy dynamic module, written in Rust, that runs as an HTTP filter on the
+egress gateway's outer `CONNECT` listener and caches egress policy SNI rules
+per actor certificate in a thread-local LRU cache so repeat `CONNECT` requests
+from the same actor can bypass the `ext_proc` sidecar.


can bypass the ext_proc sidecar

I think ext-proc side car is doing more things today (like simple authn), do we need that? Or we are saying if the egress policy is cached authn already happened? If yes, is that secure?

I am not sure we need that authn but just checking

I've updated readme with clarifications. Yes policy is cached after actor and destination port were authorized.

Lior Lieberman (LiorLieberman) · 2026-10-06T16:28:59Z

+per actor certificate in a thread-local LRU cache so repeat `CONNECT` requests
+from the same actor can bypass the `ext_proc` sidecar.
+
+## How it works


shuold we store only what we care about for that stage of the module vs caching the whole egress policy? e.g a small list of https_hostnames (meaning you mitm those) and tls_hostnames (meaning you passthrough). We dont need all cred injection here etc,

Right now policy is only SNIs and hostnames for the destination port. However I will add key injection as I will implement timed caching and local enforcement of inner policies, including key injection, in the followup. This should make the inner part more efficient as well and avoid callout per request.

not sure I follow, the policy has a bunch more things than sni and hostnames right? or you are saying we already store only sni and hostanems destination per port in the cache?

My comment was about "we should not cache the whole egresspolicy object, rather the just what we care about which is snis and hostnames per port"

ok, I checked the code cache already returns just the list of hosts for the port. So its good. I was confused by the readme that still say "Each CachedPolicy entry stores the serialized egress policy JSON " which reads like the whole EgressPolicy

Lior Lieberman (LiorLieberman) · 2026-10-06T16:30:40Z

+(`connection.sha256_peer_certificate_digest`) and `<destination_port>` is the
+destination port extracted from the `:authority` request header.
+
+### Request path (`on_request_headers`)


isnt this section too detailed? can we just replace with a visual?

I've edited it a bit. It seems like details are ok.

Lior Lieberman (LiorLieberman) · 2026-10-06T16:30:51Z

+     `shared_with_upstream: ONCE`, making it available to the inner listener's
+     `egress-policy` listener filter.
+
+### Response path (`on_response_headers`)


same comment. too detailed and easy to get stale

Shortened it a bit.

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Lior Lieberman (LiorLieberman)

generally lgtm, a few more comments. also - do we have any small benchmarks that lets us compare how this performs vs without this change? so we know what we gain right away?

Lior Lieberman (LiorLieberman) · 2026-10-07T20:43:26Z

nit but I still think thats really too detailed for a readme and it will get stale pretty quickly. is it more a developing.md more? or contributing.md?

What would you like me to do?

Lior Lieberman (LiorLieberman) · 2026-10-07T20:46:44Z

+// TestActorEgressPolicyCachePerPort sends 5 requests to http://example.net and
+// then 5 requests to https://example.com, verifying that per-port policy
+// caching records 2 cache misses, 8 cache hits, and 2 outer CONNECT ext_proc
+// callouts.


nit: Probably too verbose - should we just say it tests caching logic per port? or some one meaningful sentence? If someone changes it without an agent, this comment easily going to be misleading

Please suggest how to improve this.

Lior Lieberman (LiorLieberman) · 2026-10-07T20:47:04Z

+// TestActorEgressDialedPortOverridesHostHeader sends requests to
+// https://example.com/ (default TCP port 443) with "example.com:1" in the Host
+// header and to http://example.net/ (default TCP port 80) with "example.net:2"
+// in the Host header, verifying that the egress gateway dials the TCP port the
+// actor connected to rather than the port from the Host header.
+// Sending 2 requests per target verifies that cached policy correctly sets the
+// destination port filter state.


Same comment as above for all function comments

Please suggest how to improve this.

Lior Lieberman (LiorLieberman) · 2026-10-07T20:49:50Z

+    # E2E_ENVOY_STATS_FLUSH_ON_ADMIN is substituted in renderAtenetEgressManifest
+    stats_flush_on_admin: ${E2E_ENVOY_STATS_FLUSH_ON_ADMIN}


This is for e2e, is there a way we dont pollute actual "production" config files with some e2e vars?

I'm open to suggestions here. This needs to be in the bootstrap config. The options are either to have a dedicated config for e2e, use format substitution. This can not even be solved by xDS since this is bootstrap option.

Just dug a bit about this, and I do not think Kustomize can change an specific value in Envoy yaml in the ConfigMap. Unless I duplicate the entire Envoy config in the Kustomize patch. But then I would need to keep them in sync which is error prone.

If I'm wrong, please let me know.

I think we dont need stats_flush_on_admin. IIUC Flush-on-admin only pushes to stats sinks and this config has no sinks. So we can remove it if thats correct

For concurrency, its a deployment level config so Kustomize can do.

# manifests/ate-install/components/egress-envoy-single-worker/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1alpha1 kind: Component patches: - target: {group: apps, version: v1, kind: Deployment, name: atenet-egress} patch: |- - op: add path: /spec/template/spec/containers/0/args/- value: --concurrency - op: add path: /spec/template/spec/containers/0/args/- value: "1" // OR take is from the env var

Lior Lieberman (LiorLieberman) · 2026-10-07T20:52:35Z

+per actor certificate in a thread-local LRU cache so repeat `CONNECT` requests
+from the same actor can bypass the `ext_proc` sidecar.
+
+## How it works


not sure I follow, the policy has a bunch more things than sni and hostnames right? or you are saying we already store only sni and hostanems destination per port in the cache?

My comment was about "we should not cache the whole egresspolicy object, rather the just what we care about which is snis and hostnames per port"

yanavlasov · 2026-10-07T21:33:18Z

generally lgtm, a few more comments. also - do we have any small benchmarks that lets us compare how this performs vs without this change? so we know what we gain right away?

We have before benchmark results. Yufan will run benchmark after this PR is submitted.

yanavlasov · 2026-10-08T01:58:57Z

not sure I follow, the policy has a bunch more things than sni and hostnames right? or you are saying we already store only sni and hostanems destination per port in the cache?
My comment was about "we should not cache the whole egresspolicy object, rather the just what we care about which is snis and hostnames per port"

Envoy does not cache the whole policy at this point. Only what is needed for the listener filter in the inner filter chain to decide if this MITM, passthrough or plain text traffic.

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Lior Lieberman (LiorLieberman) · 2026-10-08T16:16:30Z

+Envoy worker thread). Each `CachedPolicy` entry stores the serialized egress
+policy JSON string alongside the `Instant` (`stored_at`) when it was cached,


Suggested change

Envoy worker thread). Each `CachedPolicy` entry stores the serialized egress

policy JSON string alongside the `Instant` (`stored_at`) when it was cached,

Envoy worker thread). Each `CachedPolicy` entry stores the SNI rules ext_proc

returned for that port (the hostnames of its `https` and `tls_passthrough`

rules, marked `mitm` or `passthrough`) and when they were cached,

Lior Lieberman (LiorLieberman) · 2026-10-08T16:23:04Z

+// TestActorEgressPolicyCache sends 5 requests to example.com and checks that
+// the egress policy cache counters record 4 hits and 1 miss, and that the
+// outer CONNECT ext_proc filter is invoked only once.


Suggested change

// TestActorEgressPolicyCache sends 5 requests to example.com and checks that

// the egress policy cache counters record 4 hits and 1 miss, and that the

// outer CONNECT ext_proc filter is invoked only once.

// TestActorEgressPolicyCache checks that repeat CONNECTs from one actor to the

// same port skip the ext_proc callout.

Lior Lieberman (LiorLieberman) · 2026-10-08T16:23:21Z

+// TestActorEgressPolicyCachePerPort sends 5 requests to http://example.net and
+// then 5 requests to https://example.com, verifying that per-port policy
+// caching records 2 cache misses, 8 cache hits, and 2 outer CONNECT ext_proc
+// callouts.


Suggested change

// TestActorEgressPolicyCachePerPort sends 5 requests to http://example.net and

// then 5 requests to https://example.com, verifying that per-port policy

// caching records 2 cache misses, 8 cache hits, and 2 outer CONNECT ext_proc

// callouts.

// TestActorEgressPolicyCachePerPort checks that the cache is keyed by

// destination port.

Lior Lieberman (LiorLieberman) · 2026-10-08T16:23:37Z

+// TestActorEgressDialedPortOverridesHostHeader sends requests to
+// https://example.com/ (default TCP port 443) with "example.com:1" in the Host
+// header and to http://example.net/ (default TCP port 80) with "example.net:2"
+// in the Host header, verifying that the egress gateway dials the TCP port the
+// actor connected to rather than the port from the Host header.
+// Sending 2 requests per target verifies that cached policy correctly sets the
+// destination port filter state.


Suggested change

// TestActorEgressDialedPortOverridesHostHeader sends requests to

// https://example.com/ (default TCP port 443) with "example.com:1" in the Host

// header and to http://example.net/ (default TCP port 80) with "example.net:2"

// in the Host header, verifying that the egress gateway dials the TCP port the

// actor connected to rather than the port from the Host header.

// Sending 2 requests per target verifies that cached policy correctly sets the

// destination port filter state.

// TestActorEgressDialedPortOverridesHostHeader checks that the gateway dials

// the port the actor connected to, not the Host header's port, on both a cache

// miss and a hit.

yanavlasov added 15 commits October 3, 2026 12:24

Egress Policy Cache

aed40d3

Signed-off-by: Yan Avlasov <yavlasov@google.com>

WiP

335fc42

Signed-off-by: Yan Avlasov <yavlasov@google.com>

WiP

a065555

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Skip ext_proc

7b3bc73

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Implement cache freshness

e449837

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Implement cache enabling flag

94c50b9

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Fixes

22caefb

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Docs

e9b2688

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Skip tests if concurrency is not set

d8e057c

Signed-off-by: Yan Avlasov <yavlasov@google.com>

cleanup

12786f8

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Add dest port to cache key

8cb7351

Signed-off-by: Yan Avlasov <yavlasov@google.com>

fix tests

7fed564

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Merge branch 'main' into egress-policy-cache

9045bce

Merge branch 'main' into egress-policy-cache

de458b1

Add per port policy cache test

251e3e7

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Lior Lieberman (LiorLieberman) self-requested a review October 6, 2026 16:16

Lior Lieberman (LiorLieberman) reviewed Oct 6, 2026

View reviewed changes

Comment thread .github/workflows/pr-workflow.yaml

Lior Lieberman (LiorLieberman) reviewed Oct 6, 2026

View reviewed changes

yanavlasov added 11 commits October 6, 2026 19:29

Add port setting metadata

9888426

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Merge branch 'main' into egress-policy-cache

448e6f5

Add dest port validation

426b1e2

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Update doc

56f44c1

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Use dedicated value to skip ext_proc

caeb1f7

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Massage doc

2839ea6

Signed-off-by: Yan Avlasov <yavlasov@google.com>

fix tests

2275f43

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Revert egressmitm_test.go

74eb937

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Make stats flushing immediate in E2E tests

2bcd4ca

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Merge branch 'main' into egress-policy-cache

5e5e953

Update doc

cf76224

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Lior Lieberman (LiorLieberman) reviewed Oct 7, 2026

View reviewed changes

Update comment

00cea33

Signed-off-by: Yan Avlasov <yavlasov@google.com>

Lior Lieberman (LiorLieberman) reviewed Oct 8, 2026

View reviewed changes

yanavlasov mentioned this pull request Oct 10, 2026

Reduce number of egress atunnels by multiplexing CONNECTs over H/2 #2425

Open

2 tasks done

		# E2E_ENVOY_STATS_FLUSH_ON_ADMIN is substituted in renderAtenetEgressManifest
		stats_flush_on_admin: ${E2E_ENVOY_STATS_FLUSH_ON_ADMIN}

		Envoy worker thread). Each `CachedPolicy` entry stores the serialized egress
		policy JSON string alongside the `Instant` (`stored_at`) when it was cached,

-Envoy worker thread). Each `CachedPolicy` entry stores the serialized egress
-policy JSON string alongside the `Instant` (`stored_at`) when it was cached,
+Envoy worker thread). Each `CachedPolicy` entry stores the SNI rules ext_proc
+returned for that port (the hostnames of its `https` and `tls_passthrough`
+rules, marked `mitm` or `passthrough`) and when they were cached,

Conversation

yanavlasov commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Lior Lieberman (LiorLieberman) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Lior Lieberman (LiorLieberman) left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Lior Lieberman (LiorLieberman) Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

yanavlasov commented Oct 7, 2026

Uh oh!

yanavlasov commented Oct 8, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

yanavlasov commented Oct 5, 2026 •

edited

Loading

Lior Lieberman (LiorLieberman) Oct 8, 2026 •

edited

Loading