Assistance Needed: Customizing Envoy Proxy Image

[lite31]

Hey,

Could you please advise on how to customize the Envoy proxy image (by default is: docker.io/envoyproxy/envoy:distroless-v1.33.0)?

I've reviewed the CRDs and templates in the Envoy Gateway Helm chart (repo link), but I couldn't find any reference or configuration option for specifying a custom image. This is necessary when deploying a Gateway instance.

I also followed the documentation on customizing the Envoy proxy image (guide link), but I was unsuccessful.

my kubernetes resource files

eg-gatewayclass.yaml

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller

eg-custom.yaml

• Using a locally available image: docker.io/library/envoy:distroless-v1.33.0 with imagePullPolicy: Never)
• Additionally, I have configured this in three namespaces—default, iam, and envoy-gateway-system—out of caution due to my uncertainties

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: custom-proxy-config
  namespace: default
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyDeployment:
        container:
          image: "docker.io/library/envoy:distroless-v1.33.0"

iam-gateway.yaml

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: iam-gateway
  labels:
    app: iam
    blueprintpart: gatewayApiV1 # kpt-set: ${GATEWAY_API_V1_LABEL}
  namespace: iam
spec:
  gatewayClassName: eg
  # infrastructure:
  #   parametersRef:
  #     group: gateway.envoyproxy.io
  #     kind: EnvoyProxy
  #     name: custom-proxy-config
  listeners:
    - name: https
      protocol: HTTPS
      port: 443
      tls:
        mode: Terminate
        certificateRefs:
          - group: ""
            kind: Secret
            name: iam-tls-secret # kpt-set: ${GATEWAY_API_V1_SECRET_NAME_TLS}
    - name: ldaps
      protocol: TLS
      port: 636
      tls:
        mode: Terminate
        certificateRefs:
          - group: ""
            kind: Secret
            name: iam-tls-secret # kpt-set: ${GATEWAY_API_V1_SECRET_NAME_TLS}

Observations

• In environments where the default Envoy image (docker.io/envoyproxy/envoy:distroless-v1.33.0) is accessible, the Kubernetes deployment appears, and I can patch it if needed. The controller and deployment are created in the envoy-gateway-system namespace by default.
• In environments where the default image cannot be pulled, the deployment is missing from Kubernetes events, and a Gateway is not configured.

Could someone provide the exact steps needed to successfully customize the Envoy proxy container image? Any guidance would be greatly appreciated!

The documentation states, "After applying the config, you can get the deployment image and see that it has changed." But I wish to pass the deployment's image before the deployment is created, how to do that?

Thanks!

PS. Envoy Gateway (https://gateway.envoyproxy.io/docs/install/install-yaml/#install-with-yaml) v1.3.0 is used.

[zirain]

there's a sample yaml here:

gateway/examples/kubernetes/envoy-proxy-config.yaml

Line 9 in d3110e3

kind: EnvoyProxy

[lite31]

• This

  gateway/examples/kubernetes/envoy-proxy-config.yaml

  Line 9 in d3110e3

  kind: EnvoyProxy

  
  works (when a registry is used, not with locally available images)
• This https://gateway.envoyproxy.io/docs/tasks/operations/customize-envoyproxy/#customize-envoyproxy-image does not work (neither when a registry is used nor with locally available images)

So, the questions are:

• kubectl get events is not showing the failed deployments, nor the controller's logs, how can I possibly troubleshoot this?
• how to specify the imagePullPolicy that I assume can be the cause (although am not sure of it yet)?

[zirain]

kubelet should not pull image if it exists on node when image pull policy is IfNotPresent.
this's a kubernetes/kubelet problem, rather than eg.

[lite31]

The issue is not whether the image is pulled but whether kubelet knows where to look—specifically, whether it should search for docker.io.

To prevent kubelet from looking for docker.io, you must explicitly set imagePullPolicy: Never. There is no alternative.

If envoy-gateway cannot run in an offline environment (i.e., without access to an image registry) because it does not expose an imagePullPolicy configuration option, then this is a concern that needs to be addressed by eg.

[arkodg]

can you use the patch field for your use case https://gateway.envoyproxy.io/docs/tasks/operations/customize-envoyproxy/#patching-deployment-for-envoyproxy

[lite31]

Thank you, I completely overlooked that. I'll try this out when I get a chance (maybe not today, though).

Would you mind advising me on how to debug the deployment created by the controller? Right now, I have no visibility into it—no logs, no events, or perhaps I'm just looking in the wrong places?

[lite31]

Thank you both for the prompt replies and support. The patch config knob did the trick ;)

[arkodg]

the deployment is in the envoy-gateway-system namespace

[sachinshaji]

Hello All,
I have copied the below line from @nskalis comment.

If envoy-gateway cannot run in an offline environment (i.e., without access to an image registry) because it does not expose an imagePullPolicy configuration option, then this is a concern that needs to be addressed by eg.

We have the same situation where we are pulling the image from private image registry(company policy). How can I use 'imagePullPolicy' and 'secrets' to pull the image?

[lite31]

Hey, as mentioned see https://gateway.envoyproxy.io/docs/tasks/operations/customize-envoyproxy/#patching-deployment-for-envoyproxy, this is how it can be customised.

[sachinshaji]

Yeah it works :)
Thanks @nskalis