Documentation
Traffic management
Controlling mesh egress

Controlling mesh egress

Just as a waypoint can be used for traffic addressed to a service inside your cluster, a gateway can be used for traffic that leaves your cluster. To configure this, you use a ServiceEntry resource, which is bound to a waypoint used for egress control.

Enable DNS capture

By default, a ServiceEntry does not have a stable service IP known to Istio. To enable egress, you will need to enable two features in Istio that are disabled by default.

  • If you followed the quickstart guide, DNS capture is already configured.
  • If you are using Helm, add --set cni.ambient.dnsCapture=true --set pilot.env.PILOT_ENABLE_IP_AUTOALLOCATE=true to your installation commands (or the equivalent in your values.yaml file.)

Create a test workload

In order to test your egress gateway, start a curl pod:

$ kubectl apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml

You can run curl from this pod in order to call external services.

$ kubectl exec deploy/curl -- curl -sI http://httpbin.org/get
HTTP/1.1 200 OK
Date: Tue, 29 Oct 2024 01:33:37 GMT
Content-Type: application/json
Content-Length: 254
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

Querying the ztunnel logs, we see a connection directly to the external IP as we have not yet configured egress controls:

2024-10-29T02:10:57.056612Z	info	access	connection complete	src.addr=10.244.1.19:54882 src.workload="curl-759db5669d-dl5xw" src.namespace="default" dst.addr=54.84.141.5:80 direction="outbound" bytes_sent=78 bytes_recv=230 duration="616ms"

Create an egress gateway

By creating a single egress gateway, we can configure multiple ServiceEntries to control access to external sites and enforce policy.

$ kubectl create namespace common-infrastructure
$ kubectl label namespace common-infrastructure istio.io/dataplane-mode=ambient
$ istioctl waypoint apply --enroll-namespace --name egress-gateway --namespace common-infrastructure

By using --enroll-namespace, all services (including ServiceEntries) in the common-infrastructure namespace will use the egress-gateway waypoint.

In that namespace, create a ServiceEntry resource:

$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1
kind: ServiceEntry
metadata:
  name: httpbin.org
  namespace: common-infrastructure
spec:
  hosts:
  - httpbin.org
  ports:
  - number: 80
    name: http
    protocol: HTTP
  resolution: DNS
EOF

If you prefer to create the ServiceEntry in another namespace, you can label it for cross-namespace use

You can confirm that the ServiceEntry has a DNS entry set and is attached to the waypoint by querying the status field:

$ kubectl get serviceentry httpbin.org -n common-infrastructure -oyaml

Check for an address in the 240.240.x.x range:

  status:
    addresses:
    - host: httpbin.org
      value: 240.240.0.1
    - host: httpbin.org
      value: 2001:2::1
    conditions:
    - lastTransitionTime: "2024-10-27T00:45:08.374915129Z"
      message: Successfully attached to waypoint common-infrastructure/waypoint
      reason: WaypointAccepted
      status: "True"
      type: istio.io/WaypointBound

Repeat the curl command from above, and you will see the response now comes from an istio-envoy proxy:

$ kubectl exec deploy/curl -- curl -sI http://httpbin.org/get -v
* Host httpbin.org:80 was resolved.
* IPv6: 2001:2::2
* IPv4: 240.240.0.2
*   Trying 240.240.0.2:80...
* Connected to httpbin.org (240.240.0.2) port 80
> GET /get HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/1.1 200 OK
< date: Tue, 29 Oct 2024 01:24:32 GMT
< content-type: application/json
< content-length: 254
< server: istio-envoy
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 208
< x-envoy-decorator-operation: httpbin.org:80/*

The ztunnel logs show the request was sent to our waypoint:

2024-10-29T01:24:32.582832Z	info	access	connection complete	src.addr=10.244.1.19:38994 src.workload="curl-759db5669d-dl5xw" src.namespace="default" src.identity="spiffe://cluster.local/ns/default/sa/curl" dst.addr=10.244.1.18:15008 dst.hbone_addr=240.240.0.4:80 dst.service="httpbin.org" dst.workload="egress-gateway-55767f4d4d-gkd5q" dst.namespace="common-infrastructure" dst.identity="spiffe://cluster.local/ns/common-infrastructure/sa/egress-gateway" direction="outbound" bytes_sent=78 bytes_recv=285 duration="404ms"

We can now also observe the traffic, with logs, traces, and metrics. For instance, if we enable access logging in our waypoint, we can see the request we just made:

│ [2024-10-29T01:24:32.608Z] "HEAD /get HTTP/1.1" 200 - via_upstream - "-" 0 0 392 392 "-" "curl/8.7.1" "b3c2eff1-b545-4dbf-b1e3-01dd203ded93" "httpbin.org" "34.228.248.173:80" inbound-vip|80|http|httpbin.org 10.244.1.18:49894 240.240.0.4:80 10.244.1.19:55866 - d │

Policy enforcement

One of the most powerful features of an egress gateway is the ability to enforce access control policies for all outbound traffic. These can range from simple domain allowlists, to complex policies such as data loss prevention.

In this example, we will limit low-privilege applications to only to the /get endpoint, but allow a high-privilege application to access all of httpbin.org. To do this, we apply an AuthorizationPolicy attached to the ServiceEntry we created.

$ kubectl apply -f - <<EOF
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: httpbin
  namespace: common-infrastructure
spec:
  targetRefs:
  - kind: ServiceEntry
    group: networking.istio.io
    name: httpbin.org
  action: ALLOW
  rules:
  # Our admin application can access anything
  - from:
    - source:
        principals: ["cluster.local/ns/default/sa/admin"]
  # Everything else is only allowed to access /get
  - to:
    - operation:
        methods: ["GET"]
        paths: ["/get"]
EOF

From a low-privilege application, we can see our POST requests are denied:

$ curl httpbin.org/get
{
  "headers": {
    "Host": "httpbin.org",
  },
  "url": "http://httpbin.org/get"
}
$ curl -X POST httpbin.org/post
RBAC: access denied

From my admin application, the POST request succeeds as expected:

$ curl httpbin.org/post -X POST
{
  "headers": {
    "Host": "httpbin.org",
  },
  "url": "http://httpbin.org/post"
}

TLS origination

Another common policy to apply at an egress gateway is TLS origination - taking unencrypted internal HTTP connections, encrypting the requests, and then forwarding them to HTTPS servers. In the above examples, while all of our communication inside the mesh is automatically encrypted by Istio, once the request leaves the cluster it is in plaintext over the public internet – a potential security issue.

Fortunately, we can add TLS at the egress waypoint before the request leaves the cluster. This is especially useful when communicating with services that require client authentication, where access can be controlled in a centralized manner.

To upgrade to TLS requires only a small change to the ServiceEntry and a DestinationRule configuration:

$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1
kind: ServiceEntry
metadata:
  name: httpbin.org
  namespace: common-infrastructure
spec:
  hosts:
  - httpbin.org
  ports:
  - number: 80
    name: http
    protocol: HTTP
    targetPort: 443 # New: send traffic originally for port 80 to port 443
  resolution: DNS
---
# New: add TLS to requests to `httpbin.org`
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: httpbin.org-tls
  namespace: common-infrastructure
spec:
  host: httpbin.org
  trafficPolicy:
    tls:
      mode: SIMPLE
EOF

Now, all of your requests are automatically upgraded to HTTPS, without any changes to your application:

$ curl httpbin.org/get
{
  "headers": {
    "Host": "httpbin.org",
  },
  "url": "https://httpbin.org/get"
}

Cleaning up

Delete the ServiceEntry:

$ kubectl delete serviceentry httpbin.org -n common-infrastructure

Delete the waypoint and the common-infrastructure namespace:

$ istioctl waypoint delete --namespace common-infrastructure egress-gateway
$ kubectl label namespace common-infrastructure istio.io/dataplane-mode-
$ kubectl delete namespace common-infrastructure
Redirects and rewritesIntegrate third-party Gateway and Ingress controllers