Controlling mesh egress
Just as a waypoint can be used for traffic addressed to a service inside your cluster, a gateway can be used for traffic that leaves your cluster. To configure this, you use a ServiceEntry
resource, which is bound to a waypoint used for egress control.
Enable DNS capture
By default, a ServiceEntry does not have a stable service IP known to Istio. To enable egress, you will need to enable two features in Istio that are disabled by default.
If you followed the quick start guide, you can re-run istioctl:
$ istioctl install --set profile=ambient --skip-confirmation --set values.cni.ambient.dnsCapture=true --set values.pilot.env.PILOT_ENABLE_IP_AUTOALLOCATE=true
Create a test workload
In order to test your egress gateway, start a curl
pod:
$ kubectl apply -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml
You can run curl
from this pod in order to call external services.
$ kubectl exec deploy/curl -- curl -sI http://httpbin.org/get
HTTP/1.1 200 OK
Date: Tue, 29 Oct 2024 01:33:37 GMT
Content-Type: application/json
Content-Length: 254
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Querying the ztunnel logs, we see a connection directly to the external IP as we have not yet configured egress controls:
2024-10-29T02:10:57.056612Z info access connection complete src.addr=10.244.1.19:54882 src.workload="curl-759db5669d-dl5xw" src.namespace="default" dst.addr=54.84.141.5:80 direction="outbound" bytes_sent=78 bytes_recv=230 duration="616ms"
Create an egress gateway
By creating a single egress gateway, we can configure multiple ServiceEntries to control access to external sites and enforce policy.
$ kubectl create namespace common-infrastructure
$ kubectl label namespace common-infrastructure istio.io/dataplane-mode=ambient
$ istioctl waypoint apply --enroll-namespace --name egress-gateway --namespace common-infrastructure
By using --enroll-namespace
, all services (including ServiceEntries) in the common-infrastructure
namespace will use the egress-gateway
waypoint.
In that namespace, create a ServiceEntry
resource:
apiVersion: networking.istio.io/v1
kind: ServiceEntry
metadata:
name: httpbin.org
namespace: common-infrastructure
spec:
hosts:
- httpbin.org
ports:
- number: 80
name: http
protocol: HTTP
resolution: DNS
If you prefer to create the ServiceEntry in another namespace, you can label it for cross-namespace use
You can confirm that the ServiceEntry has a DNS entry set and is attached to the waypoint by querying the status field:
$ kubectl get serviceentry httpbin.org -n common-infrastructure -oyaml
Check for an address in the 240.240.x.x range:
status:
addresses:
- host: httpbin.org
value: 240.240.0.1
- host: httpbin.org
value: 2001:2::1
conditions:
- lastTransitionTime: "2024-10-27T00:45:08.374915129Z"
message: Successfully attached to waypoint common-infrastructure/waypoint
reason: WaypointAccepted
status: "True"
type: istio.io/WaypointBound
Repeat the curl
command from above, and you will see the response now comes from an istio-envoy
proxy:
$ kubectl exec deploy/curl -- curl -sI http://httpbin.org/get -v
* Host httpbin.org:80 was resolved.
* IPv6: 2001:2::2
* IPv4: 240.240.0.2
* Trying 240.240.0.2:80...
* Connected to httpbin.org (240.240.0.2) port 80
> GET /get HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< date: Tue, 29 Oct 2024 01:24:32 GMT
< content-type: application/json
< content-length: 254
< server: istio-envoy
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 208
< x-envoy-decorator-operation: httpbin.org:80/*
The ztunnel logs show the request was sent to our waypoint:
2024-10-29T01:24:32.582832Z info access connection complete src.addr=10.244.1.19:38994 src.workload="curl-759db5669d-dl5xw" src.namespace="default" src.identity="spiffe://cluster.local/ns/default/sa/curl" dst.addr=10.244.1.18:15008 dst.hbone_addr=240.240.0.4:80 dst.service="httpbin.org" dst.workload="egress-gateway-55767f4d4d-gkd5q" dst.namespace="common-infrastructure" dst.identity="spiffe://cluster.local/ns/common-infrastructure/sa/egress-gateway" direction="outbound" bytes_sent=78 bytes_recv=285 duration="404ms"
We can now also observe the traffic, with logs, traces, and metrics. For instance, if we enable access logging in our waypoint, we can see the request we just made:
│ [2024-10-29T01:24:32.608Z] "HEAD /get HTTP/1.1" 200 - via_upstream - "-" 0 0 392 392 "-" "curl/8.7.1" "b3c2eff1-b545-4dbf-b1e3-01dd203ded93" "httpbin.org" "34.228.248.173:80" inbound-vip|80|http|httpbin.org 10.244.1.18:49894 240.240.0.4:80 10.244.1.19:55866 - d │
Policy enforcement
One of the most powerful features of an egress gateway is the ability to enforce access control policies for all outbound traffic. These can range from simple domain allowlists, to complex policies such as data loss prevention.
In this example, we will limit low-privilege applications to only to the /get
endpoint, but allow a high-privilege application to access all of httpbin.org
. To do this, we apply an AuthorizationPolicy
attached to the ServiceEntry
we created.
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
name: httpbin
namespace: common-infrastructure
spec:
targetRefs:
- kind: ServiceEntry
group: networking.istio.io
name: httpbin.org
action: ALLOW
rules:
# Our admin application can access anything
- from:
- source:
principals: ["cluster.local/ns/default/sa/admin"]
# Everything else is only allowed to access /get
- to:
- operation:
methods: ["GET"]
paths: ["/get"]
From a low-privilege application, we can see our POST requests are denied:
$ curl httpbin.org/get
{
"headers": {
"Host": "httpbin.org",
},
"url": "http://httpbin.org/get"
}
$ curl -X POST httpbin.org/post
RBAC: access denied
From my admin application, the POST request succeeds as expected:
$ curl httpbin.org/post -X POST
{
"headers": {
"Host": "httpbin.org",
},
"url": "http://httpbin.org/post"
}
TLS origination
Another common policy to apply at an egress gateway is TLS origination - taking unencrypted internal HTTP connections, encrypting the requests, and then forwarding them to HTTPS servers. In the above examples, while all of our communication inside the mesh is automatically encrypted by Istio, once the request leaves the cluster it is in plaintext over the public internet – a potential security issue.
Fortunately, we can add TLS at the egress waypoint before the request leaves the cluster. This is especially useful when communicating with services that require client authentication, where access can be controlled in a centralized manner.
To upgrade to TLS requires only a small change to the ServiceEntry
and a DestinationRule
configuration:
apiVersion: networking.istio.io/v1
kind: ServiceEntry
metadata:
name: httpbin.org
namespace: common-infrastructure
spec:
hosts:
- httpbin.org
ports:
- number: 80
name: http
protocol: HTTP
targetPort: 443 # New: send traffic originally for port 80 to port 443
resolution: DNS
---
# New: add TLS to requests to `httpbin.org`
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
name: httpbin.org-tls
namespace: common-infrastructure
spec:
host: httpbin.org
trafficPolicy:
tls:
mode: SIMPLE
Now, all of your requests are automatically upgraded to HTTPS, without any changes to your application:
$ curl httpbin.org/get
{
"headers": {
"Host": "httpbin.org",
},
"url": "https://httpbin.org/get"
}
Cleaning up
Delete the ServiceEntry:
$ kubectl delete serviceentry httpbin.org -n common-infrastructure
Delete the waypoint and the common-infrastructure
namespace:
$ istioctl waypoint delete --namespace common-infrastructure egress-gateway
$ kubectl label namespace common-infrastructure istio.io/dataplane-mode-
$ kubectl delete namespace common-infrastructure