Mirroring

Mirror live traffic to a shadow service in ambient mesh to safely test changes in production without affecting the primary request path.

Traffic mirroring, also called shadowing, lets you test changes in production with minimal risk. Mirroring sends a copy of live traffic to a shadow service asynchronously, outside the primary request path, so the primary service is unaffected. In this guide, you deploy two versions of a service, route all traffic to v1, then configure mirroring so that v2 receives a copy of every request.

Before you begin

Install an ambient mesh.

Add the default namespace to the ambient mesh.

kubectl label namespace default istio.io/dataplane-mode=ambient

Deploy a waypoint proxy in the default namespace. The HTTPRoute in this guide attaches to a Service as a parentRef, which requires a waypoint to enforce L7 rules.
```
istioctl waypoint apply --enroll-namespace --wait
```

Deploy sample apps

Deploy the following sample services to your cluster.

Deploy httpbin-v1.

kubectl create -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin-v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpbin
      version: v1
  template:
    metadata:
      labels:
        app: httpbin
        version: v1
    spec:
      containers:
      - image: docker.io/kennethreitz/httpbin
        imagePullPolicy: IfNotPresent
        name: httpbin
        command: ["gunicorn", "--access-logfile", "-", "-b", "0.0.0.0:80", "httpbin:app"]
        ports:
        - containerPort: 80
EOF

Deploy httpbin-v2, which will receive the mirrored traffic.

kubectl create -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin-v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpbin
      version: v2
  template:
    metadata:
      labels:
        app: httpbin
        version: v2
    spec:
      containers:
      - image: docker.io/kennethreitz/httpbin
        imagePullPolicy: IfNotPresent
        name: httpbin
        command: ["gunicorn", "--access-logfile", "-", "-b", "0.0.0.0:80", "httpbin:app"]
        ports:
        - containerPort: 80
EOF

Deploy the httpbin service.

kubectl create -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: httpbin
  labels:
    app: httpbin
spec:
  ports:
  - name: http
    port: 8000
    targetPort: 80
  selector:
    app: httpbin
EOF

Deploy a curl client to send test requests.

kubectl create -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: curl
spec:
  replicas: 1
  selector:
    matchLabels:
      app: curl
  template:
    metadata:
      labels:
        app: curl
    spec:
      containers:
      - name: curl
        image: curlimages/curl
        command: ["/bin/sleep","3650d"]
        imagePullPolicy: IfNotPresent
EOF

Create a default routing policy

By default, Kubernetes load balances across both versions of the httpbin service. Create a routing policy to send all traffic to v1.

Create version-specific services and an HTTPRoute that routes all traffic to httpbin-v1.

kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: httpbin-v1
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: httpbin
    version: v1
---
apiVersion: v1
kind: Service
metadata:
  name: httpbin-v2
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: httpbin
    version: v2
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin
spec:
  parentRefs:
  - group: ""
    kind: Service
    name: httpbin
    port: 8000
  rules:
  - backendRefs:
    - name: httpbin-v1
      port: 80
EOF

Send a request to verify that all traffic reaches httpbin-v1.

kubectl exec deploy/curl -- curl -sS http://httpbin:8000/headers

Example output:

{
  "headers": {
    "Accept": "*/*", 
    "Host": "httpbin:8000", 
    "User-Agent": "curl/8.7.1"
  }
}

Check the logs to confirm that httpbin-v1 receives requests and httpbin-v2 receives none.

kubectl logs deploy/httpbin-v1 -c httpbin

Example output:

10.244.1.36 - - [29/Oct/2024:08:59:16 +0000] "GET /headers HTTP/1.1" 200 105 "-" "curl/8.7.1"

kubectl logs deploy/httpbin-v2 -c httpbin

Example output:

<none>

Mirror traffic to httpbin-v2

Update the HTTPRoute to mirror traffic to httpbin-v2.

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin
spec:
  parentRefs:
  - group: ""
    kind: Service
    name: httpbin
    port: 8000
  rules:
  - filters:
    - type: RequestMirror
      requestMirror:
        backendRef:
          name: httpbin-v2
          port: 80
    backendRefs:
    - name: httpbin-v1
      port: 80
EOF

The RequestMirror filter sends 100% of traffic to httpbin-v1 as the primary backend, while also sending a copy to httpbin-v2. Mirrored requests have their Host/Authority headers appended with -shadow and are fire-and-forget — responses from the mirror are discarded.

Send a test request.

kubectl exec deploy/curl -- curl -sS http://httpbin:8000/headers

Check the logs for both deployments. Both v1 and v2 should now show access log entries. The entries in v2 are mirrored copies of the requests served by v1.

kubectl logs deploy/httpbin-v1 -c httpbin

Example output:

127.0.0.1 - - [07/Mar/2018:19:02:43 +0000] "GET /headers HTTP/1.1" 200 321 "-" "curl/7.35.0"
127.0.0.1 - - [07/Mar/2018:19:26:44 +0000] "GET /headers HTTP/1.1" 200 321 "-" "curl/7.35.0"

kubectl logs deploy/httpbin-v2 -c httpbin

Example output:

127.0.0.1 - - [07/Mar/2018:19:26:44 +0000] "GET /headers HTTP/1.1" 200 361 "-" "curl/7.35.0"

Cleanup

Delete the sample resources.

kubectl delete httproute httpbin
kubectl delete svc httpbin httpbin-v1 httpbin-v2
kubectl delete deploy httpbin-v1 httpbin-v2 curl

Traffic splitting Redirects and rewrites