Traffic management in ambient mode

Once you have services enrolled in a mesh, you can use Istio’s traffic management features to perform countless useful functions — A/B testing, canary rollouts, reliability management; the list goes on. As with sidecar mode, ambient mode involves the use of gateways (proxies to get traffic into and out of the mesh) and introduces waypoints (a type of gateway used to manage traffic between services, which replaces the function of the sidecar directly).

Configuration

There are gateways, and then there are Gateways

Sometimes the English language fails us in spectacular ways. On behalf of the Kubernetes and Istio communities, I apologise in advance for the overloading of the word “Gateway” that I am about to inflict on you.

Here’s how it plays out:

• A gateway is a load balancer (implemented by a layer 7 proxy) in a service mesh.
• There are four main types of gateway in an ambient mesh:
  • North-South gateways, which take traffic from outside a mesh and send it to internal destinations
    • Ingress gateways are north-south gateways that take traffic from outside a cluster and bring it into the mesh.
    • Egress gateways are north-south gateways that can be used to manage traffic leaving a mesh.
  • Waypoints, which sit between services within a cluster
  • East-West gateways, which sit between clusters to build a multi-cluster mesh

We can see how these gateways are used with some conceptual examples. This is a 10,000 foot overview, which doesn’t include nodes, ztunnels or the Kubernetes Services that underpin this infrastructure.

• Pods A and B are both configured to use their own waypoints. Traffic from clients inside the cluster will be routed through these waypoints.
• Pod A is available from the internet via an ingress gateway. Traffic from an ingress gateway does not traverse a configured waypoint unless specifically configured to do so (as discussed below).
• Pod B is available from the internet via an ingress gateway, but is configured so that traffic goes through its configured waypoint.
• Pod C is in cluster 2. Traffic to it from sources in cluster 1 will pass through cluster 2’s east-west gateway¹.
• Pod C is configured to use an egress gateway to make connections to services outside the mesh.

In the past

Ingress and egress gateways exist in both sidecar mode and ambient mode. Traditionally, deploying the infrastructure (the proxy servers and load balancers required to expose them) was done by the cluster operator using Helm charts. The behaviour of a north-south gateway is configured using an API type called Gateway. This API lets you define hostnames, SSL certificates, etc., for existing deployments and services. (For disambiguation, we may refer to this type of Gateway as an “Istio Gateway”.)

Configuration of routing through an Istio Gateway, or the mesh of sidecars, is via VirtualServices.

Introducing Gateway API

Gateway API took learnings from various Kubernetes ingress implementations, including Istio, and built a standardized, vendor-neutral API. These APIs generally serve the same purposes as Istio Gateway and VirtualService, with a few key differences:

• In Istio APIs, a Gateway configures an existing gateway Deployment/Service that has been deployed. In Gateway API, the Gateway resource (same name, different API group) both configures and deploys a gateway. (For disambiguation, this type of Gateway may be called a “Kubernetes Gateway”.)

• In the Istio VirtualService, all protocols are configured within a single resource. In the Gateway APIs, each protocol type has its own resource, such as HTTPRoute and TCPRoute.

In sidecar mode, you can configure services using either API — but not both — and the two are generally compatible. Because the Istio APIs have been around since 2018, while Gateway API only went to Beta in 2022, it’s far more likely that an existing Istio user is using the VirtualService APIs.

In ambient mode

Because of the opportunity afforded by a major change to Istio, the maintainers decided that Gateway API would be the sole API supported for waypoints within an ambient mesh. Instead of using VirtualService, you should use HTTPRoute (and its siblings). There are some exceptions, which are detailed below.

Istio gateways can still be configured using the VirtualService API, but we recommend that you consider migrating to Kubernetes gateways so you do not have to constantly think about the differences between ingress and waypoint routes².

Egress gateways in ambient mesh are, in effect, waypoints for external services.

Gateway API allows you to deploy all four types of gateway using a resource also named Gateway. You configure traffic routing through those gateways using the HTTPRoute resource.

It’s time for a new API!

We recommend that everyone starting fresh with ambient mesh use only the Gateway API. While migrating to ambient mode, you should seriously consider if you can make this upgrade.

Gateway API is a “lowest common denominator” API, and some features of Istio cannot yet be described using it. The legacy VirtualService API is still required for some tasks when configuring routing through waypoints. You can mix the two within the mesh, but not for the same host. If you have a VirtualService defined for a hostname and you create one or more HTTPRoutes, the VirtualService will be overwritten and its rules will not be implemented.

Any use of VirtualService with waypoints is currently marked as Alpha in upstream Istio; that effectively means that the maintainers believe it works, but haven’t stringently tested it.

Changes required when moving from VirtualService to Gateway API

Some of the features of Istio cannot be expressed in Gateway API. This may be because the work to add the required fields has not yet been done, or because the goal of the API is to provide a generic abstraction and the feature is Envoy-specific.

Subsets and services

In Istio’s traditional traffic management paradigm, the recommended approach is to use the subset field in a Destination Rule to address traffic to a subset of a service — where the selector is the same (app: reviews) but the matching labels are more granular (version: v3).

The canonical example for the Bookinfo service would look like this:

apiVersion: v1
kind: Service
metadata:
  name: reviews
  labels:
    app: reviews
    service: reviews
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: reviews
---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: reviews
spec:
  host: reviews
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
  - name: v3
    labels:
      version: v3
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 50
    - destination:
        host: reviews
        subset: v3
      weight: 50

What we have here is a single Service which selects all versions of the reviews workload, and a VirtualService used to narrow down to the subset that you want to target, under various circumstances. This example will route 50% of traffic to v1 and 50% to v3, which you might use as a part of an A/B test or a rollout of a new version.

Instead of using a subset, you could define two extra Services, which are more granular in their selection of workloads:

apiVersion: v1
kind: Service
metadata:
  name: reviews-v1
  labels:
    app: reviews
    service: reviews
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: reviews
    version: v1
---
apiVersion: v1
kind: Service
metadata:
  name: reviews-v3
  labels:
    app: reviews
    service: reviews
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: reviews
    version: v3
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: reviews-route-two-versions
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews-v1
      weight: 50
    - destination:
        host: reviews-v3
      weight: 50

This is exactly how subset addressing is performed in the Gateway API:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: reviews-route-two-versions
spec:
  parentRefs:
  - group: ""
    kind: Service
    name: reviews
    port: 9080
  rules:
  - backendRefs:
    - name: reviews-v1
      port: 9080
      weight: 50
    - name: reviews-v3
      port: 9080
      weight: 50

Syntactical differences aside, you can see the similarities between the two routes: both take traffic targeted at reviews and send half each to the reviews-v1 and reviews-v3 services.

Fault injection

Istio supports fault injection using the VirtualService API. Support hasn’t yet been added to Gateway API, but it is possible to use a VirtualService to use this feature.

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: ratings
spec:
  hosts:
  - ratings
  http:
  - fault:
      delay:
        fixedDelay: 4s
        percentage:
          value: 100.0
    route:
    - destination:
        host: ratings

Remember that any time you use a VirtualService, you cannot also use a HTTPRoute for the same hostname.

TCP and TLS routing

HTTP and gRPC routing is generally available in the Gateway API, which means the HTTPRoute and GRPCRoute objects are installed in the default (“standard”) channel. TLS and TCP routing are in the so-called “experimental” channel, which is implemented in additional Custom Resources and must be installed separately. Because they map to well-tested behaviour in Istio, we are confident that the features work well. What we can’t be sure of is that the APIs won’t change in future.

HTTP retries

Gateway API has experimental support for counted HTTP retries and retry budgets. VirtualService exposes Envoy’s feature set, including more complex conditions and retrying to different localities.

Other features

Some features of VirtualService are naturally no longer required due to HTTPRoute being a separate object:

• VirtualService allows you to filter the workloads that a route matches using sourceLabels. With Gateway API, you create a service scoped to just those labels.
• You can delegate a HTTP route in a VirtualService, which was most commonly used to allow merging VirtualServices from different namespaces. Waypoints can allow routes from different namespaces, but other delegation tasks are not possible in Gateway API.

Other gaps still exist:

• Wildcard hostnames are not supported in ServiceEntry.
• CORS policy requires the experimental channel.
• Direct response, case-insensitive matching, and regex rewrite are not available.

Executing a migration

Solo’s migration tool for ambient mesh does not currently migrate the legacy Istio APIs to Gateway API. However, the community ingress2gateway project contains tooling that can be used to translate Istio objects into Gateway API objects.

From the project README:

The provider translates Istio API entities: Gateway and VirtualService to Gateway API: Gateway, HTTPRoute, TLSRoute, TCPRoute and ReferenceGrants.

The API translator converts the API fields that have a direct equivalent in the Gateway API. If a certain field of the Istio API cannot be translated directly, this field would be logged and ignored during the translation. It’s up to the user to handle such cases according to their needs.

Please note that the migration tool does not handle subsets. You will have to convert these to scoped Services yourself.

On the flow of traffic

Routing traffic into the mesh

Whether using a legacy Istio gateway or a gateway provisioned by the Gateway API, the proxy needs to be able to use the HBONE protocol to deliver traffic to the appropriate ztunnel or a waypoint. As with support for sidecars, you must configure Istio with the ambient profile in order for your gateways to support HBONE.

The default behavior of Istio is to send traffic that transits a gateway to the destination directly, even if that destination uses a waypoint. This is to avoid cases of double handling, for example, where rules for traffic splitting or fault injection could be applied twice. However, that means you may have to add certain policies on both the gateway and the waypoint, to ensure consistent enforcement regardless of the source of the traffic.

You can enable ingress waypoint routing on a service, such that traffic will be sent from the gateway to the configured waypoint, not to the destination service. See the ambient mesh documentation on gateways for more information.

Configuring waypoints

Targeting services vs. targeting gateways

A waypoint can receive configuration via two different methods, as described in part 2 of this series:

• routes targeting the services that the waypoint uses (a parentRef of kind: Service)
• routes targeting the waypoint itself (a parentRef of kind: Gateway).

How you choose to set up your routes will depend on how you relate services and waypoints.

EnvoyFilter

The EnvoyFilter API allows you to patch the configuration used by an Envoy proxy — traditionally, a sidecar or a gateway.

Waypoint proxies are also implemented with Envoy, but the listener configuration and filter chain is very different to the traditional model. As such, EnvoyFilter is not currently supported with waypoint proxies in upstream Istio.

Gloo Mesh, Solo’s commercial product based on Istio, adds support for EnvoyFilter to waypoints in the Enterprise license tier. A configuration from a sidecar mode installation will not work without modification, and we recommend you talk to your Solo customer success representative to help convert your existing configuration.

Connection pooling

In Sidecar mode, Istio lets you control connection pooling at both the client and the server.

If you use the inboundConnectionPool or per-port connectionPool configuration on the Sidecar object, you can take this configuration and apply it into a connectionPool configuration in a DestinationRule instead.

Solo’s migration tool will detect whether this change is needed.

Managing traffic egressing the mesh

A common misconception amongst Istio users is that a sidecar is a secure egress control point. This is not actually true; the only way to guarantee egress control is at the ingress of another object, namely an egress gateway.

Egress gateways in ambient mode are much simpler: you can define a ServiceEntry for a domain, which will cause it to be routed to a waypoint. You have all the standard routing and policy controls on that waypoint.

Wildcard hostnames are not supported. However, Gloo Mesh users at the enterprise tier have access to ztunnel egress policies, where all traffic can be denied or forwarded to a specific egress gateway.

The Solo migration tool will identify cases where the outboundTrafficPolicy configuration has been used, and where Gloo Mesh users will need to adjust their configuration to use ztunnel egress policies instead.