System and app requirements

System and app requirements

Ambient mesh, as of Istio 1.24, runs in, and supports workloads in, a single Kubernetes cluster.

Multi-cluster support is available in Gloo Mesh, an enterprise distribution of ambient mesh.

Platform-specific prerequisites

Some customization of the installation of ambient mesh is needed in certain Kubernetes environments. Our quickstart guide attempts to detect and automatically apply these configurations. For production or manual installation, the required changes are noted in the Istio documentation.

Cloud providers

Amazon Elastic Kubernetes Service (EKS)

If you are using EKS:

  • with Amazon’s VPC CNI
  • with Pod ENI trunking enabled
  • and you are using EKS pod-attached SecurityGroups via SecurityGroupPolicy

POD_SECURITY_GROUP_ENFORCING_MODE must be explicitly set to standard, or pod health probes (which are by-default silently exempted from all policy enforcement by the VPC CNI) will fail. This is because Istio uses a link-local SNAT address for kubelet health probes, which Amazon’s VPC CNI is not aware of, and the VPC CNI does not have an option to exempt link-local addresses from policy enforcement.

Refer to the Istio documentation for details.

Google Kubernetes Engine (GKE)

On GKE, critical components can only be installed into a namespace with a resource quota defined. To install ambient mesh in the default istio-system namespace, first apply this resource:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: gcp-critical-pods
  namespace: istio-system
spec:
  hard:
    pods: 1000
  scopeSelector:
    matchExpressions:
    - operator: In
      scopeName: PriorityClass
      values:
      - system-node-critical

Refer to the Istio documentation for details.

Red Hat OpenShift

Add -n kube-system --set global.platform=openshift to your Helm installation commands to set the required configuration for OpenShift.

$ helm install istio-cni istio/cni -n kube-system --set profile=ambient --set global.platform=openshift --wait

Local installation

No extra configuration is required when installing ambient mesh on kind.

k3d

Add -n kube-system --set global.platform=k3d to your Helm installation commands to set the required configuration for k3d. For example:

$ helm install istio-cni istio/cni -n istio-system --set profile=ambient --set global.platform=k3s --wait
k3s

Add -n kube-system --set global.platform=k3s to your Helm installation commands to set the required configuration for k3s. For example:

$ helm install istio-cni istio/cni -n istio-system --set profile=ambient --set global.platform=k3s --wait
Microk8s

Add -n kube-system --set global.platform=microk8s to your Helm installation commands to set the required configuration for microk8s. For example:

$ helm install istio-cni istio/cni -n istio-system --set profile=ambient --set global.platform=microk8s --wait
Minikube

Add -n kube-system --set global.platform=minikube to your Helm installation commands to set the required configuration for minikube. For example:

$ helm install istio-cni istio/cni -n istio-system --set profile=ambient --set global.platform=minikube --wait

CNI-specific prerequisites

Cilium
  • Cilium currently defaults to proactively deleting other CNI plugins and their config, and must be configured with cni.exclusive = false to properly support chaining. See the Cilium documentation for more details.

  • Cilium’s BPF masquerading is currently disabled by default, and has issues with Istio’s use of link-local IPs for Kubernetes health checking. Enabling BPF masquerading via bpf.masquerade=true is not currently supported, and results in non-functional pod health checks in Istio ambient. Cilium’s default iptables masquerading implementation should continue to function correctly.

  • Due to how Cilium manages node identity and internally allow-lists node-level health probes to pods, applying any default-DENY NetworkPolicy in a Cilium CNI install underlying Istio in ambient mode will cause kubelet health probes (which are by-default silently exempted from all policy enforcement by Cilium) to be blocked. This is because Istio uses a link-local SNAT address for kubelet health probes, which Cilium is not aware of, and Cilium does not have an option to exempt link-local addresses from policy enforcement.

    This can be resolved by applying the following CiliumClusterWideNetworkPolicy:

    apiVersion: "cilium.io/v2"
    kind: CiliumClusterwideNetworkPolicy
    metadata:
      name: "allow-ambient-hostprobes"
    spec:
      description: "Allows SNAT-ed kubelet health check probes into ambient pods"
      enableDefaultDeny:
        egress: false
        ingress: false
      endpointSelector: {}
      ingress:
      - fromCIDR:
        - "169.254.7.127/32"

    This policy override is not required unless you already have other default-deny NetworkPolicies or CiliumNetworkPolicies applied in your cluster.

    Please see issue #49277 and CiliumClusterWideNetworkPolicy for more details.

Application compatibility

Ambient mesh works without any modification required to your existing Kubernetes deployments. You can label a namespace to add all of its workloads to the mesh, or opt-in certain deployments as needed. With ambient mesh, you bypass some of the previously restrictive elements of sidecar-based service mesh. Server-send-first protocols now work, most reserved ports are now available, and the ability for containers to bypass the sidecar — either maliciously or not — is eliminated.

Deployment guidance

It is safe to deploy ambient mesh to a running cluster. Installing the Istio components does not change anything to do with traffic routing - adding workloads to the mesh is a separate step which is easily reverted.

We encourage you to install ambient mesh and become familiar with the secure overlay mode before adding waypoint proxies. For many users, the secure overlay mode alone will meet their needs, and can deliver mTLS and L4 authorization at as low as 10% of the cost of running Istio in sidecar mode.