System and app requirements
Ambient mesh, as of Istio 1.24, runs in, and supports workloads in, a single Kubernetes cluster.
Platform-specific prerequisites
Some customization of the installation of ambient mesh is needed in certain Kubernetes environments. Our quickstart guide attempts to detect and automatically apply these configurations. For production or manual installation, the required changes are noted in the Istio documentation.
Cloud providers
Amazon Elastic Kubernetes Service (EKS)
If you are using EKS:
- with Amazon’s VPC CNI
- with Pod ENI trunking enabled
- and you are using EKS pod-attached SecurityGroups via SecurityGroupPolicy
POD_SECURITY_GROUP_ENFORCING_MODE
must be explicitly set to standard
, or pod health probes (which are by-default silently exempted from all policy enforcement by the VPC CNI) will fail. This is because Istio uses a link-local SNAT address for kubelet health probes, which Amazon’s VPC CNI is not aware of, and the VPC CNI does not have an option to exempt link-local addresses from policy enforcement.
Google Kubernetes Engine (GKE)
On GKE, critical components can only be installed into a namespace with a resource quota defined. To install ambient mesh in the default istio-system
namespace, first apply this resource:
apiVersion: v1
kind: ResourceQuota
metadata:
name: gcp-critical-pods
namespace: istio-system
spec:
hard:
pods: 1000
scopeSelector:
matchExpressions:
- operator: In
scopeName: PriorityClass
values:
- system-node-critical
Red Hat OpenShift
Add -n kube-system --set global.platform=openshift
to your Helm installation commands to set the required configuration for OpenShift.
$ helm install istio-cni istio/cni -n kube-system --set profile=ambient --set global.platform=openshift --wait
Local installation
No extra configuration is required when installing ambient mesh on kind.
k3d
Add -n kube-system --set global.platform=k3d
to your Helm installation commands to set the required configuration for k3d. For example:
$ helm install istio-cni istio/cni -n istio-system --set profile=ambient --set global.platform=k3s --wait
k3s
Add -n kube-system --set global.platform=k3s
to your Helm installation commands to set the required configuration for k3s. For example:
$ helm install istio-cni istio/cni -n istio-system --set profile=ambient --set global.platform=k3s --wait
Microk8s
Add -n kube-system --set global.platform=microk8s
to your Helm installation commands to set the required configuration for microk8s. For example:
$ helm install istio-cni istio/cni -n istio-system --set profile=ambient --set global.platform=microk8s --wait
Minikube
Add -n kube-system --set global.platform=minikube
to your Helm installation commands to set the required configuration for minikube. For example:
$ helm install istio-cni istio/cni -n istio-system --set profile=ambient --set global.platform=minikube --wait
CNI-specific prerequisites
Cilium
-
Cilium currently defaults to proactively deleting other CNI plugins and their config, and must be configured with
cni.exclusive = false
to properly support chaining. See the Cilium documentation for more details. -
Cilium’s BPF masquerading is currently disabled by default, and has issues with Istio’s use of link-local IPs for Kubernetes health checking. Enabling BPF masquerading via
bpf.masquerade=true
is not currently supported, and results in non-functional pod health checks in Istio ambient. Cilium’s default iptables masquerading implementation should continue to function correctly. -
Due to how Cilium manages node identity and internally allow-lists node-level health probes to pods, applying any default-DENY
NetworkPolicy
in a Cilium CNI install underlying Istio in ambient mode will causekubelet
health probes (which are by-default silently exempted from all policy enforcement by Cilium) to be blocked. This is because Istio uses a link-local SNAT address for kubelet health probes, which Cilium is not aware of, and Cilium does not have an option to exempt link-local addresses from policy enforcement.This can be resolved by applying the following
CiliumClusterWideNetworkPolicy
:apiVersion: "cilium.io/v2" kind: CiliumClusterwideNetworkPolicy metadata: name: "allow-ambient-hostprobes" spec: description: "Allows SNAT-ed kubelet health check probes into ambient pods" enableDefaultDeny: egress: false ingress: false endpointSelector: {} ingress: - fromCIDR: - "169.254.7.127/32"
This policy override is not required unless you already have other default-deny
NetworkPolicies
orCiliumNetworkPolicies
applied in your cluster.Please see issue #49277 and CiliumClusterWideNetworkPolicy for more details.
Application compatibility
Ambient mesh works without any modification required to your existing Kubernetes deployments. You can label a namespace to add all of its workloads to the mesh, or opt-in certain deployments as needed. With ambient mesh, you bypass some of the previously restrictive elements of sidecar-based service mesh. Server-send-first protocols now work, most reserved ports are now available, and the ability for containers to bypass the sidecar — either maliciously or not — is eliminated.
Deployment guidance
It is safe to deploy ambient mesh to a running cluster. Installing the Istio components does not change anything to do with traffic routing - adding workloads to the mesh is a separate step which is easily reverted.
We encourage you to install ambient mesh and become familiar with the secure overlay mode before adding waypoint proxies. For many users, the secure overlay mode alone will meet their needs, and can deliver mTLS and L4 authorization at as low as 10% of the cost of running Istio in sidecar mode.