Migrating to ambient mesh from Istio in sidecar mode

Migrate from Istio sidecar mode to ambient mode with zero downtime using the Solo migration tool and Solo builds of Istio.

The Solo ambient mesh migration tool provides a prescriptive path for migrating from Istio’s sidecar mode to ambient mode. This migration is zero-downtime when used with the free Solo builds of Istio, but can also translate Kubernetes manifests for users of upstream Istio.

While the tool walks you through each step, understanding ambient mode and its differences from sidecars is helpful. Review the documentation for an overview of the architecture and differences. In particular, understanding the security architecture of ambient mesh is critical if you use authorization policies.

The tool does not manipulate any configuration in your cluster. It only reads configuration and suggests actions that you must take to adjust configuration. When following the procedures recommended by the tool and using a Solo build of Istio, a cluster can be migrated from sidecar mode to ambient without downtime. However, the tool cannot account for all possible cluster setups; operators are expected to review the suggestions and test them before acting on them.

Before you begin

Throughout this guide, a simplified representation of the traffic path of communication between a client and server is shown. As you progress through each step, more components are deployed and enabled. Make sure the tool passes all checks before moving to the next step.

In sidecar mode, client-side sidecars are responsible for traffic routing decisions (depicted here as “routing”), and server-side sidecars are responsible for authentication and authorization (depicted as “policy”).

There are two main considerations for migration:

• Client-side sidecars make all routing decisions locally, without awareness of waypoint proxies
• Authorization policies are applied to server-side sidecars using a selector, whereas in ambient mode, policies target waypoints using a reference

The Solo builds of Istio include support for migration by enhancing sidecar mode to be aware of waypoint proxies, and to route traffic to a waypoint if a workload is enrolled to use it, using the same logic as ztunnel.

Istio policies that are bound to a waypoint must be adjusted to use a targetRef instead of a selector. The migration tool detects the most common policies and provides new configuration, which you then validate and apply.

Download the migration helper

Install the gloo CLI client:

curl -sL https://storage.googleapis.com/gloo-cli/install.sh | sh
export PATH=$HOME/.gloo/bin:$PATH

There are a number of options that can be provided to the migration commands, including the ability to run from YAML files instead of a live cluster. These can be found in an appendix to this page.

Prepare your cluster for ambient mesh

To migrate to ambient mesh, your cluster must have:

• Istio v1.25 or later
• Istio components configured with support for ambient mode
• The Gateway API CRDs installed

Certain Istio features are not available in ambient mode.

• Multicluster mesh
• VM integration
• SPIRE

If you attempt to upgrade a cluster that uses these features, the tool advises you.

Run the prerequisites step to confirm your cluster is ready for operating in ambient mode, and you are not using any features that are not supported:

An example cluster that is ready for ambient mode:

gloo ambient migrate
• Starting phase pre-reqs...
✅ Phase pre-reqs succeeded!
  ✅ Cluster CNI compatibility: passed
  ✅ Istio version compatibility: passed
  ✅ Multicluster usage compatibility: passed
  ✅ Virtual Machine usage compatibility: passed
  ✅ SPIRE usage compatibility: passed

All existing istiod installations in the cluster must run Istio 1.25 or higher. The next phase ensures the Istio installation has support for ambient mode. If an upgrade is needed, you may consider enabling ambient mode in the same step.

Next, run the cluster-setup check to confirm your Istio environment is ready:

gloo ambient migrate --phase cluster-setup
• Starting phase cluster-setup...
❌ Phase cluster-setup failed!
  ❌ Ambient mode enabled: failed. 1 error occurred:
  	* Istiod istio-system/istiod must have 'PILOT_ENABLE_AMBIENT=true'. Upgrade Istio with '--set profile=ambient'.
  
  
  ❌ DaemonSets deployed: failed. 2 errors occurred:
  	* ztunnel not found
  	* Istio CNI agent not found
  
  
  ❌ Sidecars support ambient mode: failed. 6 errors occurred:
  	* Sidecar default/details-v1-79dfbd6fff-gvmh7 is missing 'ENABLE_HBONE'. Upgrade Istio with '--set profile=ambient' and restart the pod.
  	* Sidecar default/productpage-v1-dffc47f64-tt9r8 is missing 'ENABLE_HBONE'. Upgrade Istio with '--set profile=ambient' and restart the pod.
  	* Sidecar default/ratings-v1-65f797b499-fsxqf is missing 'ENABLE_HBONE'. Upgrade Istio with '--set profile=ambient' and restart the pod.
  	* Sidecar default/reviews-v1-5c4d6d447c-tz928 is missing 'ENABLE_HBONE'. Upgrade Istio with '--set profile=ambient' and restart the pod.
  	* Sidecar default/reviews-v2-65cb66b45c-dzptm is missing 'ENABLE_HBONE'. Upgrade Istio with '--set profile=ambient' and restart the pod.
  	* Sidecar default/reviews-v3-f68f94645-h9c8l is missing 'ENABLE_HBONE'. Upgrade Istio with '--set profile=ambient' and restart the pod.
  
  
  ✅ Required CRDs installed: passed

Most users must install ambient mode and restart their sidecar workloads to pass this step.

Enable support for ambient mode

Most sidecar mode installations do not use the ambient profile, which includes the ztunnel component. Follow the steps in the upgrade guide to install the Solo builds.

This ensures Istio is installed with support for ambient mode, which enables you to start enrolling workloads into an ambient mesh, and enables the existing dataplanes (sidecar and gateways) to communicate with ambient workloads.

For an installation with istioctl, ensure you have the same version of istioctl as the Istio version you want to install, then update the installation:

export HUB=us-docker.pkg.dev/soloio-img/istio
export TAG=1.26.2-solo
istioctl install --set profile=ambient --set hub=$HUB --set tag=$TAG

For an installation with Helm:

export HUB=us-docker.pkg.dev/soloio-img/istio
export TAG=1.26.2-solo
helm repo add istio https://istio-release.storage.googleapis.com/charts
helm repo update istio
helm install istio-base istio/base -n istio-system --set global.hub=$HUB --set global.tag=$TAG --create-namespace --wait
helm install istiod istio/istiod --namespace istio-system --set global.hub=$HUB --set global.tag=$TAG --set profile=ambient --wait
helm install istio-cni istio/cni -n istio-system --set global.hub=$HUB --set global.tag=$TAG --set profile=ambient --wait
helm install ztunnel istio/ztunnel -n istio-system --set global.hub=$HUB --set global.tag=$TAG --wait

At this stage, you have ambient mode support installed, but not in use:

Ensure your workloads are running an ambient-aware sidecar

Configuring an Istio installation for ambient mode enables support for sidecars to use the HBONE protocol. The sidecars must be restarted for this change to take effect. Restart the Deployments in a namespace:

kubectl rollout restart deployment -n <namespace>

Before progressing to the next step, ensure the check now fully passes:

gloo ambient migrate --phase cluster-setup
• Starting phase cluster-setup...
✅ Phase cluster-setup succeeded!
  ✅ Ambient mode enabled: passed
  ✅ DaemonSets deployed: passed
  ✅ Sidecars support ambient mode: passed
  ✅ Required CRDs installed: passed

Deploy waypoints

In sidecar mode, all services have L7 processing, whether or not they need this functionality. Ambient mode allows you to choose which services are opted into L7 processing.

To migrate from sidecars, evaluate each service to determine whether it requires a waypoint proxy. The migration tool analyzes cluster configuration to determine which services require waypoints to retain their existing functionality. For each recommendation, the tool provides configuration to deploy these waypoints.

The suggestions from the tool are a best-effort analysis based on the policies in the cluster. Review the recommended waypoint configurations carefully before deploying waypoints for best results. For example, the tool might detect customized load balancing for a service and recommend a waypoint, as waypoints are required to implement custom load balancing policies. However, you may decide that the value of this policy does not justify the waypoint, and choose to skip the recommendation. If you are not sure, deploy the waypoint.

In some cases, the tool might not recommend a waypoint where you actually want apps to use one. Even without any policies configured, sidecars provide a variety of functionalities such as mTLS, HTTP observability, and HTTP request-level load balancing. While ztunnel provides mTLS and can optionally provide HTTP observability in Solo Enterprise for Istio, it does not provide request-level load balancing. Applications requiring this can benefit from deploying a waypoint, even when the tool does not recommend one.

An example cluster where a single waypoint is recommended:

gloo ambient migrate --phase deploy-waypoints
• Starting phase deploy-waypoints...
⚠️ Phase deploy-waypoints has recommendations!
  🔮 Namespace "application" may require a waypoint for the following services:
     * Service "application/hello-world" depends on VirtualService "application/hello-world"

  ℹ️ Generated waypoints written to /tmp/istio-migrate/recommended-waypoints.yaml

The recommended-waypoints.yaml file contains the recommended waypoint configuration:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: waypoint
  namespace: application
spec:
  gatewayClassName: istio-waypoint
  listeners:
  - name: mesh
    port: 15008
    protocol: HBONE

Apply this configuration through your regular deployment pipeline, or using kubectl:

kubectl apply -f /tmp/istio-migrate/recommended-waypoints.yaml 

For more information about configuring waypoints, see Configure waypoints.

At the end of this stage, you have deployed waypoint proxies, but they are not in use.

Migrate policies

At this point, any necessary waypoints are deployed, but they are not fully configured or in use. The next step is to configure them. As in the previous step, traffic does not yet route through the waypoint; this occurs in a later step once the waypoints are fully configured.

For policies that modify services, VirtualServices and DestinationRules, no changes are required. The tool copies the configuration and generates policies to be applied to waypoints. However, policies that modify specific workloads must be modified to apply to the waypoint.

The tool automatically provides recommendations on how to apply the following resources to waypoints:

• AuthorizationPolicy
• RequestAuthentication
• Telemetry
• WasmPlugin
• EnvoyFilter (when running with --enterprise)

It also detects and warns against policies that are not supported or required in ambient mode.

An example in which a few policies are migrated to the waypoint, as well as a warning about an unsupported configuration:

gloo ambient migrate --phase migrate-policies
• Starting phase migrate-policies...
⚠️ Phase migrate-policies has recommendations!
  ⚠️ Sidecar/legacy/sidecar will be removed in a later step: Found unsupported configurations:
    * Configuration scoping is detected. Typically, this is used for scalability purposes, which is not necessary with ambient.
  🔮 Add AuthorizationPolicy/app1/echo-from-waypoint: Service/app1/echo must allow traffic from its waypoint
  🔮 Apply AuthorizationPolicy/app1/echo-legacy-policies: Existing configuration is copied from policy app1/legacy-policies to be enforced at the waypoint.
  🔮 Apply AuthorizationPolicy/app1/echo-policies: Existing configuration is copied from policy app1/policies to be enforced at the waypoint.
  🔮 Apply AuthorizationPolicy/app1/waypoint-allow-nothing: Existing configuration is copied from namespace policy app1/allow-nothing to be enforced at the waypoint.
  ℹ️ Recommended policies written to /tmp/istio-migrate/recommended-policies.yaml

The recommended-policies.yaml file contains the recommended policies that are generated. As before, apply these through your regular deployment pipeline, or using kubectl:

kubectl apply -f recommended-policies.yaml

At the end of this stage, you have deployed policies which allow you to safely enroll your workloads in ambient mode.

A note on authorization policy

AuthorizationPolicy in particular requires careful handling due to the changes in how policies are enforced in ambient mode. In ambient mode, Layer 4 and Layer 7 policies can be enforced against services, applied at the waypoint. Additionally, Layer 4 policies can be enforced against workloads.

This requires the following changes:

• If a workload policy contains HTTP attributes, a waypoint is required, as recommended in the previous phase. The policy must then be moved to configure the waypoint, rather than the workload.
• If a waypoint is deployed, the workload must allow traffic from the waypoint.

The migration tool helps move policies to the waypoints as required, and additionally generates policies to allow traffic from the waypoints.

Recommendation details

Note that, at this point, all policy recommendations are in addition to the existing policies. This ensures that during the migration process, policies continue to apply to all traffic requests, regardless of whether traffic is routed through a waypoint. Later phases transition traffic to start flowing through the waypoint, and then optionally recommend cleaning up the unused policies once the migration is complete.

For example, consider a case where an authorization policy allows a single client to access an application:

When a waypoint is deployed, the tool recommends additional policies to support the same behavior for traffic from the waypoint, in addition to traffic that does not go through the waypoint. This ensures a zero-downtime transition.

Enable waypoints

Now that waypoints have been deployed and configured, you can start to use them. The tool detects which waypoints to enable and provides steps to enable them.

Once enabled, traffic flows through the waypoint. Enablement can be done on a service-by-service basis to allow you to monitor the health of the system as you progress. There is no need to quickly move everything at once. If a service behaves unexpectedly, revert the waypoint enablement. See the waypoint configuration documentation for more details.

An example cluster where one waypoint is ready for enablement:

gloo ambient migrate --phase use-waypoints
• Starting phase use-waypoints...
⚠️ Phase use-waypoints has recommendations!
  ⚠️ Warning: detected waypoint app1/waypoint but it is not used by anything
  🔮 Detected namespace app1 requires a waypoint, but not configured to use one. Configure it with: kubectl label namespace app1 istio.io/use-waypoint=waypoint

Enable the waypoint with the command provided:

kubectl label namespace app1 istio.io/use-waypoint=waypoint

Traffic now flows from the client-side sidecar to the waypoint, without any local routing or policy decisions, and then to the server-side sidecar. Each connection is separately encrypted with mTLS.

Sequencing waypoint enablement and sidecar removal

All waypoints should be enabled before you start to remove sidecars from workloads. However, this is not strictly required, which may be useful in scenarios where some workloads take a while to complete migration.

In order for all policies to be correctly applied, the request flow needs either:

• A sidecar on the client and server side
• A waypoint proxy enabled

Do not remove sidecars from a namespace once that namespace has a waypoint enabled, because clients in that namespace may still call servers in other namespaces that do not yet have an enabled waypoint.

The following diagram shows how a client sidecar successfully communicates with a migrated namespace and with a namespace that is not yet migrated. If you remove the client sidecar before the unmigrated namespace is migrated, some policies are not correctly applied to traffic requests.

Simplify policy

At this point, all traffic is flowing through waypoints. Policies can be simplified by removing any policies that are specific to sidecars, as they are no longer relevant.

In the previous step, you deployed copies of existing policies. The original sources of these can now be removed.

gloo ambient migrate --phase policy-simplification
• Starting phase policy-simplification...
⚠️ Phase policy-simplification has recommendations!
  🔮 AuthorizationPolicy was migrated and can be deleted: kubectl delete authorizationpolicies.security.istio.io -n app1 allow-nothing
  🔮 AuthorizationPolicy was migrated and can be deleted: kubectl delete authorizationpolicies.security.istio.io -n app1 legacy-policies
  🔮 AuthorizationPolicy was migrated and can be deleted: kubectl delete authorizationpolicies.security.istio.io -n app1 policies

Remove sidecars

When all services have been configured to use waypoints, and all waypoints have been configured to apply equivalent configurations, it is safe to remove the sidecars from workloads.

The sidecar removal phase detects workloads still using sidecars and recommends moving to ambient mode:

gloo ambient migrate --phase remove-sidecars
• Starting phase remove-sidecars...
⚠️ Phase remove-sidecars has recommendations!
  🔮 namespace app1 can disable its sidecars: kubectl label ns app1 istio-injection- istio.io/dataplane-mode=ambient
  ⚠️ sidecar found for Pod app1/echo-7fb78cb7c5-plsjl

To remove injection from a namespace and add it to the ambient mesh, run the recommended commands:

kubectl label ns app1 istio-injection- istio.io/dataplane-mode=ambient

When your applications are next restarted or redeployed, they run without sidecars, and their traffic is captured by ztunnel.

Appendix: About the migration tool

The ambient migration tool inspects the state of the cluster and makes recommendations about next steps. Each phase of the migration runs a series of checks, and validates that the checks are completed before proceeding. If checks in a phase fail, the tool gives recommendations for how to resolve them. Details about each phase are documented above.

The migration process involves running gloo ambient migrate a number of times. Each time, the tool analyzes the current state of the cluster and recommends next steps.

There are a few useful options to consider: