How to deploy a blue deployment in an azure managed online endpoint while having a green deployment with 100% traffic allocation, and avoid downtime

Question

How to deploy a blue deployment in an azure managed online endpoint while having a green deployment with 100% traffic allocation, and avoid downtime

Matias Larsson 125

I have a green deployment with 100% traffic allocation, and I need to know how can I deploy a blue deployment to the same endpoint and not have the traffic in green set to 0% while blue is building up (this seems to be the default behaviour).

What I am currently doing is: After having the blue deployment ready for serving, I update the allocation to either {"blue": 0%, "green": 100%}, or to {"blue": 100%, "green": 0%}, but I cannot avoid the 2-4 mins downtime I get on the deployment that was serving traffic originally.

Accepted answer

0 additional answers

Your answer

Answer 1

Pavankumar Purilla 8,335 Microsoft External Staff Moderator

Hi Matias Larsson,

To deploy a blue deployment in an Azure Managed Online Endpoint while maintaining 100% traffic allocation to the existing green deployment without causing downtime, you need to explicitly create the blue deployment with zero traffic allocation. By default, if you create a new deployment (blue) without specifying traffic allocation, Azure will automatically adjust the traffic so that blue receives 100% of the traffic during its startup, which temporarily sets green to 0% and results in a brief service interruption. To avoid this, use the Azure CLI or SDK to create the blue deployment with --traffic-weight 0, ensuring that green continues serving all traffic while blue is being provisioned. Once the blue deployment is successfully created and healthy, you can test it in isolation using the azureml-model-deployment header in inference requests. After verifying that the blue deployment performs correctly, you can gradually shift traffic from green to blue (e.g., 90% green, 10% blue) or fully reallocate it (100% blue, 0% green) without any downtime. This safe rollout strategy allows seamless updates while maintaining service availability.
For more information: az ml online-deployment.

I hope this information helps. Thank you!

Matias Larsson 125 Reputation points

2025-04-30T23:26:06.9066667+00:00

Thank you so much @Pavankumar Purilla ! Your explanation about blue/green deployments in Azure Managed Online Endpoints is exceptionally clear and helpful. I've been implementing this using the Python SDK (ManagedOnlineDeployment class) rather than the CLI, and I'm encountering an issue with the critical 2-4 minute window between deployment creation and traffic allocation adjustment where we experience downtime. While you mentioned using --traffic-weight 0 with the CLI, I'm struggling to find the equivalent parameter in the Python SDK documentation. The ManagedOnlineDeployment constructor doesn't appear to have a direct traffic allocation parameter, and updating the endpoint's traffic configuration after deployment creation still results in that brief service interruption.Would you be able to share how to specify the initial 0% traffic allocation directly during deployment creation using the Python SDK? This would be tremendously valuable for ensuring our production services maintain 100% availability during deployments. I greatly appreciate your expertise on this matter!
Pavankumar Purilla 8,335 Reputation points Microsoft External Staff Moderator

2025-05-02T07:24:54.04+00:00

Hi Matias Larsson,
To avoid downtime during the deployment of a new model (blue) while ensuring the existing model (green) continues serving 100% of the traffic, you can follow this approach: First, create the blue deployment using the ManagedOnlineDeployment class without altering the traffic allocation. This means that during the deployment creation, no traffic is routed to the blue deployment, and the green deployment will continue handling all traffic by default. Once the blue deployment is successfully created, test its functionality using a custom header (e.g., azureml-model-deployment) to verify that it is working as expected. After confirming the blue deployment’s health, update the traffic allocation by calling ml_client.online_endpoints.begin_create_or_update() to shift traffic between green and blue as needed. This allows you to keep green serving 100% of the traffic until the blue deployment is fully validated, thus avoiding any downtime and ensuring a smooth transition when updating traffic.

For more information: Deploy the model to the endpoint
Matias Larsson 125 Reputation points

2025-05-05T15:57:23.9866667+00:00

Thanks again for your answer!

Share via

How to deploy a blue deployment in an azure managed online endpoint while having a green deployment with 100% traffic allocation, and avoid downtime

0 additional answers

Your answer