Azure ML - Managed Endpoint - Safe rollout

NAUGHTON Stephen 1

Hello,

My question surrounds the safe rollout feature (blue-green deployment) available with Azure ML - Managed Endpoints.

Specifically, in our environment we run a set of production models on blue. We then deploy a newly trained set of models to green but they receive no traffic. When testing is complete and we want to promote the new models to production, we simply switch traffic from blue to green.

Is there any potential for dropped requests when the load balancer switches traffic from the blue endpoints to green?

Thanks,
Stephen

YutongTie-MSFT 50,796 Reputation points

2022-09-14T23:02:17.18+00:00

Hello @NAUGHTON Stephen

Thanks for using Microsoft Q&A platform. I think you are seeking a way to stop the request to switch the traffic by traffic manager, am I correct? Please clarify the question, thanks.

Regards,
Yutong
NAUGHTON Stephen 1 Reputation point

2022-09-15T08:55:16.277+00:00
Hi Yutong,

Thanks for your response.

My question is about the blue-green rollout feature available in Azure Machine Learning Managed Endpoints:

https://learn.microsoft.com/en-us/azure/machine-learning/how-to-safely-rollout-managed-endpoints

During a production release we have an existing set of models deployed on the "blue" endpoints which receive all live traffic. To release new models, we create a new "green" set of endpoints. Then when testing is complete we move traffic from blue to green using the following CLI command:

az ml online-endpoint update --name $ENDPOINT_NAME --traffic "blue=0 green=100"

My question is:

When this command is executing and traffic is being switched from blue to green, are the currently running requests terminated or allowed to finish?

Also, is there any risk that new incoming requests might be dropped while traffic is switched?

Thanks,
Stephen

Share via

Azure ML - Managed Endpoint - Safe rollout

Your answer