A/B testing using Azure Container Instance.

Question

I have two ML models A and B registered in my workspace and I want to deploy them to Azure Container Instance and perform A/B testing or continuous rollout or Canary deployment. Can it be done without using AKS and only on ACI.

Answer 1

Hello @HARISH KUMAR

Thanks for reaching out to us. I can understand you want to deploy your ML models to ACI to do the A/B test. For question can models to be deployed only ACI, the answer is yes, but please be aware that ACI has some limitation compared to AKS, please check if ACI is a good choice for your model:

About models:
ACI is suitable only for small models that are under 1 GB in size.
We recommend using single-node AKS to dev-test larger models.
The number of models to be deployed is limited to 1,000 models per deployment (per container).

About Vnet:
When using Azure Container Instances in a virtual network, the virtual network must be in the same resource group as your Azure Machine Learning workspace.
When using Azure Container Instances inside the virtual network, the Azure Container Registry (ACR) for your workspace cannot also be in the virtual network.

Other points:
prefer not to manage your own Kubernetes cluster
Are OK with having only a single replica of your service, which may impact uptime
You are advised to debug locally before deploying to the web service,

AKS Advantages for your reference as well:
Fast response time
Autoscaling of the deployed service
Logging
Model data collection
Authentication
TLS termination
Hardware acceleration options such as GPU and field-programmable gate arrays (FPGA)

If you feel like ACI fulfill your need, then you can follow below guidance to do the deployment:
https://learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-azure-container-instance#deploy-to-aci

Hope this helps, please let me know if you have more questions.

Regards,
Yutong

-Please kindly accept the answer if you feel helpful, thanks.