How to deploy Azure ML batch endpoint from docker image?

Question

Hi, I have my own deep learning task that requires 2-3 different ml models, I built the code and containerized it, i.e. the python env and code is in the docker image.

I am running fastapi servers inside docker to run code.

Deployed it in aws sagemaker async endpoint and it is working fine.

Now, I need to deploy it to azure ml batch endpoint, but there's no documentation as such to deploy it using custom docker container.

Can someone help me?

Answer 1

Deploying a custom Docker container to an Azure ML batch endpoint can be tricky due to limited documentation. However, see if the steps below will help you:

Prepare Your Docker Image: Ensure your Docker image is ready and pushed to a container registry accessible by Azure (e.g., Azure Container Registry).

Create an Azure ML Workspace: If you haven’t already, set up an Azure ML workspace.

1. Set Up Your Environment: Install the Azure CLI and the Azure ML extension:

      az extension add -n ml
   

2. Create a Batch Endpoint: Use the Azure CLI or Python SDK to create a batch endpoint. Here’s an example using the CLI:

      az ml batch-endpoint create --name <your-endpoint-name> --file <your-endpoint-config-file>
   

3. Deploy Your Model: Create a deployment for your batch endpoint using your custom Docker image. Here’s an example configuration:

      name:
   

4. Test Your Deployment: Once deployed, you can test your batch endpoint to ensure everything works as expected.

References:

https://learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-custom-container?view=azureml-api-2&tabs=cli

https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-batch-model-deployments?view=azureml-api-2&tabs=cli

---

Please remember to "Accept the answer” and “up-vote” wherever the information provided helps you; this can benefit other community members.