How to package a registered model with Docker

Article
03/02/2023

This article shows how to package a registered Azure Machine Learning model with Docker.

Prerequisites

This article assumes you have already trained and registered a model in your machine learning workspace. To learn how to train and register a scikit-learn model, follow this tutorial.

Package models

In some cases, you might want to create a Docker image without deploying the model. Or you might want to download the image and run it on a local Docker installation. You might even want to download the files used to build the image, inspect them, modify them, and build the image manually.

Model packaging enables you to do these things. It packages all the assets needed to host a model as a web service and allows you to download either a fully built Docker image or the files needed to build one. There are two ways to use model packaging:

Download a packaged model: Download a Docker image that contains the model and other files needed to host it as a web service.

Generate a Dockerfile: Download the Dockerfile, model, entry script, and other assets needed to build a Docker image. You can then inspect the files or make changes before you build the image locally.

Both packages can be used to get a local Docker image.

Tip

Creating a package is similar to deploying a model. You use a registered model and an inference configuration.

Important

To download a fully built image or build an image locally, you need to have Docker installed in your development environment.

Download a packaged model

The following example builds an image, which is registered in the Azure container registry for your workspace:

package = Model.package(ws, [model], inference_config)
package.wait_for_creation(show_output=True)

After you create a package, you can use package.pull() to pull the image to your local Docker environment. The output of this command will display the name of the image. For example:

Status: Downloaded newer image for myworkspacef78fd10.azurecr.io/package:20190822181338.

After you download the model, use the docker images command to list the local images:

REPOSITORY                               TAG                 IMAGE ID            CREATED             SIZE
myworkspacef78fd10.azurecr.io/package    20190822181338      7ff48015d5bd        4 minutes ago       1.43 GB

To start a local container based on this image, use the following command to start a named container from the shell or command line. Replace the <imageid> value with the image ID returned by the docker images command.

docker run -p 6789:5001 --name mycontainer <imageid>

This command starts the latest version of the image named myimage. It maps local port 6789 to the port in the container on which the web service is listening (5001). It also assigns the name mycontainer to the container, which makes the container easier to stop. After the container is started, you can submit requests to http://localhost:6789/score.

Generate a Dockerfile and dependencies

The following example shows how to download the Dockerfile, model, and other assets needed to build an image locally. The generate_dockerfile=True parameter indicates that you want the files, not a fully built image.

package = Model.package(ws, [model], inference_config, generate_dockerfile=True)
package.wait_for_creation(show_output=True)
# Download the package.
package.save("./imagefiles")
# Get the Azure container registry that the model/Dockerfile uses.
acr=package.get_container_registry()
print("Address:", acr.address)
print("Username:", acr.username)
print("Password:", acr.password)

This code downloads the files needed to build the image to the imagefiles directory. The Dockerfile included in the saved files references a base image stored in an Azure container registry. When you build the image on your local Docker installation, you need to use the address, user name, and password to authenticate to the registry. Use the following steps to build the image by using a local Docker installation:

From a shell or command-line session, use the following command to authenticate Docker with the Azure container registry. Replace <address>, <username>, and <password> with the values retrieved by package.get_container_registry().
```
docker login <address> -u <username> -p <password>
```
To build the image, use the following command. Replace <imagefiles> with the path of the directory where package.save() saved the files.
```
docker build --tag myimage <imagefiles>
```
This command sets the image name to myimage.

To verify that the image is built, use the docker images command. You should see the myimage image in the list:

REPOSITORY      TAG                 IMAGE ID            CREATED             SIZE
<none>          <none>              2d5ee0bf3b3b        49 seconds ago      1.43 GB
myimage         latest              739f22498d64        3 minutes ago       1.43 GB

To start a new container based on this image, use the following command:

docker run -p 6789:5001 --name mycontainer myimage:latest

Example client to test the local container

The following code is an example of a Python client that can be used with the container:

import requests
import json

# URL for the web service.
scoring_uri = 'http://localhost:6789/score'

# Two sets of data to score, so we get two results back.
data = {"data":
        [
            [ 1,2,3,4,5,6,7,8,9,10 ],
            [ 10,9,8,7,6,5,4,3,2,1 ]
        ]
        }
# Convert to JSON string.
input_data = json.dumps(data)

# Set the content type.
headers = {'Content-Type': 'application/json'}

# Make the request and display the response.
resp = requests.post(scoring_uri, input_data, headers=headers)
print(resp.text)

For example clients in other programming languages, see Consume models deployed as web services.

Stop the Docker container

To stop the container, use the following command from a different shell or command line:

docker kill mycontainer