Rediger

Del via

Create model packages (preview)

  • Artikkel

Model package is a capability in Azure Machine Learning that allows you to collect all the dependencies required to deploy a machine learning model to a serving platform. Creating packages before deploying models provides robust and reliable deployment and a more efficient MLOps workflow. Packages can be moved across workspaces and even outside of Azure Machine Learning.

Important

This feature is currently in public preview. This preview version is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities.

For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

In this article, you learn how to package a model for deployment.

Prerequisites

Before following the steps in this article, make sure you have the following prerequisites:

  • An Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning.

  • An Azure Machine Learning workspace. If you don't have one, use the steps in the How to manage workspacesarticle to create one.

  • Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the owner or contributor role for the Azure Machine Learning workspace, or a custom role. For more information, see Manage access to an Azure Machine Learning workspace.

  • A model to package. This example, uses an MLflow model registered in the workspace.

    Caution

    Model packaging is not supported for models in the Azure AI model catalog, including large language models. Models in the Azure AI model catalog are optimized for inference on Azure AI deployment targets and are not suitable for packaging.

About this example

In this example, you will learn how to package models in Azure Machine Learning.

Clone the repository

The example in this article is based on code samples contained in the azureml-examples repository. To run the commands locally without having to copy/paste YAML and other files, first clone the repo and then change directories to the folder:

git clone https://github.com/Azure/azureml-examples --depth 1
cd azureml-examples/cli

This section uses the example in the folder endpoints/online/deploy-packages/custom-model.

Connect to your workspace

Connect to the Azure Machine Learning workspace where you'll do your work.

az account set --subscription <subscription>
az configure --defaults workspace=<workspace> group=<resource-group> location=<location>

Package a model

You can create model packages explicitly to allow you to control how the packaging operation is done. Use this workflow when:

  • You want to customize how the model package is created.
  • You want to deploy the model package outside Azure Machine Learning.
  • You want to use model packages in an MLOps workflow.

You can create model packages by specifying the:

  • Model to package: Each model package can contain only a single model. Azure Machine Learning doesn't support packaging of multiple models under the same model package.
  • Base environment: Environments are used to indicate the base image, and in Python packages dependencies your model need. For MLflow models, Azure Machine Learning automatically generates the base environment. For custom models, you need to specify it.
  • Serving technology: The inferencing stack used to run the model.

Register the model

Model packages require the model to be registered in either your workspace or in an Azure Machine Learning registry. In this example, you already have a local copy of the model in the repository, so you only need to publish the model to the registry in the workspace. You can skip this section if the model you're trying to deploy is already registered.

MODEL_NAME='sklearn-regression'
MODEL_PATH='model'
az ml model create --name $MODEL_NAME --path $MODEL_PATH --type custom_model

Create the base environment

Base environments are used to indicate the base image and the model Python package dependencies. Our model requires the following packages to be used as indicated in the conda file:

conda.yaml

name: model-env
channels:
  - conda-forge
dependencies:
  - python=3.9
  - numpy=1.23.5
  - pip=23.0.1
  - scikit-learn=1.2.2
  - scipy=1.10.1
  - xgboost==1.3.3

Note

How is the base environment different from the environment you use for model deployment to online and batch endpoints? When you deploy models to endpoints, your environment needs to include the dependencies of the model and the Python packages that are required for managed online endpoints to work. This brings a manual process into the deployment, where you have to combine the requirements of your model with the requirements of the serving platform. On the other hand, use of model packages removes this friction, since the required packages for the inference server will automatically be injected into the model package at packaging time.

Create the environment as follows:

Create an environment definition:

sklearn-regression-env.yml

$schema: https://azuremlschemas.azureedge.net/latest/environment.schema.json
name: sklearn-regression-env
image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04
conda_file: conda.yaml
description: An environment for models built with XGBoost and Scikit-learn.

Then create the environment:

az ml environment create -f environment/sklearn-regression-env.yml

Create a package specification

You can create model packages in Azure Machine Learning, using the Azure CLI or the Azure Machine Learning SDK for Python. The custom package specification supports the following attributes:

Attribute Type Description Required
target_environment str The name of the package to create. The result of a package operation is an environment in Azure Machine Learning. Yes
base_environment_source object The base image to use to create the package where dependencies for the model are specified. Yes, unless model is MLflow.
base_environment_source.type str The type of the base image. Only using another environment as the base image is supported (type: environment_asset) is supported.
base_environment_source.resource_id str The resource ID of the base environment to use. Use format azureml:<name>:<version> or a long resource id.
inferencing_server object The inferencing server to use. Yes
inferencing_server.type azureml_online
custom		 Use azureml_online for the Azure Machine Learning inferencing server, or custom for a custom online server like TensorFlow serving or Torch Serve. Yes
inferencing_server.code_configuration object The code configuration with the inference routine. It should contain at least one Python file with methods init and run. Yes, unless model is MLflow.
model_configuration object The model configuration. Use this attribute to control how the model is packaged in the resulting image. No
model_configuration.mode download
copy		 Indicate how the model would be placed in the package. Possible values are download (default) and copy. Use download when you want the model to be downloaded from the model registry at deployment time. This option create smaller docker images since the model is not included on it. Use copy when you want to disconnect the image from Azure Machine Learning. Model will be copied inside of the docker image at package time. copy is not supported on private link-enabled workspaces. No

  1. Create a package specification as follows:

    package-moe.yml

    $schema: http://azureml/sdk-2-0/ModelVersionPackage.json
base_environment_source:
    type: environment_asset
    resource_id: azureml:sklearn-regression-env:1
target_environment: sklearn-regression-online-pkg
inferencing_server: 
    type: azureml_online
    code_configuration:
      code: src
      scoring_script: score.py

  2. Start the model package operation:

    az ml model package -n $MODEL_NAME -v $MODEL_VERSION --file package-moe.yml

  3. The result of the package operation is an environment.

Package a model that has dependencies in private Python feeds

Model packages can resolve Python dependencies that are available in private feeds. To use this capability, you need to create a connection from your workspace to the feed and specify the PAT token configuration. The following Python code shows how you can configure the workspace where you're running the package operation.

from azure.ai.ml.entities import WorkspaceConnection
from azure.ai.ml.entities import PatTokenConfiguration

# fetching secrets from env var to secure access, these secrets can be set outside or source code
git_pat = os.environ["GIT_PAT"]

credentials = PatTokenConfiguration(pat=git_pat)

ws_connection = WorkspaceConnection(
    name="<workspace_connection_name>",
    target="<git_url>",
    type="git",
    credentials=credentials,
)

ml_client.connections.create_or_update(ws_connection)

Once the connection is created, build the model package as described in the section for Package a model. In the following example, the base environment of the package uses a private feed for the Python dependency bar, as specified in the following conda file:

conda.yml

name: foo
channels:
  - defaults
dependencies:
  - python
  - pip
  - pip:
    - --extra-index-url <python_feed_url>
    - bar

If you're using an MLflow model, model dependencies are indicated inside the model itself, and hence a base environment isn't needed. Instead, specify private feed dependencies when logging the model, as explained in Logging models with a custom signature, environment or samples.

Package a model that is hosted in a registry

Model packages provide a convenient way to collect dependencies before deployment. However, when models are hosted in registries, the deployment target is usually another workspace. When creating packages in this setup, use the target_environment property to specify the full location where you want the model package to be created, instead of just its name.

The following code creates a package of the t5-base model from a registry:

  1. Connect to the registry where the model is located and the workspace in which you need the model package to be created:

    az login

  2. Get a reference to the model you want to package. In this case we are packaging the model t5-base from azureml registry.

    MODEL_NAME="t5-base"
MODEL_VERSION=$(az ml model show --name $MODEL_NAME --label latest --registry-name azureml | jq .version -r)

  3. Configure a package specification. Since the model we want to package is MLflow, base environment and scoring script is optional.

    package.yml

    $schema: http://azureml/sdk-2-0/ModelVersionPackage.json
target_environment: pkg-t5-base-online
inferencing_server: 
    type: azureml_online

  4. Start the operation to create the model package:

    az ml model package --name $MODEL_NAME \
                    --version $MODEL_VERSION \
                    --registry-name azureml \
                    --file package.yml

  5. The package is now created in the target workspace and ready to be deployed.

Package models to deploy outside of Azure Machine Learning

Model packages can be deployed outside of Azure Machine Learning if needed. To guarantee portability, you only need to ensure that the model configuration in your package has the mode set to copy so that the model itself is copied inside the generated docker image instead of referenced from the model registry in Azure Machine Learning.

The following code shows how to configure copy in a model package:

package-external.yml

$schema: http://azureml/sdk-2-0/ModelVersionPackage.json
base_environment_source:
    type: environment_asset
    resource_id: azureml:sklearn-regression-env:1
target_environment: sklearn-regression-docker-pkg
inferencing_server: 
    type: azureml_online
    code_configuration:
      code: src
      scoring_script: score.py
model_configuration:
  mode: copy

Next step