Create model packages (preview)

Model package is a capability in Azure Machine Learning that allows you to collect all the dependencies required to deploy a machine learning model to a serving platform. Creating packages before deploying models provides robust and reliable deployment and a more efficient MLOps workflow. Packages can be moved across workspaces and even outside of Azure Machine Learning.

Important

This feature is currently in public preview. This preview version is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities.

For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

In this article, you learn how to package a model for deployment.

Prerequisites

Before following the steps in this article, make sure you have the following prerequisites:

• An Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning.

• An Azure Machine Learning workspace. If you don't have one, use the steps in the How to manage workspacesarticle to create one.

• Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the owner or contributor role for the Azure Machine Learning workspace, or a custom role. For more information, see Manage access to an Azure Machine Learning workspace.

• A model to package. This example, uses an MLflow model registered in the workspace.

  Caution

  Model packaging is not supported for models in the Azure AI model catalog, including large language models. Models in the Azure AI model catalog are optimized for inference on Azure AI deployment targets and are not suitable for packaging.

About this example

In this example, you will learn how to package models in Azure Machine Learning.

Clone the repository

The example in this article is based on code samples contained in the azureml-examples repository. To run the commands locally without having to copy/paste YAML and other files, first clone the repo and then change directories to the folder:

Azure CLI

git clone https://github.com/Azure/azureml-examples --depth 1
cd azureml-examples/cli

Python

!git clone https://github.com/Azure/azureml-examples --depth 1
!cd azureml-examples/sdk/python

This section uses the example in the folder endpoints/online/deploy-packages/custom-model.

Connect to your workspace

Connect to the Azure Machine Learning workspace where you'll do your work.

Azure CLI

az account set --subscription <subscription>
az configure --defaults workspace=<workspace> group=<resource-group> location=<location>

Python

The workspace is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section, you connect to the workspace in which you perform deployment tasks.

1. Import the required libraries:

   from azure.identity import DefaultAzureCredential
   from azure.ai.ml import MLClient
   from azure.ai.ml.entities import (
       AzureMLOnlineInferencingServer,
       ModelPackage,
       CodeConfiguration,
       BaseEnvironment,
       ModelConfiguration,
   )
   from azure.ai.ml.entities import (
       ManagedOnlineEndpoint,
       ManagedOnlineDeployment,
       Environment,
       Model,
   )
   

2. If you're running in a compute instance in Azure Machine Learning, create an MLClient as follows:

   ml_client = MLClient.from_config(DefaultAzureCredential())
   

   Otherwise, configure your workspace details and get a handle to the workspace:

   subscription_id = "<subscription>"
   resource_group = "<resource-group>"
   workspace = "<workspace>"
   
   ml_client = MLClient(DefaultAzureCredential(), subscription_id, resource_group, workspace)
   

Package a model

You can create model packages explicitly to allow you to control how the packaging operation is done. Use this workflow when:

• You want to customize how the model package is created.
• You want to deploy the model package outside Azure Machine Learning.
• You want to use model packages in an MLOps workflow.

You can create model packages by specifying the:

• Model to package: Each model package can contain only a single model. Azure Machine Learning doesn't support packaging of multiple models under the same model package.
• Base environment: Environments are used to indicate the base image, and in Python packages dependencies your model need. For MLflow models, Azure Machine Learning automatically generates the base environment. For custom models, you need to specify it.
• Serving technology: The inferencing stack used to run the model.

Register the model

Model packages require the model to be registered in either your workspace or in an Azure Machine Learning registry. In this example, you already have a local copy of the model in the repository, so you only need to publish the model to the registry in the workspace. You can skip this section if the model you're trying to deploy is already registered.

Azure CLI

MODEL_NAME='sklearn-regression'
MODEL_PATH='model'
az ml model create --name $MODEL_NAME --path $MODEL_PATH --type custom_model

Python

model_name = "sklearn-regression"
model_path = "model"

model = ml_client.models.create_or_update(Model(name=model_name, path=model_path))

Create the base environment

Base environments are used to indicate the base image and the model Python package dependencies. Our model requires the following packages to be used as indicated in the conda file:

conda.yaml

name: model-env
channels:
  - conda-forge
dependencies:
  - python=3.9
  - numpy=1.23.5
  - pip=23.0.1
  - scikit-learn=1.2.2
  - scipy=1.10.1
  - xgboost==1.3.3

Note

How is the base environment different from the environment you use for model deployment to online and batch endpoints? When you deploy models to endpoints, your environment needs to include the dependencies of the model and the Python packages that are required for managed online endpoints to work. This brings a manual process into the deployment, where you have to combine the requirements of your model with the requirements of the serving platform. On the other hand, use of model packages removes this friction, since the required packages for the inference server will automatically be injected into the model package at packaging time.

Create the environment as follows:

Azure CLI

Create an environment definition:

sklearn-regression-env.yml

$schema: https://azuremlschemas.azureedge.net/latest/environment.schema.json
name: sklearn-regression-env
image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04
conda_file: conda.yaml
description: An environment for models built with XGBoost and Scikit-learn.

Then create the environment:

az ml environment create -f environment/sklearn-regression-env.yml

Python

base_environment = ml_client.environments.create_or_update(
    Environment(
        name=f"{model_name}-env",
        image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04",
        conda_file="environment/conda.yml",
    )
)

Create a package specification

You can create model packages in Azure Machine Learning, using the Azure CLI or the Azure Machine Learning SDK for Python. The custom package specification supports the following attributes:

Azure CLI

Attribute	Type	Description	Required
target_environment	str	The name of the package to create. The result of a package operation is an environment in Azure Machine Learning.	Yes
base_environment_source	object	The base image to use to create the package where dependencies for the model are specified.	Yes, unless model is MLflow.
base_environment_source.type	str	The type of the base image. Only using another environment as the base image is supported (type: environment_asset) is supported.
base_environment_source.resource_id	str	The resource ID of the base environment to use. Use format azureml:<name>:<version> or a long resource id.
inferencing_server	object	The inferencing server to use.	Yes
inferencing_server.type	azureml_online custom	Use azureml_online for the Azure Machine Learning inferencing server, or custom for a custom online server like TensorFlow serving or Torch Serve.	Yes
inferencing_server.code_configuration	object	The code configuration with the inference routine. It should contain at least one Python file with methods init and run.	Yes, unless model is MLflow.
model_configuration	object	The model configuration. Use this attribute to control how the model is packaged in the resulting image.	No
model_configuration.mode	download copy	Indicate how the model would be placed in the package. Possible values are download (default) and copy. Use download when you want the model to be downloaded from the model registry at deployment time. This option create smaller docker images since the model is not included on it. Use copy when you want to disconnect the image from Azure Machine Learning. Model will be copied inside of the docker image at package time. copy is not supported on private link-enabled workspaces.	No

Python

Attribute	Type	Description	Required
target_environment	str	The name of the package to create. The result of a package operation is an environment in Azure Machine Learning.	Yes
base_environment_source	BaseEnvironment	The base image to use to create the package where dependencies for the model are specified.	Yes, unless model type is MLflow.
base_environment_source.type	BaseEnvironmentType	The type of the base image. Only using another environment (EnvironmentAsset) as the base image is supported.
base_environment_source.resource_id	str	The resource ID of the base environment to use. Use format azureml:<name>:<version> or a long-format resource id.
inferencing_server	AzureMLOnlineInferencingServer CustomOnlineInferenceServer	The inferencing server to use. Use AzureMLOnlineInferencingServer to Azure Machine Learning inferencing server, or CustomOnlineInferenceServer for a custom online server like TensorFlow serving, Torch Serve, etc. If set to AzureMLInferencingServer and the model type isn't Mlflow, a code configuration section should be specified, containing at least one Python file with methods init and run. If set to CustomOnlineInferenceServer, an online server configuration section should be specified.	Yes
model_configuration	ModelConfiguration	The model configuration. Use this attribute to control how the model is packaged in the resulting image.	No
model_configuration.mode	ModelInputMode	Specify how the model would be placed in the package. Possible values are ModelInputMode.DOWNLOAD (default) and ModelInputMode.COPY. Use ModelInputMode.DOWNLOAD when you want the model to be downloaded from the model registry at deployment time. This option create smaller docker images since the model is not included on it. Use ModelInputMode.COPY when you want to disconnect the image from Azure Machine Learning. Model will be copied inside of the docker image at package time. ModelInputMode.COPY is not supported on private link-enabled workspaces.	No

1. Create a package specification as follows:

   Azure CLI

   package-moe.yml

   $schema: http://azureml/sdk-2-0/ModelVersionPackage.json
   base_environment_source:
       type: environment_asset
       resource_id: azureml:sklearn-regression-env:1
   target_environment: sklearn-regression-online-pkg
   inferencing_server: 
       type: azureml_online
       code_configuration:
         code: src
         scoring_script: score.py
   

   Python

   To create a model package, create a package specification as follows:

   pakage_config = ModelPackage(
       target_environment="sklearn-regression-online-pkg",
       base_environment_source=BaseEnvironment(
           type="asset",
           resource_id=f"azureml:{base_environment.name}:{base_environment.version}",
       ),
       inferencing_server=AzureMLOnlineInferencingServer(
           code_configuration=CodeConfiguration(code="src", scoring_script="score.py")
       ),
   )

2. Start the model package operation:

   Azure CLI

   az ml model package -n $MODEL_NAME -v $MODEL_VERSION --file package-moe.yml
   

   Python

   model_package = ml_client.models.package(model_name, model.version, pakage_config)

3. The result of the package operation is an environment.

Package a model that has dependencies in private Python feeds

Model packages can resolve Python dependencies that are available in private feeds. To use this capability, you need to create a connection from your workspace to the feed and specify the PAT token configuration. The following Python code shows how you can configure the workspace where you're running the package operation.

from azure.ai.ml.entities import WorkspaceConnection
from azure.ai.ml.entities import PatTokenConfiguration

# fetching secrets from env var to secure access, these secrets can be set outside or source code
git_pat = os.environ["GIT_PAT"]

credentials = PatTokenConfiguration(pat=git_pat)

ws_connection = WorkspaceConnection(
    name="<workspace_connection_name>",
    target="<git_url>",
    type="git",
    credentials=credentials,
)

ml_client.connections.create_or_update(ws_connection)

Once the connection is created, build the model package as described in the section for Package a model. In the following example, the base environment of the package uses a private feed for the Python dependency bar, as specified in the following conda file:

conda.yml

name: foo
channels:
  - defaults
dependencies:
  - python
  - pip
  - pip:
    - --extra-index-url <python_feed_url>
    - bar

If you're using an MLflow model, model dependencies are indicated inside the model itself, and hence a base environment isn't needed. Instead, specify private feed dependencies when logging the model, as explained in Logging models with a custom signature, environment or samples.

Package a model that is hosted in a registry

Model packages provide a convenient way to collect dependencies before deployment. However, when models are hosted in registries, the deployment target is usually another workspace. When creating packages in this setup, use the target_environment property to specify the full location where you want the model package to be created, instead of just its name.

The following code creates a package of the t5-base model from a registry:

1. Connect to the registry where the model is located and the workspace in which you need the model package to be created:

   Azure CLI

   az login
   

   Python

   registry_client = MLClient(
       credential=DefaultAzureCredential(),
       subscription_id=ml_client.subscription_id,
       resource_group_name=ml_client.resource_group_name,
       workspace_reference=ml_client.workspace_name,
       registry_name=registry_name,
   )

2. Get a reference to the model you want to package. In this case we are packaging the model t5-base from azureml registry.

   Azure CLI

   MODEL_NAME="t5-base"
   MODEL_VERSION=$(az ml model show --name $MODEL_NAME --label latest --registry-name azureml | jq .version -r)
   

   Python

   model_name = "t5-base"
   model = registry_client.models.get(name=model_name, label="latest")

3. Configure a package specification. Since the model we want to package is MLflow, base environment and scoring script is optional.

   Azure CLI

   package.yml

   $schema: http://azureml/sdk-2-0/ModelVersionPackage.json
   target_environment: pkg-t5-base-online
   inferencing_server: 
       type: azureml_online
   

   Python

   import time
   
   model_package_name = f"pkg-{model.name}-{model.version}"
   model_package_version = str(int(time.time()))
   
   package_config = ModelPackage(
       target_environment=model_package_name,
       target_environment_version=model_package_version,
       inferencing_server=AzureMLOnlineInferencingServer(),
   )

4. Start the operation to create the model package:

   Azure CLI

   az ml model package --name $MODEL_NAME \
                       --version $MODEL_VERSION \
                       --registry-name azureml \
                       --file package.yml
   

   Python

   model_package = registry_client.models.package(
       model.name, model.version, package_config
   )

5. The package is now created in the target workspace and ready to be deployed.

Package models to deploy outside of Azure Machine Learning

Model packages can be deployed outside of Azure Machine Learning if needed. To guarantee portability, you only need to ensure that the model configuration in your package has the mode set to copy so that the model itself is copied inside the generated docker image instead of referenced from the model registry in Azure Machine Learning.

The following code shows how to configure copy in a model package:

Azure CLI

package-external.yml

$schema: http://azureml/sdk-2-0/ModelVersionPackage.json
base_environment_source:
    type: environment_asset
    resource_id: azureml:sklearn-regression-env:1
target_environment: sklearn-regression-docker-pkg
inferencing_server: 
    type: azureml_online
    code_configuration:
      code: src
      scoring_script: score.py
model_configuration:
  mode: copy

Python

pakage_config = ModelPackage(
    target_environment="sklearn-regression-online-pkg",
    base_environment_source=BaseEnvironment(
        type="asset",
        resource_id=f"azureml:{base_environment.name}:{base_environment.version}",
    ),
    inferencing_server=AzureMLOnlineInferencingServer(
        code_configuration=CodeConfiguration(code="src", scoring_script="score.py")
    ),
    model_configuration=ModelConfiguration(mode="copy"),
)

Next step

• Package and deploy a model to Online Endpoints.
• Package and deploy a model to App Service.