Deploy your MLflow model to a batch endpoint

7 minutes

An easy way to deploy a model to a batch endpoint is to use an MLflow model. Azure Machine Learning will automatically generate the scoring script and environment for MLflow models.

To deploy an MLflow model, you need to have created an endpoint. Then you can deploy the model to the endpoint.

Register an MLflow model

To avoid needed a scoring script and environment, an MLflow model needs to be registered in the Azure Machine Learning workspace before you can deploy it to a batch endpoint.

To register an MLflow model, you'll use the Model class, while specifying the model type to be MLFLOW_MODEL. To register the model with the Python SDK, you can use the following code:

from azure.ai.ml.entities import Model
from azure.ai.ml.constants import AssetTypes

model_name = 'mlflow-model'
model = ml_client.models.create_or_update(
    Model(name=model_name, path='./model', type=AssetTypes.MLFLOW_MODEL)
)

In this example, we're taking the model files from a local path. The files are all stored in a local folder called model. The folder must include the MLmodel file, which describes how the model can be loaded and used.

Tip

Learn more about the MLmodel format.

Deploy an MLflow model to an endpoint

To deploy an MLflow model to a batch endpoint, you'll use the BatchDeployment class.

When you deploy a model, you'll need to specify how you want the batch scoring job to behave. The advantage of using a compute cluster to run the scoring script (which is automatically generated by Azure Machine Learning), is that you can run the scoring script on separate instances in parallel.

When you configure the model deployment, you can specify:

instance_count: Count of compute nodes to use for generating predictions.
max_concurrency_per_instance: Maximum number of parallel scoring script runs per compute node.
mini_batch_size: Number of files passed per scoring script run.
output_action: What to do with the predictions: summary_only or append_row.
output_file_name: File to which predictions will be appended, if you choose append_row for output_action.

Tip

Explore the reference documentation to create a batch deployment with the Python SDK v2.

To deploy an MLflow model to a batch endpoint, you can use the following code:

from azure.ai.ml.entities import BatchDeployment, BatchRetrySettings
from azure.ai.ml.constants import BatchDeploymentOutputAction

deployment = BatchDeployment(
    name="forecast-mlflow",
    description="A sales forecaster",
    endpoint_name=endpoint.name,
    model=model,
    compute="aml-cluster",
    instance_count=2,
    max_concurrency_per_instance=2,
    mini_batch_size=2,
    output_action=BatchDeploymentOutputAction.APPEND_ROW,
    output_file_name="predictions.csv",
    retry_settings=BatchRetrySettings(max_retries=3, timeout=300),
    logging_level="info",
)
ml_client.batch_deployments.begin_create_or_update(deployment)

Continue

Register an MLflow model

Deploy an MLflow model to an endpoint

Feedback