MLflow and Azure Machine Learning

APPLIES TO: Azure CLI ml extension v2 (current) Python SDK azure-ai-ml v2 (current)

MLflow is an open-source framework that's designed to manage the complete machine learning lifecycle. Its ability to train and serve models on different platforms allows you to use a consistent set of tools regardless of where your experiments are running: locally on your computer, on a remote compute target, on a virtual machine, or on an Azure Machine Learning compute instance.

Tip

Azure Machine Learning workspaces are MLflow-compatible, which means you can use Azure Machine Learning workspaces in the same way that you use an MLflow tracking server. Such compatibility has the following advantages:

  • You can use Azure Machine Learning workspaces as your tracking server for any experiment you're running with MLflow, whether it runs on Azure Machine Learning or not. You only need to configure MLflow to point to the workspace where the tracking should happen.
  • You can run any training routine that uses MLflow in Azure Machine Learning without changes. MLflow also supports model management and model deployment capabilities.

Tracking with MLflow

Azure Machine Learning uses MLflow Tracking for metric logging and artifact storage for your experiments, whether you created the experiments via the Azure Machine Learning Python SDK, the Azure Machine Learning CLI, or Azure Machine Learning studio. We recommend using MLflow for tracking experiments. To get started, see Log metrics, parameters, and files with MLflow.

Note

Unlike the Azure Machine Learning SDK v1, there's no logging functionality in the SDK v2. We recommend that you use MLflow for logging.

With MLflow Tracking, you can connect Azure Machine Learning as the back end of your MLflow experiments. The workspace provides a centralized, secure, and scalable location to store training metrics and models.

Capabilities include:

You can also use MLflow to Query & compare experiments and runs with MLflow.

Important

  • MLflow in R support is limited to tracking experiment's metrics, parameters and models on Azure Machine Learning jobs. Interactive training on RStudio or Jupyter Notebooks with R kernels is not supported. Model management and registration is not supported using the MLflow R SDK. As an alternative, use Azure ML CLI or Azure ML studio for model registration and management. View the following R example about using the MLflow tracking client with Azure Machine Learning.
  • MLflow in Java support is limited to tracking experiment's metrics and parameters on Azure Machine Learning jobs. Artifacts and models can't be tracked using the MLflow Java SDK. As an alternative, use the Outputs folder in jobs along with the method mlflow.save_model to save models (or artifacts) you want to capture. View the following Java example about using the MLflow tracking client with the Azure Machine Learning.

Model registries with MLflow

Azure Machine Learning supports MLflow for model management. This support represents a convenient way to support the entire model lifecycle for users who are familiar with the MLflow client.

To learn more about how to manage models by using the MLflow API in Azure Machine Learning, view Manage model registries in Azure Machine Learning with MLflow.

Model deployments of MLflow models

You can deploy MLflow models to Azure Machine Learning so that you can apply the model management capabilities and no-code deployment offering in Azure Machine Learning. Azure Machine Learning supports deploying models to both real-time and batch endpoints. You can use the azureml-mlflow MLflow plug-in, the Azure Machine Learning CLI v2, and the user interface in Azure Machine Learning studio.

Learn more at Deploy MLflow models to Azure Machine Learning.

Training MLflow projects (preview)

You can submit training jobs to Azure Machine Learning by using MLflow projects (preview). You can submit jobs locally with Azure Machine Learning tracking or migrate your jobs to the cloud via Azure Machine Learning compute.

Learn more at Train machine learning models with MLflow projects and Azure Machine Learning.

MLflow SDK, Azure Machine Learning v2, and Azure Machine Learning studio capabilities

The following table shows which operations are supported by each of the tools available in the machine learning lifecycle.

Feature MLflow SDK Azure Machine Learning v2 (CLI/SDK) Azure Machine Learning studio
Track and log metrics, parameters, and models
Retrieve metrics, parameters, and models 1
Submit training jobs with MLflow projects 2
Submit training jobs with inputs and outputs
Submit training jobs by using machine learning pipelines
Manage experiments and runs
Manage MLflow models 3
Manage non-MLflow models
Deploy MLflow models to Azure Machine Learning (Online & Batch) 4
Deploy non-MLflow models to Azure Machine Learning

Note

Example notebooks

If you're getting started with MLflow in Azure Machine Learning, we recommend that you explore the notebook examples about how to use MLflow:

Next steps