MLflow guide

MLflow is an open source platform for managing the end-to-end machine learning lifecycle. It has the following primary components:

• Tracking: Allows you to track experiments to record and compare parameters and results.
• Models: Allow you to manage and deploy models from a variety of ML libraries to a variety of model serving and inference platforms.
• Projects: Allow you to package ML code in a reusable, reproducible form to share with other data scientists or transfer to production.
• Model Registry: Allows you to centralize a model store for managing models’ full lifecycle stage transitions: from staging to production, with capabilities for versioning and annotating.
• Model Serving: Allows you to host MLflow Models as REST endpoints.

Azure Databricks provides a fully managed and hosted version of MLflow integrated with enterprise security features, high availability, and other Azure Databricks workspace features such as experiment and run management and notebook revision capture. MLflow on Azure Databricks offers an integrated experience for tracking and securing machine learning model training runs and running machine learning projects.

First-time users should begin with the quickstart, which demonstrates the basic MLflow tracking APIs. The subsequent articles introduce each MLflow component with example notebooks and describe how these components are hosted within Azure Databricks.

MLflow supports Java, Python, R, and REST APIs.

MLflow data is encrypted by Azure Databricks using a platform-managed key. Encryption using Enable customer-managed keys for managed services is not supported.

• Get started with MLflow experiments
• Track machine learning training runs
• Log, load, register, and deploy MLflow models
• Run MLflow Projects on Azure Databricks
• MLflow Model Registry on Azure Databricks
• Databricks Autologging
• Copy MLflow objects between workspaces
• Tutorial: End-to-end ML models on Azure Databricks