Model training examples

This section includes examples showing how to train machine learning models on Azure Databricks using many popular open-source libraries.

You can also use AutoML, which automatically prepares a dataset for model training, performs a set of trials using open-source libraries such as scikit-learn and XGBoost, and creates a Python notebook with the source code for each trial run so you can review, reproduce, and modify the code.

Machine learning examples

Package	Notebook(s)	Features
scikit-learn	Machine learning tutorial	Unity Catalog, classification model, MLflow, automated hyperparameter tuning with Hyperopt and MLflow
scikit-learn	End-to-end example	Unity Catalog, classification model, MLflow, automated hyperparameter tuning with Hyperopt and MLflow, XGBoost
MLlib	MLlib examples	Binary classification, decision trees, GBT regression, Structured Streaming, custom transformer
xgboost	XGBoost examples	Python, PySpark, and Scala, single node workloads and distributed training

Hyperparameter tuning examples

For general information about hyperparameter tuning in Azure Databricks, see Hyperparameter tuning.

Package	Notebook	Features
Optuna	Get started with Optuna	Optuna, distributed Optuna, scikit-learn, MLflow
Hyperopt	Distributed hyperopt	Distributed hyperopt, scikit-learn, MLflow
Hyperopt	Compare models	Use distributed hyperopt to search hyperparameter space for different model types simultaneously
Hyperopt	Distributed training algorithms and hyperopt	Hyperopt, MLlib
Hyperopt	Hyperopt best practices	Best practices for datasets of different sizes