Deep learning based recommender systems

Important

This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Azure Databricks previews.

This page provides notebook examples for building recommendation systems using Serverless GPU compute. These examples demonstrate how to create efficient recommendation models using modern deep learning approaches.

Tutorial	Description
Two-tower recommendation model	Learn how to convert recommendation data into Mosaic Data Shard (MDS) format and then use that data to create a two-tower recommendation model.
Fine-tune embedding models with llm-foundry	Fine-tune a BERT-style embedding model on serverless GPU compute using contrastive learning with the llm-foundry framework and Composer's trainer.

Two-tower recommendation model

These notebooks demonstrate how to convert your recommendation data into Mosaic Data Shard (MDS) format and then use that data to create a two-tower recommendation model. This approach is particularly effective for large-scale recommendation systems.

Data preparation: Convert recommendation model dataset to MDS format

First, convert your recommendation dataset to the MDS format for efficient data loading:

Convert data

Get notebook

Model training: Two-tower recommender model using PyTorch Lightning

Train the two-tower recommender model using the prepared dataset and the PyTorch Lightning Trainer API on multiple GPU nodes (A10 or H100 GPUs).

PyTorch recommender

Get notebook