Large language models (LLMs)

These examples fine-tune and post-train large language models (LLMs) on AI Runtime. They cover parameter-efficient methods like Low-Rank Adaptation (LoRA) and full supervised fine-tuning across libraries including TRL, Unsloth, Axolotl, and LLM Foundry. For batch inference examples, see Batch inference.

Example	Interface	Description
Reinforcement learning of Gemma4-2B	Notebook	Post-train Gemma4-2B with Group Relative Policy Optimization (GRPO) and Low-Rank Adaptation (LoRA) on a single H100 GPU using Unsloth.
Supervised Fine-tuning (Full) of Qwen3-4B	Notebook	Full-weight fine-tune Qwen3-4B on a single H100 GPU with TRL, using BF16 mixed precision and gradient checkpointing.
Supervised Fine-tuning (LoRA) of Llama-3.2-3B	Notebook	LoRA fine-tune Llama-3.2-3B on a single GPU with the Unsloth library.
Supervised Fine-tuning (LoRA) of GPT-OSS 20B	Notebook	LoRA fine-tune GPT-OSS 20B on 8 H100 GPUs with TRL and distributed data parallelism.
Supervised Fine-tuning (Full) of Llama-3.2-1B	Notebook	Full-weight fine-tune Llama-3.2-1B on 8 H100 GPUs with TRL and DeepSpeed ZeRO Stage 3.
Supervised Fine-tuning (LoRA) of Olmo3 7B	Notebook	LoRA fine-tune Olmo3 7B across multiple GPUs with the Axolotl library.
Supervised fine-tuning (Full) and serving of Qwen3.5-0.8B	Notebook	Fine-tune Qwen3.5-0.8B on a single H100 GPU with TRL, then serve it behind a vLLM Model Serving endpoint.
Supervised Fine-tuning of Llama-3.2-3B on multi-GPUs	Notebook	LoRA fine-tune Llama-3.2-3B across multiple GPUs with the Unsloth library.
Supervised Fine-tuning (LoRA) of GPT-OSS 120B	Notebook	LoRA fine-tune GPT-OSS 120B on 8 H100 GPUs with TRL, using DDP and FSDP.
Training transformers on multi-GPUs	Notebook	Train Transformer models using PyTorch Fully Sharded Data Parallel (FSDP) to shard model parameters across multiple GPUs.
Multi-node FSDP fine-tuning of Llama 3.1 8B	CLI	Supervised fine-tune Llama 3.1 8B with PyTorch FSDP across two 8x H100 nodes, log training loss to MLflow, and save a consolidated checkpoint.
Multi-node FSDP2 fine-tuning of Qwen3.8-27B	CLI	Supervised fine-tune Qwen3.8-27B with PyTorch FSDP2 across four 8x H100 nodes.
Full-weight fine-tuning of Qwen3-8B for invoice extraction	CLI	Fine-tune Qwen3-8B for invoice and receipt entity extraction with TRL and DeepSpeed ZeRO-3 on one 8x H100 node.
Supervised fine-tuning of Qwen2.5-3B with Ray Train	CLI	Use Ray Train to fine-tune Qwen2.5-3B with one distributed data parallel worker per GPU on an 8x H100 node.
LoRA hyperparameter search for Qwen2.5-0.5B	CLI	Use Ray Tune and the asynchronous successive halving algorithm (ASHA) to search LoRA configurations across four A10 GPUs and log the best configuration to MLflow.
Longformer fine-tuning for long-document classification	CLI	Fine-tune Longformer with LoRA for long-document sentiment classification on one GPU and save the adapter to a Unity Catalog volume.
ModernBERT fine-tuning on one GPU	CLI	Fine-tune ModernBERT for topic classification on one A10 GPU and register the model in Unity Catalog.
ModernBERT fine-tuning with DDP	CLI	Fine-tune ModernBERT for topic classification with PyTorch DDP on an 8x H100 node and register the model in Unity Catalog.
LoRA fine-tuning of Nemotron 3.5 Lightning	CLI	Fine-tune Nemotron 3.5 Lightning with NeMo AutoModel, FSDP2, and expert parallelism on an 8x H100 node.
LoRA fine-tuning of Nemotron 3.5 Super VL	CLI	Fine-tune the Nemotron 3.5 Super VL vision-language model with NeMo AutoModel and LoRA on an 8x H100 node.
Fine-tune and serve Qwen2.5-0.5B	CLI	Fine-tune Qwen2.5-0.5B with LoRA, register the merged model in Unity Catalog, and deploy it to Custom Model Serving.

Video demo

This video walks through the Supervised Fine-tuning (LoRA) of Llama-3.2-3B example notebook in detail (12 minutes).