Large language models (LLMs)

Important

This feature is in Public Preview.

These notebooks fine-tune and post-train large language models (LLMs) on AI Runtime. They cover parameter-efficient methods like Low-Rank Adaptation (LoRA) and full supervised fine-tuning across libraries including TRL, Unsloth, Axolotl, and LLM Foundry. For batch inference examples, see Batch inference.

Tutorial	Description
Reinforcement learning of Gemma4-2B	Post-train Gemma4-2B with Group Relative Policy Optimization (GRPO) and Low-Rank Adaptation (LoRA) on a single H100 GPU using Unsloth.
Supervised Fine-tuning (Full) of Qwen3-4B	Full-weight fine-tune Qwen3-4B on a single H100 GPU with TRL, using BF16 mixed precision and gradient checkpointing.
Supervised Fine-tuning (LoRA) of Llama-3.2-3B	LoRA fine-tune Llama-3.2-3B on a single GPU with the Unsloth library.
Supervised Fine-tuning (LoRA) of GPT-OSS 20B	LoRA fine-tune GPT-OSS 20B on 8 H100 GPUs with TRL and distributed data parallelism.
Supervised Fine-tuning (Full) of Llama-3.2-1B	Full-weight fine-tune Llama-3.2-1B on 8 H100 GPUs with TRL and DeepSpeed ZeRO Stage 3.
Supervised Fine-tuning (LoRA) of Olmo3 7B	LoRA fine-tune Olmo3 7B across multiple GPUs with the Axolotl library.
Supervised fine-tuning (Full) and serving of Qwen3.5-0.8B	Fine-tune Qwen3.5-0.8B on a single H100 GPU with TRL, then serve it behind a vLLM Model Serving endpoint.
Supervised Fine-tuning of Llama-3.2-3B on multi-GPUs	LoRA fine-tune Llama-3.2-3B across multiple GPUs with the Unsloth library.
Supervised Fine-tuning (LoRA) of GPT-OSS 120B	LoRA fine-tune GPT-OSS 120B on 8 H100 GPUs with TRL, using DDP and FSDP.
Training transformers on multi-GPUs	Train Transformer models using PyTorch Fully Sharded Data Parallel (FSDP) to shard model parameters across multiple GPUs.

Video demo

This video walks through the Supervised Fine-tuning (LoRA) of Llama-3.2-3B example notebook in detail (12 minutes).