Introduction to generative AI apps on Azure Databricks

Mosaic AI supports both simple and complex GenAI applications, from Retrieval Augmented Generation (RAG) chatbots to tool-calling agents. This user guide explains key concepts behind GenAI apps and agent systems on Databricks and provides guidance for building, evaluating, and scaling GenAI apps.

Page	Description
Get started: no-code GenAI	Try AI Playground for UI-based testing and prototyping.
Get started: MLflow 3 for GenAI	Try MLflow for GenAI tracing, evaluation, and human feedback.
Concepts: GenAI on Databricks	Learn about GenAI models, agents, tools, and apps.
Platform: Key GenAI features	Find details on key features for GenAI on Azure Databricks.

Get started building GenAI apps

Try out UI-based and code-based GenAI on Azure Databricks.

Tutorial	Description
Get started: Query LLMs and prototype AI agents with no code	Get familiar with AI Playground for UI-based testing and prototyping.
Get started: MLflow 3 for GenAI	Try out MLflow for GenAI tracing, evaluation, and human feedback.
Get started querying LLMs on Databricks	Use Foundation Model APIs to query GenAI models using code.

Learn GenAI concepts

Get familiar with foundational GenAI concepts, such as models, agents, tools, and apps.

Guide	Description
Concepts: Generative AI on Azure Databricks	Learn about GenAI models, agents, tools, and apps.
Key challenges in building GenAI apps	Learn about key challenges of GenAI and how Databricks addresses them.
Agent system design patterns	Learn about options and trade-offs for agent designs, from simple chains to complex multi-agent systems.

Use Azure Databricks features to build GenAI apps

For no-code or low-code approaches, start by getting familiar with:

Feature	Description
Agent Bricks	Build and optimize domain-specific, high-quality AI agent systems for common use cases.
AI Playground	Query GenAI models and agents, do prompt engineering, and prototype tool-calling agents in a UI.
AI Functions	Call built-in SQL functions for AI tasks.

For code-first approaches, start by getting familiar with:

Feature	Description
MLflow for GenAI	Use MLflow for tracing and observability, evaluation and monitoring.
Foundation models in Model Serving	Use GenAI model endpoints, including Databricks-hosted Foundation Models APIs and external models.
Vector Search	Create and query vector indexes for RAG and other agent systems.
Mosaic AI Agent Framework	Build and deploy AI agents using code.
AI Gateway	Govern and monitor access to GenAI models and endpoints.

For a more detailed list, see Mosaic AI capabilities for GenAI.

General intelligence vs. data intelligence

• General intelligence refers to what the LLM inherently knows from broad pretraining on diverse text. This is useful for language fluency and general reasoning.
• Data intelligence refers to your organization's domain-specific data and APIs. This might include customer records, product information, knowledge bases, or documents that reflect your unique business environment.

Agent systems blend these two sources of knowledge: They start with an LLM's broad, generic knowledge and then bring in real-time or domain-specific data to answer detailed questions or perform specialized actions. With Azure Databricks, you can embed data intelligence into your GenAI apps at every level:

• Data sources like vector indexes and Genie

• Agents, including both custom agent designs and automated designs from Agent Bricks

• Evaluation data and metrics
• Prompt optimization based on evaluation data

• Model fine-tuning, including both custom fine-tuning and automated tuning by Agent Bricks

GenAI vs. ML vs. deep learning

The boundaries between generative artificial intelligence (GenAI), machine learning (ML), and deep learning (DL) can be fuzzy. This guide focuses on GenAI, but the following Databricks platform features support ML, deep learning, and GenAI:

• Model Serving supports ML, deep learning, and GenAI models. You might use it for GenAI batch inference and to deploy agents or fine-tuned models using custom model serving.

• Serverless GPU compute and GPU-enabled Databricks Runtime for Machine Learning can be used to train and fine-tune ML, deep learning, and GenAI models.

• MLflow experiment tracking can be used to track both classic ML and GenAI experiments and runs.
• Databricks Feature Store can be used to manage and serve structured data for both classic ML and GenAI.