Azure Databricks AI capabilities

Azure Databricks provides a platform for building, evaluating, deploying, and monitoring AI applications (AI apps). It brings together a suite of tools that tackle the challenges of developing enterprise-grade AI apps. Azure Databricks integrates with popular open source frameworks, adding enterprise-grade governance, observability, and operational tooling, collectively known as LLMOps.

This page lists major features for AI, organized by AI workflow stages.

Query AI

Azure Databricks makes state-of-the-art AI models from top model providers readily available through Databricks-hosted Foundation Models. You can also query models from external providers and your custom models. All of these models can be queried through UI, API/SDK, and SQL interfaces. This optionality lets you use AI for all use cases---general chat, complex agents, automated data pipelines, interactive data analytics, and more.

Method Features
UI
  • For development, AI Playground provides a UI for querying available AI models and agents.
  • For testing, your agents and apps can be queried and evaluated by domain experts using the Review App.
  • For production, your apps can be hosted with a UI using Databricks Apps for use inside your organization. For external apps, you can power user-facing apps using Databricks APIs.
API and SDK
SQL
  • AI functions provide task-specific and general-purpose SQL functions for querying models and agents.

Build AI

Azure Databricks provides a flexible set of tools for building AI apps, agents, tools, and models. These include UI and code-based frameworks, all of which can optimize AI systems based on your data. You can leverage any open-source AI framework and can integrate custom tools and MCP servers.

Category Features
Apps
Agents
Tools
Models and prompts

Prepare and serve data

Azure Databricks simplifies data for AI by unifying governance of traditional data and AI workloads. With all data managed under Unity Catalog with fine-grained access controls, it is easy to adjust data engineering and AI boundaries to fit your organization. Data can be prepared for AI using any data engineering tools such as Lakeflow pipelines. A table in Unity Catalog can be served for AI, using a vector index for unstructured data or a feature table for structured data.

Type of data Features
Unstructured (text, images, etc.)
  • AI Search automatically indexes your knowledge base at scale for semantic or hybrid search.
Structured (tables)
  • Serverless SQL lets you integrate tables and SQL queries into your AI app for analytics or transformations.
  • Genie agents can be used in multi-agent systems to answer natural language queries about your structured data.
  • Online feature stores provide real-time feature access for your AI app.

Deploy and serve AI

Azure Databricks provides production-ready serving systems for AI apps, agents, and models backed by Databricks Apps. These scalable deployments can be used for both real-time serving and batch inference. All deployments integrate with observability and evaluation and monitoring tooling.

Development framework Deployment and serving
Databricks Apps Apps and agents can be deployed using Databricks Apps, which provides UI- and API-based deployment.
Knowledge Assistant Knowledge Assistant automates deployment of your agent to Model Serving endpoints.

Trace, evaluate, and monitor AI

Azure Databricks provides managed MLflow for AI observability, evaluation, and monitoring. Open-source APIs make integration and portability simple, while the managed service provides production-ready endpoints. Databricks-managed MLflow can be used for AI apps and agents hosted on Databricks and hosted elsewhere.

Category Features
Tracing and observability
  • MLflow Tracing allows you to instrument your AI agents and apps to collect telemetry and observability data for evaluation, production monitoring, and auditing.
Evaluation
Monitoring
  • Production monitoring lets you measure quality on production traces, using the same judges and scorers from development-time evaluation.
Human feedback

LLMOps

Azure Databricks provides a full suite of tools for AI operations, or "LLMOps." Unified governance of data and AI assets under Unity Catalog simplifies and secures deployment of AI across an organization. AI Gateway simplifies managing models from many AI model providers. MLflow and Declarative Automation Bundles provide versioning and infrastructure-as-code for implementing robust LLMOps processes.

Category Features
Data and AI asset governance Unity Catalog provides unified governance for data and AI assets. Data assets include files, tables, vector indexes, and feature stores. AI assets include models, tools, and connections for MCP servers and other APIs.
Model endpoint governance AI Gateway provides central governance and monitoring for AI model endpoints.
Prompt versioning MLflow provides a Prompt Registry, as well as experiment tracking.
Infrastructure as code MLOps Stacks, which is built on top of Databricks Assets Bundles, provides code-based management and deployment of infrastructure and workflows.

Open source support

Azure Databricks provides full support for the rapidly growing open-source ecosystem for AI.

For development, you can use any open-source framework and deploy it using Databricks Apps. Databricks services can be used by third-party AI tools and apps by using MCP servers or the REST API or SDKs.

For observability, evaluation, and monitoring, MLflow Tracing provides native autologging for 20+ open-source AI frameworks, and you can add custom tracing to any other frameworks or code. The traces can then be used with MLflow Evaluation and production monitoring. The traces follow OpenTelemetry trace specs and can be exported to third-party tools.

Additional resources

  • Last updated on