This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Databricks provides an ecosystem of tools to help you develop applications and solutions that integrate with Azure Databricks and programmatically manage Databricks resources and data.
This article provides an overview of these tools and recommendations for the best tools for common developer scenarios.
The following table provides a list of developer tools provided by Databricks.
| Tool | Description |
|---|---|
| Authentication and authorization | Configure authentication and authorization for your tools, scripts, and apps to work with Azure Databricks. |
| Databricks Connect | Connect to Azure Databricks using popular integrated development environments (IDEs) such as PyCharm, IntelliJ IDEA, Eclipse, RStudio, and JupyterLab. If you are using Visual Studio Code, Databricks recommends the Databricks extension for Visual Studio Code, which is built on top of Databricks Connect, as it provides additional features to enable easier configuration. |
| Databricks extension for Visual Studio Code | Connect to your remote Azure Databricks workspaces from the Visual Studio Code integrated development environment (IDE). |
| PyCharm Databricks plugin | Configure a connection to a remote Databricks workspace and run files on Databricks clusters from PyCharm. This plugin is developed and provided by JetBrains in partnership with Databricks. |
| Databricks SDKs | Automate Azure Databricks from code libraries written for popular languages such as Python, Java, Go, and R. Instead of sending REST API calls directly using curl/ Postman, you can use an SDK to interact with Databricks using a programming language of your choice. The Databricks SDKs support the complete REST API and provide other features, including unified authentication and pagination, that make them easy to use and extend to cover many scenarios. |
| SQL drivers and tools | Connect to Azure Databricks to run SQL commands and scripts, interact programmatically with Azure Databricks, and integrate Azure Databricks SQL functionality into applications written in popular languages such as Python, Go, JavaScript and TypeScript. |
| Databricks CLI | Access Azure Databricks functionality using the Databricks command-line interface (CLI). The CLI wraps the Databricks REST API, so instead of sending REST API calls directly using curl or Postman, you can use the Databricks CLI to interact with Databricks. |
| Databricks Asset Bundles | Implement industry-standard development, testing, and deployment (CI/CD) best practices for your Azure Databricks data and AI projects using Databricks Asset Bundles (DABs). |
| Databricks Terraform provider and Terraform CDKTF for Databricks | Provision Azure Databricks infrastructure and resources using Terraform. |
| CI/CD tools | Integrate popular CI/CD systems and frameworks such as GitHub Actions, Jenkins, and Apache Airflow. |
Tip
You can also connect many additional popular third-party tools to clusters and SQL warehouses to access data in Azure Databricks. See the Technology partners.
The following table outlines Databricks tool recommendations for common developer scenarios.
| Tool | Usage recommendation |
|---|---|
| Databricks extension for Visual Studio Code PyCharm Databricks plugin For other IDEs, use Databricks CLI with Databricks Connect |
- Interactive development and debugging from a local IDE |
| Databricks CLI | - Direct interaction with Databricks from the command line - Shell scripting - Experimentation - Invoke the REST API directly - Manage local authentication profiles - Sync code from the IDE to the Databricks workspace |
| Databricks Asset Bundles (a feature of the CLI) | - Manage workflows and deploy projects to Databricks - Apply CI/CD best practices - Co-version, co-author, co-deploy your resources and assets as one unit - Supports the most common resources |
| Databricks Terraform provider | - Infrastructure as code, CI/CD - Administer and create workspaces, catalogs, metastores, and enforce permissions - Guarantee environment portability and disaster recovery - Many supported resources |
| Databricks Python SDK Databricks Java SDK Databricks Go SDK Databricks R SDK |
- Application development - Integrate with existing deployment systems - Create custom Databricks workflows and new web services |
| Databricks REST API | - Automate processes where an SDK in your preferred programming language is not available - Advanced scenarios only - Almost all Databricks resources are available |
Training
Module
Explore developer tools for workspace interaction - Training
Learn how you can interact with the Azure Machine Learning workspace. You can use the Azure Machine Learning studio, the Python SDK (v2), or the Azure CLI (v2).
Certification
Microsoft Certified: Azure Developer Associate - Certifications
Build end-to-end solutions in Microsoft Azure to create Azure Functions, implement and manage web apps, develop solutions utilizing Azure storage, and more.