Use IDEs with Azure Databricks

You can connect many popular third-party integrated development environments (IDEs) to an Azure Databricks cluster. This allows you to write code on your local development machine by using the Spark APIs and then run that code as jobs remotely on an Azure Databricks cluster.

Name Use this when you want to…
dbx by Databricks Labs Use an open source tool which is designed to provide functionality for rapid development lifecycle and continuous integration and continuous delivery/deployment (CI/CD) on the Azure Databricks platform. dbx simplifies jobs launch and deployment processes across multiple environments. It also helps to package your project and deliver it to your Azure Databricks environment in a versioned fashion. Designed in a CLI-first manner, it is built to be actively used both inside CI/CD pipelines and as a part of local tooling (such as local IDEs).
IDE usage with dbx Use third-party integrated development environments (IDEs), such as Visual Studio Code, PyCharm, IntelliJ IDEA, and Eclipse, along with dbx by Databricks Labs, to automate Azure Databricks.
dbx sync Perform real-time synchronization of changes to files on your local development machine with their corresponding files in your Azure Databricks workspaces.

Note

Databricks also supports a tool named Databricks Connect. However, Databricks plans no new feature development for Databricks Connect at this time. Also, Databricks Connect has several limitations.