Unable to access airflow server URL in Databricks

Question

I am using Databricks web terminal to install airflow and access the server. I think I have successfully set up the environment but I am unable to access the airflow server (UI).

Hi,

I was learning how to integrate Databricks with Airflow

I have followed the steps as mentioned in this article :

https://learn.microsoft.com/en-us/azure/databricks/workflows/jobs/how-to/use-airflow-with-jobs

I have python 3.8.10 as mentioned in the article.

In the above screenshot I have initiated db and created user admin and I have installed all the necessary packages as well.

Note: I have cropped email and password for security issues.

The above screenshot says the server is initiated but how can I access it. In local I assume we can access it via https://localhost:8080 what about in Databricks?.

Kindly do the needful!!!

Answer 1

@Rohit Dobbanaboina - Thanks for the question and using MS Q&A platform.

As metioned in the official documentation: Orchestrate Azure Databricks jobs with Apache Airflow

You can access locally in [http://localhost:8080/]. Credentials are exposed in the command line (normally it is admin) while installing the Airflow Azure Databricks integration.

You can configure an Azure Databricks connection using PAT token as metioned here: Create an Azure Databricks personal access token for Airflow.

Your Airflow installation contains a default connection for Azure Databricks. To update the connection to connect to your workspace using the personal access token you created above:

1. In a browser window, open [http://localhost:8080/connection/list/].
2. Under Conn ID, locate databricks_default and click the Edit record button.
3. Replace the value in the Host field with the workspace instance name of your Azure Databricks deployment.
4. In the Extra field, enter the following value:

{"token": "PERSONAL_ACCESS_TOKEN"}

Replace PERSONAL_ACCESS_TOKEN with your Azure Databricks personal access token.

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.