How to generate Bearer Token for Azure Databricks with private networking?

Question

I want to deploy Some python notebooks in Azure Databricks Workspace.

The ADB workspace has two private endpoints (Ui and auth), and its public network access is disabled.

I have a self-hosted ubuntu agent for running the pipeline in which i have configured azure.databricks.cicd.tools module for ADB.

I have created a PowerShell script, to generate a bearer token using ADB enterprise application.

Here is the script below:

When public network access in enabled, then bearer token is getting generated successfully. But when i disable the public access and use private endpoint, it is not generating the bearer token.

Please provide some solutions to this

Answer 1

Hello Smriti, Smriti,

When you are using Self hosted ubuntu agent, can you please confirm if your agent has access to the private endpoint?

If it doesn't have access, please configure your agent to use a Vnet and a private endpoint for databricks. Then this will allows your self hosted aganet to communicate with databricks privately.

Once you have configured your agent to use Vnet and private endpoint, please modify your PS script to use the private endpoint instead of the public url.

I hope this helps.