Admin Console login issue
Hello All. I am new to databricks. I am using Free trial subscription as part of my learning. I have created a premium databricks workspace. But when I am trying to login to admin console to setup unity catalog I am not getting the "Manage…
Optimizing Databricks Cluster Usage in DataFactory Pipeline
I'm working with various Data bricks notebooks to read, transform, and write data to and from ADLS. These notebooks are connected in a DataFactory pipeline as follows: NB1 -->NB2-->NB3--->NB4 I've set up a connection from DataFactory to…
Unable to login into accounts.azuredatabricks.net or manage account from databricks
I have a azure subscription "Pay-as-you-go". I create a resource group and created a databricks resource. When I launch Databricks worksace I am able to login into my databricks portal. But then I try to…
Databricks out of memory issue on driver causing it to restart
I am having a strange issue with relatively simple job that is causing the driver to restart. Before anyone suggests it: There are no displays, collects, conversions to pandas, and the data size is tiny (a little over 3000 rows of data). The cluster…
How to fix "The specified path already exists" issue raised by Databricks Delta Live Table pipeline executions
Hello, I have several DLT pipelines that are generating an exception "The specified path already exists". The exception indicates that there is an issue with the internal checkpoint file of the DLT streaming…
![](https://techprofile.blob.core.windows.net/images/VfQFAmOikEWfBHko2XlWTA.png?8D7F33)
Databricks workspace deployment failed with BadRequest error
Databricks workspace creation and deployment failed with the following BadRequest error {"error":{"code":"BadRequest","message":"BAD_REQUEST: Account per user limit exceeded, each user is only allowed to join…
![](https://techprofile.blob.core.windows.net/images/VfQFAmOikEWfBHko2XlWTA.png?8D7F33)
Azure data brick performance tuning
I have azure data brick notebook with 40 task(pyspark) dealing with 2.5 million database record (azure sql) and it took 4.5 hours to run on acceptance env, however it took only 35 min on test env with 1 million database record. Could you please advise…
![](https://techprofile.blob.core.windows.net/images/hAFzqf2_AwAAAAAAAAAAAA.png?8DB62B)
Cannot create MAP visualization in dashboard in Databricks
I want to create a MAP visualization in the Databricks dashboard, but there are no options for map visualization. Map visualization is created in the SQL editor but when I tried to import that visualization to the dashboard, It did not show that…
Performance Issue with UC Read from Federated SQL Table vs JDBC Read from SQL Server
Hi everyone, I'm currently facing a significant performance issue when comparing the execution times of a query sent through JDBC versus a similar query executed through Databricks SQL (using Unity Catalog to access a federated SQL table). JDBC Query: …
Unable to deploy the Notebooks to the Databricks workspace when we use Databricks in East US2 region.
Hell Team, When i am trying to deploy notebooks from Azure Repo to Databricks workspace using Databricks deployment task we are getting the below Error. This error is specifically into the East US2 region only . ##[debug]Pushing file…
Failure starting repl. Try detaching and re-attaching the notebook.
I( have been facing issues related to cluster. the moment i run a command it fails giving me the following error message "Failure starting repl. Try detaching and re-attaching the notebook." Please help. I tried restarting and changing the…
Best Practices for Creating an Identical Development Environment to Production
I have a production environment with Azure Data Factory, Databricks, and Storage, and I need to replicate all of this for a development environment. What are the best practices for achieving this? I'm facing challenges with Azure Data Factory; I've tried…
![](https://techprofile.blob.core.windows.net/images/5kfUQwbXfkqnY7o5wQqfdA.png?8DAA55)
Unable to create databricks workspace in Azure
Hi, I have been trying to create azure data bricks workspace, on my azure free trail account, however reeceiving an error stating "The resource write operation failed to complete successfully, because it reached terminal provisioning state…
![](https://techprofile.blob.core.windows.net/images/VfQFAmOikEWfBHko2XlWTA.png?8D7F33)
Enablement of Unity Catalog
Our organization is a network partner with Databricks, and we have monthly premium tier credits for Azure services. I currently hold a contributor role; however, to enable Unity Catalog on Databricks, I require Azure Databricks admin access or Azure…
![](https://techprofile.blob.core.windows.net/images/VfQFAmOikEWfBHko2XlWTA.png?8D7F33)
Automated testing in azure datalake
I want to automate the testing process of azure data lake resources, especially azure data factory and azure SQL database. I tried creating some test cases in azure devops test plans and build pipelines for testing SQL database schema, mappings etc. I…
Can't login Databricks
I used the account role as owner, global administrator. And I checked the resource and Databricks workspace regions match. There is exactly one resource. The workspace you are trying to access does not exist in this Azure region, or your account…
Azure Databricks Workspace Unable to Access Workspace Storage Account
Databricks appears unable to access the workspace storage account. When attempting to display notebook cell results, an error returns: "Failed to upload command result to DBFS. Error message: dbstoragekbbxxxxxxxxx.dfs.core.windows.net: Name or…
![](https://techprofile.blob.core.windows.net/images/VfQFAmOikEWfBHko2XlWTA.png?8D7F33)
Databricks runtime version 14.3 throwing error
Hello, I have multiple databricks job reporting failures when I use Databricks runtime version 14.3 (14.3.x-scala2.12) The same job passes if I use Databricks runtime version 12.2 Attached is the error seen in stderr.log of the cluster, also mentioned…
![](https://techprofile.blob.core.windows.net/images/VfQFAmOikEWfBHko2XlWTA.png?8D7F33)
Databricks Excel ODBC driver bug
Hello! I might have experienced a bug with the ODBC driver. We have an issue where given certain priviledges in databricks, the ODBC driver is unable to show any schemas/tables. When we click the 'expand' button on any catalog in the list (of which we…
![](https://techprofile.blob.core.windows.net/images/VfQFAmOikEWfBHko2XlWTA.png?8D7F33)
API source data and intermediatory storage
I am moving data from a highly secure system with compliance as report as service (RaaS) as I discussed in the questions below.…