Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,139 questions
0 answers
Admin Console login issue
Hello All. I am new to databricks. I am using Free trial subscription as part of my learning. I have created a premium databricks workspace. But when I am trying to login to admin console to setup unity catalog I am not getting the "Manage…
asked 2024-08-30T18:35:18.15+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 84%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESG%3C/text%3E%3C/svg%3E)
Sankha Guha
0
Reputation points
edited the question 2024-08-30T18:36:27.4466667+00:00
Sankha Guha
0
Reputation points
0 answers
Optimizing Databricks Cluster Usage in DataFactory Pipeline
I'm working with various Data bricks notebooks to read, transform, and write data to and from ADLS. These notebooks are connected in a DataFactory pipeline as follows: NB1 -->NB2-->NB3--->NB4 I've set up a connection from DataFactory to…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(240, 32%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EG%3C/text%3E%3C/svg%3E)
0 answers
Unable to login into accounts.azuredatabricks.net or manage account from databricks
I have a azure subscription "Pay-as-you-go". I create a resource group and created a databricks resource. When I launch Databricks worksace I am able to login into my databricks portal. But then I try to…
Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
1,152 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,139 questions
asked 2024-08-30T16:03:18.56+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(19.2, 71%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAM%3C/text%3E%3C/svg%3E)
Anupam Mishra
0
Reputation points
asked 2024-08-30T16:03:18.56+00:00
Anupam Mishra
0
Reputation points
0 answers
Databricks out of memory issue on driver causing it to restart
I am having a strange issue with relatively simple job that is causing the driver to restart. Before anyone suggests it: There are no displays, collects, conversions to pandas, and the data size is tiny (a little over 3000 rows of data). The cluster…
asked 2024-08-30T15:57:43.52+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 57.00000000000001%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJA%3C/text%3E%3C/svg%3E)
John Aherne
141
Reputation points
asked 2024-08-30T15:57:43.52+00:00
John Aherne
141
Reputation points
0 answers
How to fix "The specified path already exists" issue raised by Databricks Delta Live Table pipeline executions
Hello, I have several DLT pipelines that are generating an exception "The specified path already exists". The exception indicates that there is an issue with the internal checkpoint file of the DLT streaming…
asked 2024-08-28T09:26:38.4266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 5%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
Slim MISSAOUI
0
Reputation points
commented 2024-08-30T12:14:46.6833333+00:00
PRADEEPCHEEKATLA-MSFT
88,051
Reputation points Microsoft Employee
0 answers
Databricks workspace deployment failed with BadRequest error
Databricks workspace creation and deployment failed with the following BadRequest error {"error":{"code":"BadRequest","message":"BAD_REQUEST: Account per user limit exceeded, each user is only allowed to join…
asked 2024-08-22T12:40:58.17+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 21%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMS%3C/text%3E%3C/svg%3E)
Mudit Surana
20
Reputation points
commented 2024-08-30T12:08:38.4033333+00:00
PRADEEPCHEEKATLA-MSFT
88,051
Reputation points Microsoft Employee
1 answer
Azure data brick performance tuning
I have azure data brick notebook with 40 task(pyspark) dealing with 2.5 million database record (azure sql) and it took 4.5 hours to run on acceptance env, however it took only 35 min on test env with 1 million database record. Could you please advise…
asked 2024-08-29T17:50:14.14+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 35%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPJ%3C/text%3E%3C/svg%3E)
Pankaj Joshi
306
Reputation points
answered 2024-08-30T06:40:15.9033333+00:00
Vinodh247
16,201
Reputation points
1 answer
Cannot create MAP visualization in dashboard in Databricks
I want to create a MAP visualization in the Databricks dashboard, but there are no options for map visualization. Map visualization is created in the SQL editor but when I tried to import that visualization to the dashboard, It did not show that…
asked 2024-08-30T05:49:11.09+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 81%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKC%3C/text%3E%3C/svg%3E)
Kalpesh Chavda
0
Reputation points
edited the question 2024-08-30T06:17:23.6533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 86%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
AmaranS
5,710
Reputation points Microsoft Vendor
0 answers
Performance Issue with UC Read from Federated SQL Table vs JDBC Read from SQL Server
Hi everyone, I'm currently facing a significant performance issue when comparing the execution times of a query sent through JDBC versus a similar query executed through Databricks SQL (using Unity Catalog to access a federated SQL table). JDBC Query: …
asked 2024-08-29T11:31:44.7033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 5%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Simonas Solovjovas
0
Reputation points
commented 2024-08-29T23:29:37.68+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Bhargava-MSFT
30,381
Reputation points Microsoft Employee
1 answer
Unable to deploy the Notebooks to the Databricks workspace when we use Databricks in East US2 region.
Hell Team, When i am trying to deploy notebooks from Azure Repo to Databricks workspace using Databricks deployment task we are getting the below Error. This error is specifically into the East US2 region only . ##[debug]Pushing file…
asked 2024-08-15T16:14:59.21+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 62%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Sateesh Kumar B
0
Reputation points
commented 2024-08-29T18:41:56.6+00:00
Bhargava-MSFT
30,381
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Failure starting repl. Try detaching and re-attaching the notebook.
I( have been facing issues related to cluster. the moment i run a command it fails giving me the following error message "Failure starting repl. Try detaching and re-attaching the notebook." Please help. I tried restarting and changing the…
asked 2023-10-12T08:45:55.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 39%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESV%3C/text%3E%3C/svg%3E)
S V Ravi Chandra Kolluri
35
Reputation points
commented 2024-08-29T08:51:12.1666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 37%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBC%3C/text%3E%3C/svg%3E)
Biaf Chalabi
0
Reputation points
2 answers
Best Practices for Creating an Identical Development Environment to Production
I have a production environment with Azure Data Factory, Databricks, and Storage, and I need to replicate all of this for a development environment. What are the best practices for achieving this? I'm facing challenges with Azure Data Factory; I've tried…
asked 2024-08-09T18:21:56.7533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 31%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Hanna
220
Reputation points
commented 2024-08-29T05:30:24.37+00:00
AnnuKumari-MSFT
32,566
Reputation points Microsoft Employee
1 answer
Unable to create databricks workspace in Azure
Hi, I have been trying to create azure data bricks workspace, on my azure free trail account, however reeceiving an error stating "The resource write operation failed to complete successfully, because it reached terminal provisioning state…
asked 2024-08-28T23:48:34.0966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 89%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Susmitha
0
Reputation points
answered 2024-08-29T03:15:02.5933333+00:00
PRADEEPCHEEKATLA-MSFT
88,051
Reputation points Microsoft Employee
1 answer
Enablement of Unity Catalog
Our organization is a network partner with Databricks, and we have monthly premium tier credits for Azure services. I currently hold a contributor role; however, to enable Unity Catalog on Databricks, I require Azure Databricks admin access or Azure…
asked 2024-08-28T12:22:32.0833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 90%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENS%3C/text%3E%3C/svg%3E)
Nidhi Singh
0
Reputation points
answered 2024-08-29T02:35:35.3333333+00:00
PRADEEPCHEEKATLA-MSFT
88,051
Reputation points Microsoft Employee
1 answer
Automated testing in azure datalake
I want to automate the testing process of azure data lake resources, especially azure data factory and azure SQL database. I tried creating some test cases in azure devops test plans and build pipelines for testing SQL database schema, mappings etc. I…
asked 2024-08-18T17:33:33.7666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 45%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Smriti
0
Reputation points
edited the question 2024-08-29T00:48:27.8866667+00:00
Bhargava-MSFT
30,381
Reputation points Microsoft Employee
0 answers
Can't login Databricks
I used the account role as owner, global administrator. And I checked the resource and Databricks workspace regions match. There is exactly one resource. The workspace you are trying to access does not exist in this Azure region, or your account…
asked 2024-08-25T03:19:36.2366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 38%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKC%3C/text%3E%3C/svg%3E)
Kelsey Chen
0
Reputation points
commented 2024-08-28T15:14:44.7466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 46%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EN%3C/text%3E%3C/svg%3E)
NIKHILANETHIKUNTA
1,350
Reputation points Microsoft Vendor
1 answer
Azure Databricks Workspace Unable to Access Workspace Storage Account
Databricks appears unable to access the workspace storage account. When attempting to display notebook cell results, an error returns: "Failed to upload command result to DBFS. Error message: dbstoragekbbxxxxxxxxx.dfs.core.windows.net: Name or…
asked 2024-08-22T16:10:55.7966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 70%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHR%3C/text%3E%3C/svg%3E)
Hutton, Randy
0
Reputation points
commented 2024-08-28T10:51:05.3933333+00:00
PRADEEPCHEEKATLA-MSFT
88,051
Reputation points Microsoft Employee
0 answers
Databricks runtime version 14.3 throwing error
Hello, I have multiple databricks job reporting failures when I use Databricks runtime version 14.3 (14.3.x-scala2.12) The same job passes if I use Databricks runtime version 12.2 Attached is the error seen in stderr.log of the cluster, also mentioned…
asked 2024-08-27T06:24:26.33+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(208, 18%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Amit Singh
60
Reputation points
commented 2024-08-28T10:40:53.38+00:00
PRADEEPCHEEKATLA-MSFT
88,051
Reputation points Microsoft Employee
1 answer
Databricks Excel ODBC driver bug
Hello! I might have experienced a bug with the ODBC driver. We have an issue where given certain priviledges in databricks, the ODBC driver is unable to show any schemas/tables. When we click the 'expand' button on any catalog in the list (of which we…
asked 2024-08-20T08:23:14.69+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 50%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAN%3C/text%3E%3C/svg%3E)
ADM Nick Sørensen
0
Reputation points
answered 2024-08-28T10:39:42.7633333+00:00
PRADEEPCHEEKATLA-MSFT
88,051
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
API source data and intermediatory storage
I am moving data from a highly secure system with compliance as report as service (RaaS) as I discussed in the questions below.…
asked 2024-08-22T04:04:42.67+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
180
Reputation points
accepted 2024-08-27T16:02:40.9733333+00:00
azure_learner
180
Reputation points