Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,175 questions
1 answer
Salesforce connector : I am getting TaskCanceledException error while doing full load for Service__c and Service_Step__c entity.
Hello, We have recently transitioned to the Salesforce Bulk API 2.0 Connector in Azure Data Factory (ADF) for our integration processes. As part of our use case, we perform a full data load for all Salesforce entities once a month. However, we are…
asked 2024-10-01T07:32:09.8266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 0%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDU%3C/text%3E%3C/svg%3E)
Dhaval Uday Shah
0
Reputation points
commented 2024-10-01T22:18:03.5+00:00
Dhaval Uday Shah
0
Reputation points
0 answers
Can't login to Azure Databrick account console
Hi, I'm a Global Admin but can't login to the databricks account. Tried with different users, different browsers, can't login to the console. https://accounts.azuredatabricks.net/ Please, any help? ,
asked 2024-10-01T15:55:27.73+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(284.8, 82%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EOS%3C/text%3E%3C/svg%3E)
Ofir Shaty
0
Reputation points
commented 2024-10-01T19:58:18.35+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
1,575
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Loading data from Databricks pyspark dataframe into Snowflake
Hello, I am trying to load a large dataframe (~ 50 million records) from Databricks into Snowflake. I am using the below code and it takes more than 2 hours to load the data into the target Snowflake table. Is there a different configuration/approach…
asked 2024-09-27T19:43:16.0933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 78%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAT%3C/text%3E%3C/svg%3E)
Alok Thampi
111
Reputation points
edited an answer 2024-10-01T15:55:43.46+00:00
PRADEEPCHEEKATLA-MSFT
89,296
Reputation points Microsoft Employee
0 answers
How to achieve the transaction replications setup from many on prem sql in azure.server
Currently we are using transactional replication to bring real time data from on prem sql source servers to azure db destination. I need to migrate this setup in azure with real time availability of data. I used the cdc ad event triggered methods but…
asked 2024-10-01T02:12:59.6466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(41.6, 44%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJL%3C/text%3E%3C/svg%3E)
Jothi Lakshmi P M
0
Reputation points
commented 2024-10-01T09:14:52.54+00:00
PRADEEPCHEEKATLA-MSFT
89,296
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
ADF pipeline to read the data from UC table to adls gen2 account
Hello Team, We have a requirement to create Azure Datafactory pipeline to read the data from UC table, access on the table is provided ( to Azure Datafactory Managed Identity) and copy the data into adls gen2. Is there a way or article to implement this?…
asked 2024-04-11T19:05:05.9733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(51.2, 77%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAG%3C/text%3E%3C/svg%3E)
Ashwini Gaikwad
130
Reputation points
edited a comment 2024-10-01T08:27:53.21+00:00
Kuldeep Singh
16
Reputation points
1 answer
Notebook IP address blocked by workspace IP ACL
Hello, I have a databricks workspace for which I have defined an IP address ACL so that it can be reached by selected addresses (our corporate network). I am executing some python code on a notebook in the workspace. The code is trying to connect to a…
asked 2024-09-25T10:47:30.6533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 12%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERP%3C/text%3E%3C/svg%3E)
Repetti Pierangelo
20
Reputation points
commented 2024-10-01T06:39:09.2733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
10,330
Reputation points Microsoft Vendor
1 answer
Databricks Excel ODBC driver bug
Hello! I might have experienced a bug with the ODBC driver. We have an issue where given certain priviledges in databricks, the ODBC driver is unable to show any schemas/tables. When we click the 'expand' button on any catalog in the list (of which we…
asked 2024-08-20T08:23:14.69+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 50%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAN%3C/text%3E%3C/svg%3E)
ADM Nick Sørensen
0
Reputation points
edited a comment 2024-10-01T06:01:23.91+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 25%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Ashley
1
Reputation point
3 answers
One of the answers was accepted by the question author.
Processed HTTP request failed. Failed to connect databricks
I have also verify it through the access token but still on the same error. Please look into it.
asked 2024-09-25T09:05:59.4866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 55.00000000000001%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBS%3C/text%3E%3C/svg%3E)
Balwant Singh Chouhan
10
Reputation points
accepted 2024-10-01T05:27:26.52+00:00
Balwant Singh Chouhan
10
Reputation points
0 answers
Need for volumes in Databricks
Why do we need Volumes when we can access the location using external locations? The doc says it is to add governance, but we can already govern using external locations. So, why add another layer of governance? I am guessing that instead of giving…
asked 2024-09-25T06:47:42.8166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 67%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDS%3C/text%3E%3C/svg%3E)
Dhruv Singla
105
Reputation points
commented 2024-09-30T18:27:08.28+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
15,685
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Unable to Create Storage Container and Delete Storage Account in Azure Portal (Access Denied Error)
I'm encountering an issue in the Azure portal where I can't create a new storage container or delete an existing storage account.When attempting to create a storage container named "demo," I receive the following error: "Failed to create…
asked 2024-09-30T13:55:58.4233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 15%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV0%3C/text%3E%3C/svg%3E)
vikranth-0706
285
Reputation points
accepted 2024-09-30T15:47:15.4433333+00:00
vikranth-0706
285
Reputation points
4 answers
JDBC\ODBC connection to Azure Dataricks fails
Hi, I referred the following links in net and tried all, but still I am not connecting to Azure Databricks database https://docs.databricks.com/integrations/bi/power-bi.html https://docs.databricks.com/integrations/bi/jdbc-odbc-bi.html …
asked 2020-10-19T12:22:43.827+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 40%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
Rajesh
36
Reputation points
commented 2024-09-30T11:32:59.8566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 1%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Sai Kiran Bandi (LTIMINDTREE LIMITED)
0
Reputation points Microsoft Vendor
0 answers
How can you access Azure resources using user-assigned managed identities within Azure Databricks?
We're trying to move to using managed identities where we can. Is it possible to use user-assigned managed identities to access resources such as sql server databases, application insights, log analytics etc. through Azure Databricks? We thought added…
asked 2024-09-24T00:51:08.3666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 15%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAM%3C/text%3E%3C/svg%3E)
Aatish Master
0
Reputation points
commented 2024-09-30T04:38:06.26+00:00
PRADEEPCHEEKATLA-MSFT
89,296
Reputation points Microsoft Employee
1 answer
Enabling Azure PIM Disables user within DataBricks
We have successfully stood up Azure Databricks in our tenant. We are leveraging SCIM and User Provisioning to grant our users SSO access into DataBricks. We are trying to layer in an additional layer of security to meet our current security standards by…
asked 2023-09-01T14:55:42.55+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 66%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Smileyville
1
Reputation point
answered 2024-09-30T03:21:29.3066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 95%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
Pakala, Sharada
0
Reputation points
1 answer
How to read delta/csv files in ADLS thru Databricks SQL Warehouse
I am trying to read a csv/delta file in ADLS thru Databricks SQL Warehouse. I am trying DSN less connection string option, as per the below article https://stackoverflow.com/questions/65334114/is-there-a-way-to-connect-to-databricks-using-ado-net I have…
asked 2024-09-20T22:51:30.9433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 56.00000000000001%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPP%3C/text%3E%3C/svg%3E)
Prabhakaran Paulraj
0
Reputation points
commented 2024-09-29T07:14:49.5366667+00:00
Vinodh247
19,976
Reputation points
1 answer
One of the answers was accepted by the question author.
differnce between synapse and databricks
What are major difference between synapse and data bricks and under what circumstances one should use synapse and vice versa
asked 2021-10-12T08:45:54.76+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 34%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Shivam Ramola
66
Reputation points
commented 2024-09-27T09:33:53.17+00:00
Suketu Nayak
10
Reputation points
0 answers
Streaming upserts constantly report mysterious log, are very slow
Our streaming batch upserts are very slow and constantly report: 20/12/03 23:38:04 WARN RetryTolerableRenameFSDataOutputStream: Failed to rename temp file…
asked 2020-12-03T23:40:38.943+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(169.60000000000002, 22%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAH%3C/text%3E%3C/svg%3E)
Alex Hoffer
1
Reputation point
commented 2024-09-27T09:09:36.51+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 38%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
Lakhs
0
Reputation points
0 answers
Parallel Writing of Data to ADLS Delta Storage Causes "The specified path already exists.", 409, PUT" Error
I have a Spark (Spark 3.1.2, Scala 2.12) application that reads json records from a table. The records are distributed across cluster executors. In the application, I have used foreach function to loop through the records in the table. For each record,…
asked 2023-11-06T19:54:40.9533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 7.000000000000001%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EOA%3C/text%3E%3C/svg%3E)
Orowole, Ayebogbon-XT
0
Reputation points
commented 2024-09-27T09:08:42.7633333+00:00
Lakhs
0
Reputation points
1 answer
Not able to see Manage Account in databricks worksapce
I am the owner of a databricks workspace. I want to create a metastore but I am not able to see Manage Account in databricks worksapce. I have created a premium workspace and I have a trial subscription.
asked 2024-09-20T18:19:26.2833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 10%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
Muhammad Riyaz Ahsan
0
Reputation points
commented 2024-09-27T03:54:07.0866667+00:00
PRADEEPCHEEKATLA-MSFT
89,296
Reputation points Microsoft Employee
2 answers
Azure databricks integration with Github Enterprise
I am having DevOps configured with Azure databricks but now we are migrating from DevOps to Github Enterprise. To point our Azure databricks with Github enterprise repository from Azure databricks workspace post linking the Github encounter the below…
asked 2024-09-19T11:28:27.61+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 11%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERT%3C/text%3E%3C/svg%3E)
Rohitkumar Tatiya
0
Reputation points
commented 2024-09-27T03:53:22.3+00:00
PRADEEPCHEEKATLA-MSFT
89,296
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Data Factory Data flow Error Cluster creation failed
I have a data factory pipeline that's been running for a year without any issues then suddenly we received an error Internal Server Error In Synapse batch operation: 'Cluster creation failed due to unexpected error. Please retry' This error came from a…
asked 2024-02-28T14:31:00.75+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 40%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECC%3C/text%3E%3C/svg%3E)
Colinn Calaguas
45
Reputation points
commented 2024-09-26T08:33:05.4733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(9.6, 95%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
Ashir Khan
0
Reputation points