Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,091 questions
0 answers
How to write Azure Databrick code to restrict RLS and automatically add to AD Group
Hi there, I would like to ask how I can write a code to restrict Row-Level-Security in Azure Databrick. Let me describe the situation: 1. Input - I have an Excel file named External_Suppliers_ADGroup2 that uploaded into the Databricks - …
asked 2024-07-31T17:53:56.45+00:00
Daniel Hoang Nguyen
0
Reputation points
asked 2024-07-31T17:53:56.45+00:00
Daniel Hoang Nguyen
0
Reputation points
1 answer
Can't access the Azure Databricks Accounts Console
Hi there, When I tried to log on to https://accounts.azuredatabricks.net/login, I encountered an error. It says: Selected user account does not exist in tenant 'Microsoft Services' and cannot access the application…
asked 2024-07-30T12:22:12.19+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 39%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZZ%3C/text%3E%3C/svg%3E)
Ziwei Zeng
0
Reputation points
commented 2024-07-31T13:04:54.9233333+00:00
Ziwei Zeng
0
Reputation points
1 answer
Copy data from UTF-8 encrypted file using Azure data factory
Hi All Any suggestions, Need to copy data from UTF-8 encrypted file. if source contains any UTF-16 file then need to ignore of throw Error. Using Azure data factory, any other service can do this also suggest. Regards Ashutosh
asked 2024-07-30T16:11:11.5233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 9%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAP%3C/text%3E%3C/svg%3E)
Ashutosh Prashar
0
Reputation points
answered 2024-07-31T12:39:05.98+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 89%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
AzureAce
27
Reputation points
0 answers
Azure Databricks login issue and not able to call jobs from ADF as well
Getting the same window
asked 2024-07-31T06:13:39.93+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 7.000000000000001%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Ajit Sharma
0
Reputation points
edited a comment 2024-07-31T11:48:07.39+00:00
Ajit Sharma
0
Reputation points
0 answers
Transfer / pass a parameter outside databricks workflow
Hello, in a synapse pipeline, through a web activity i call / execute a databricks workflow job. The job has couple of notebooks that execute depending on some logic inside the job itself. When the notebooks finish, i want to output some…
asked 2024-07-31T11:09:22.9333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(182.40000000000003, 22%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBP%3C/text%3E%3C/svg%3E)
Boris Perchinkov
0
Reputation points
asked 2024-07-31T11:09:22.9333333+00:00
Boris Perchinkov
0
Reputation points
2 answers
SAS token generation by Databricks to access CSV files from ADLS container folder
Hi Team, There are some csv files zips inside the ADLS container folder. These zip files need to be downloaded for data correction. Downloading the file requires SAS token embedded with zip file path. Databricks has been used to generate the token and…
asked 2024-07-17T05:08:24.3033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 91%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Subhadip Roy
21
Reputation points
commented 2024-07-31T10:27:18.35+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
Nehruji R
4,846
Reputation points Microsoft Vendor
1 answer
Creating a Zip File from Blob Storage Using Python in Azure Databricks
Hello, I am working on a task where I need to create a zip file for multiple files stored in blob storage, without having to read the files again or using local storage. I am using Python in Azure Databricks and would like to leverage its capabilities…
asked 2024-07-24T06:12:46.7266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 13%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECM%3C/text%3E%3C/svg%3E)
Chahat Malik
0
Reputation points
commented 2024-07-31T10:17:01.61+00:00
Nehruji R
4,846
Reputation points Microsoft Vendor
0 answers
Unable to connect to Azure Databricks workspace with error : "Response not successful : received status code 503"
Hi, Since today, I've been unable to connect to Azure Databricks workspaces. It was working fin yesterday, and it stil works for all my colleagues today. The error message I got when trying to log into Azure Databricks is : SSO is enabled, we have…
asked 2024-07-31T08:43:03.45+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 91%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
Mikael BURBAN
1
Reputation point
commented 2024-07-31T10:13:59.7733333+00:00
Mikael BURBAN
1
Reputation point
1 answer
One of the answers was accepted by the question author.
Increasing Public/Private subnet for databricks workspace
Is it actually possible to increase the public/private subnet for an existing databricks workspace ? I have tried changing the subnet from /26 to /24 and deligated it back to the Databricks workspace, what I have noticed is that I was able to start…
asked 2022-12-28T07:03:31.333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 96%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
veerabhadra reddy kovvuri
181
Reputation points
accepted 2024-07-31T10:04:58.91+00:00
veerabhadra reddy kovvuri
181
Reputation points
1 answer
One of the answers was accepted by the question author.
Databricks: Executor heartbeat timed out after xxx ms
I get following error in Azure Databricks Delta Live Table pipeline. I tried increasing the nodes and node type, but still getting the same error. Appreciate community help. Thanks org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED]…
asked 2024-07-31T02:35:23.6833333+00:00
Hiran Amarathunga
65
Reputation points
accepted 2024-07-31T09:07:05.2466667+00:00
Hiran Amarathunga
65
Reputation points
0 answers
Associate route table to Databricks Vnet subnets
We have a resource group for AVD/Domain resources and so we have a gateway to connect to local resources. We stood up a Databricks resource and need it to use the gateway from the original resource. We have an active peering network and can communicate…
asked 2024-07-30T19:43:57.8533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 50%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Stephen
0
Reputation points
commented 2024-07-30T22:57:41.7366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(25.6, 0%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESW%3C/text%3E%3C/svg%3E)
Silvia Wibowo
3,571
Reputation points Microsoft Employee
1 answer
Virtual Network settings on Storage account interaction with Databricks Table Monitoring
I have set up my Databricks Unity Catalog on an Azure Data Lake storage account which uses my companies virtual network to allow access. I have all privileges on my account, so I am able to create, alter or delete catalogs, schemas and tables using a…
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,956 questions
Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
1,113 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,091 questions
asked 2024-07-25T11:14:24.92+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 77%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAL%3C/text%3E%3C/svg%3E)
Asger Lillie Larsen
0
Reputation points
edited the question 2024-07-30T21:25:07.9366667+00:00
Luis Arias
6,216
Reputation points
1 answer
One of the answers was accepted by the question author.
In Explore Azure Databricks Exercise the script to setup the databricks does not work. Why?
I run the script and I receive the following error. Starting script at 07/29/2024 19:14:44 Registering resource providers... Microsoft.Storage : Registered Microsoft.Compute : Registered Microsoft.Databricks : Registered Preparing to deploy. This may…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(160, 28.000000000000004%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECG%3C/text%3E%3C/svg%3E)
2 answers
Not able to access "Manage Account" setting in my Azure Portal
Hello, I have create a free Azure account for learning. Also, I have created 2 workspaces in the databricks, but i am not getting the "Manage Account" setting under my profile section in databricks. Please find the below image thats shows, i…
asked 2024-07-27T18:02:46.8933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 11%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EK%3C/text%3E%3C/svg%3E)
Kashyap
0
Reputation points
answered 2024-07-30T05:38:21.0566667+00:00
PRADEEPCHEEKATLA-MSFT
85,981
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
How can i read Share point excel files in azure databricks notebook?
Handling sharepoint files in databricks notebook to perform some transformations.
asked 2021-09-02T06:48:34.233+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 76%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENS%3C/text%3E%3C/svg%3E)
Naveen Suvvada
21
Reputation points
commented 2024-07-29T21:50:00.2566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(48, 57.99999999999999%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Anvesh Reddy
0
Reputation points
0 answers
Azure Databricks Too Many Requests errors
We are getting many errors with loading notebooks and also now running jobs on clusters due to Databricks saying it has too many requests. For example, getting the below error message: run failed with error message Cluster '0724-103023-f2llqh3p' was…
asked 2024-07-24T10:51:26.46+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 19%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Matthieu Marshall
6
Reputation points
commented 2024-07-29T14:17:21.5466667+00:00
Matthieu Marshall
6
Reputation points
1 answer
I want to give a static public IP address to my Databricks clusters
I could go over the "private" Microsoft network backbone using something like ExpressRoute. Connecting to a firewall outside of Databricks. But for now I just need to connect over the public internet to build a proof of concept. Would need to…
asked 2024-07-25T07:03:22.8+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(182.40000000000003, 36%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EB%3C/text%3E%3C/svg%3E)
Bruce
0
Reputation points
edited the question 2024-07-29T08:54:54.6266667+00:00
KapilAnanth-MSFT
41,876
Reputation points Microsoft Employee
1 answer
Kafka Connector for Databricks
Can you forward me documentation to use Kafka Connector for Apache Spark Streaming job to connect to Azure Eventhub. I am looking for maven library version etc.
asked 2024-07-22T14:43:43.1133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(182.40000000000003, 80%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPP%3C/text%3E%3C/svg%3E)
pseudo p
0
Reputation points
commented 2024-07-29T06:50:39.3433333+00:00
PRADEEPCHEEKATLA-MSFT
85,981
Reputation points Microsoft Employee
2 answers
Append in Liquid Cluster enabled table is not completing on DBR 15.3 version
I am trying do analysis with a Partition Table and Liquid Clustered table. As per Azure Databricks recommendation, I am using DBR 15.2 to execute the code. I have created a clustered table as and using an append operation which is specified below. Few…
asked 2024-07-13T07:29:40.88+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 35%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESG%3C/text%3E%3C/svg%3E)
Sudipta Goswami
20
Reputation points
commented 2024-07-29T06:44:34.3033333+00:00
PRADEEPCHEEKATLA-MSFT
85,981
Reputation points Microsoft Employee
1 answer
Permissions Management in Azure Databricks Standard
Hi, In Azure Databricks STANDARD TIER, Is it possible to create an API token that will bot be able to access data? I'm looking to delegate my infrastructure management to a third party vendor, so I'd like them to have permissions to update clusters but…
asked 2024-07-28T13:58:38.4+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 72%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EY%3C/text%3E%3C/svg%3E)
YBE
0
Reputation points
edited an answer 2024-07-29T04:34:10.0933333+00:00
PRADEEPCHEEKATLA-MSFT
85,981
Reputation points Microsoft Employee