How to connect Databricks to Storage account in Azure when I have no access to app registriation
How to connect Databricks to Storage account in Azure when I have no access to app registriation. Does this mean I can only go to my administrator to fix this?
How to connect On-Prem FTP server using Azure Databrick
There are some csv/xlsx files in On-Prem FTP Server which Azure Databricks need to connect and load it to Delta table. Please advise What are the pre-requisite to connect On-Prem FTP server. Is there any firewall/IP need to be whitelisted. Any other…
How to Setup Azure OpenAI for Databricks running into error Error code: 403 - {'error': {'code': '403', 'message': 'Access denied due to Virtual Network/Firewall rules.'}}
After creating a new resource for Azure OpenAI service , We ran into this error not able to access OpenAI via api_key and endpoint (private) due to the error message indicates that access is denied due to Virtual Network/Firewall rules. How can we…
how to resolve this error while getting data from databricks from power bi.Not able to load table in power bi
DataSource.Error: ODBC: ERROR [HY000] [Microsoft][DSI] (20039) Cannot store ""."".""."REMARKS" value in temporary table without truncation. (Column metadata implied a maximum of 512 bytes, while provided value is…
Databricks Libraries Installation SSL Error
HELP ME TO SOLVE THIS ERROR- PLEASE ELABORATE IN DETAIL... Library installation attempted on the driver node of cluster 0606-104958-63c5i7so and failed. Please refer to the following error message to fix the library or contact Databricks support. Error…
Adding Widgets in SQL Notebooks for Data Warehouse
How can I add widgets for SQL using notebooks in Data Warehouse?
Still being billed for Azure after I cancelled it
I signed up for the free Azure and then I cancelled the subscription, but I'm still receiving bills. How do I go about removing everything off my name?
Get data from Azure SQL Database Managed Instance(Linked Service) using Synapse notebook spark pool
I have a linked service from Azure SQL Database Managed Instance and I want to get the data via Synapse spark pool In simple words I want to connect to the Azure SQL Database Managed Instance(Linked Service) and get the data via Synapse Notebook spark…
Joining huge delta tables in Databricks
Hello, I am trying to join few delta tables as per the code below. select <applicable columns> FROM ReportTable G LEFT JOIN EKBETable EKBE ON EKBE.BELNR = G.ORDER_ID LEFT JOIN PurchaseOrder POL ON EKBE.EBELN = POL.PO_NO The PurchaseOrder table…
Error with Create Table USING DELTA LOCATION in training exercise
In the exercise https://microsoftlearning.github.io/mslearn-databricks/Instructions/Exercises/03-Delta-lake-in-Azure-Databricks.html the line of code spark.sql("CREATE TABLE AdventureWorks.ProductsExternal USING DELTA LOCATION…
How to achieve the transaction replications setup from many on prem sql in azure.server
Currently we are using transactional replication to bring real time data from on prem sql source servers to azure db destination. I need to migrate this setup in azure with real time availability of data. I used the cdc ad event triggered methods but…
Can't login to Azure Databrick account console
Hi, I'm a Global Admin but can't login to the databricks account. Tried with different users, different browsers, can't login to the console. https://accounts.azuredatabricks.net/ Please, any help? ,
Notebook IP address blocked by workspace IP ACL
Hello, I have a databricks workspace for which I have defined an IP address ACL so that it can be reached by selected addresses (our corporate network). I am executing some python code on a notebook in the workspace. The code is trying to connect to a…
I want to give a static public IP address to my Databricks clusters
I could go over the "private" Microsoft network backbone using something like ExpressRoute. Connecting to a firewall outside of Databricks. But for now I just need to connect over the public internet to build a proof of concept. Would need to…
Loading data from Databricks pyspark dataframe into Snowflake
Hello, I am trying to load a large dataframe (~ 50 million records) from Databricks into Snowflake. I am using the below code and it takes more than 2 hours to load the data into the target Snowflake table. Is there a different configuration/approach…
ADF pipeline to read the data from UC table to adls gen2 account
Hello Team, We have a requirement to create Azure Datafactory pipeline to read the data from UC table, access on the table is provided ( to Azure Datafactory Managed Identity) and copy the data into adls gen2. Is there a way or article to implement this?…
Databricks Excel ODBC driver bug
Hello! I might have experienced a bug with the ODBC driver. We have an issue where given certain priviledges in databricks, the ODBC driver is unable to show any schemas/tables. When we click the 'expand' button on any catalog in the list (of which we…
Processed HTTP request failed. Failed to connect databricks
I have also verify it through the access token but still on the same error. Please look into it.
Need for volumes in Databricks
Why do we need Volumes when we can access the location using external locations? The doc says it is to add governance, but we can already govern using external locations. So, why add another layer of governance? I am guessing that instead of giving…
Unable to Create Storage Container and Delete Storage Account in Azure Portal (Access Denied Error)
I'm encountering an issue in the Azure portal where I can't create a new storage container or delete an existing storage account.When attempting to create a storage container named "demo," I receive the following error: "Failed to create…