A vendor is trying to dump data in our azure blob storage and they are able to do it from the local environment but are not able to do it from their cloud platform
This is the error they are getting. I don't exactly know where the issue is.
Databricks Error (An error occurred while calling o399.ls. : org.apache.hadoop.fs.FileAlreadyExistsException: Operation failed: "This endpoint does not support BlobStorageEvents or SoftDelete. Please disable these account features if you would like to us)
I have continously occur this error, An error occurred while calling o399.ls. : org.apache.hadoop.fs.FileAlreadyExistsException: Operation failed: "This endpoint does not support BlobStorageEvents or SoftDelete. Please disable these account features…
Support Needed for Accessing Azure Translator from Azure Databricks with Identity Authentication
I am looking for support in accessing Azure Translator from Azure Databricks using Identity Authentication. Currently, I am able to access it via an Access Key, but we want to transition to Identity Authentication.
Databricks runtime version 14.3 throwing error
Hello, I have multiple databricks job reporting failures when I use Databricks runtime version 14.3 (14.3.x-scala2.12) The same job passes if I use Databricks runtime version 12.2 Attached is the error seen in stderr.log of the cluster, also mentioned…
Databricks workspace deployment failed with BadRequest error
Databricks workspace creation and deployment failed with the following BadRequest error {"error":{"code":"BadRequest","message":"BAD_REQUEST: Account per user limit exceeded, each user is only allowed to join…
Azure Databricks keeps notebooks in its cloud account free of cost?
I was reading the architecture of Databricks from the following video. And I came through this If notebooks are stored in the control plane on the Databricks cloud account, then does that mean Databricks allocates storage for the client free of cost? I…
System deny assignment created by Azure Databricks
I need to create static IP for Databricks but I faced errors to create cloud NAT The client 'live.com#myemail' with object id 'objectid' has permission to perform action 'Microsoft.Resources/deployments/validate/action' on scope…
Databricks cluster stuck when trying to start in state of "Finding instances for new nodes"
Yesterday I had an issue with multiple notebooks attempting to run against an interactive cluster that they trigger to start but unfortunately the cluster stays in a spinning state saying it is "Finding instances for new nodes"... I even tried…
Pipeline Failed due to error: InternalServerError executing request at adf
Around 15 triggers i have and only one dataset having trigger is failing everyday due to the error: InternalServerError executing request. The pipeline with attached triggers are published already. Any workaround for this issue will be more helpful.
Can't login Databricks
I used the account role as owner, global administrator. And I checked the resource and Databricks workspace regions match. There is exactly one resource. The workspace you are trying to access does not exist in this Azure region, or your account…
Unable to login to Azure Databricks Account Console
I have a personal Azure pay-as-you-go subscription in which I have the 'Global Administrator' role. I am also the databricks account administrator. Since two weeks I am not able anymore to access the databricks account console.
I need to get data from HTTP APIs in Azure storage Gen 2 account. What should I use?
Basically I have HTTP API and I have GraphQL queries through which I can get the data. The limitations are, in one request, we can only get data for a month and only 2000 records at a time. For example if we have 10000 records for July 2024, then I need…
Unable to login to Azure Databricks Account Console
I have a personal Azure pay-as-you-go subscription in which I have the 'Global Administrator' role. I am also the databricks account administrator. Until two weeks ago, I was able to access the databricks account console without any issues, but I am…
Azure policy does not work for resources created by Azure Databricks
I'm working on using a policy to automatically assign the sku.name tag to virtual machines created by Databricks. These tags are important for my team's cost management analysis. However, I discovered in another discussion that resources marked as…
Data Factory monitoring by inserting data table
Hello, I would like to know the best way to insert Datafactory activity logs into my Databricks delta table, so that I can use dashbosrd and create monitoring in Databricks itself , can you help me? I would like every 5 minutes for all activity logs in…
Data size of databricks delta tables
It has been observed that the size of delta tables are much less as compared to when checked the underlying delta files in the storage account. Suppose a databricks delta table raw.deltaTableA has size of 2MB if we check the size of underlying delta…
while creating cluster in databricks i am getting following error Azure Quota Exceeded Exception: Error code: SkuNotAvailable, error message: The requested VM size for resource 'Following SKUs have failed for Capacity Restrictions: Standard_DS3_v2' is cur
while creating cluster in databricks i am getting following error Azure Quota Exceeded Exception: Error code: SkuNotAvailable, error message: The requested VM size for resource 'Following SKUs have failed for Capacity Restrictions: Standard_DS3_v2' is…
Azure Databricks Workspace Unable to Access Workspace Storage Account
Databricks appears unable to access the workspace storage account. When attempting to display notebook cell results, an error returns: "Failed to upload command result to DBFS. Error message: dbstoragekbbxxxxxxxxx.dfs.core.windows.net: Name or…
CREATE TABLE does not overwrite location whereas CREATE OR REPLACE TABLE does
I am working on Azure Databricks, with Databricks Runtime version being - 14.3 LTS (includes Apache Spark 3.5.0, Scala 2.12). I am facing the following issue. Suppose I have a view named v1 and a database f1_processed created from the following…
Managed table overwrites existing location for delta but not for others
I am working on Azure Databricks, with Databricks Runtime version being - 14.3 LTS (includes Apache Spark 3.5.0, Scala 2.12). I am facing the following issue. Suppose I have a view named v1 and a database f1_processed created from the following…