Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,572 questions
2 answers
Azure cost optimization by Azure AI
Hi friends, Azure charges per hour ,but if I use the service for only 30 mins will I am charge for 1 hour? How does this costing mechanism works? And my requirement is to identity idle resources such as VMs and Databricks clusters , I need to use Azure…
asked 2024-03-25T10:28:55.4166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 15%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anshal
1,886
Reputation points
commented 2024-04-30T21:50:07.64+00:00
Amira Bedhiafi
15,676
Reputation points
2 answers
coalesce and broadcast join
HI, what exactly happen between coalesce and broadcast join in backend on databricks level
asked 2024-04-19T09:04:04.1866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Vineet S
165
Reputation points
commented 2024-04-30T21:37:38.48+00:00
Amira Bedhiafi
15,676
Reputation points
3 answers
partition in db
HI , what hppend in databricks backend when partion is applied to table level
1 answer
One of the answers was accepted by the question author.
How can i dynamically Flatten A Complex Nested JSON Structure in ADF
I am in the process of designing a solution. I need to load Rest API based complex JSON data into a flat file. I want to use the same pipeline for fetching different API result objects. My question is how can i dynamically flatten variant JSON files…
asked 2024-04-29T05:22:57.75+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(9.6, 71%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENQ%3C/text%3E%3C/svg%3E)
Noman Qureshi
20
Reputation points
commented 2024-04-30T07:10:55.72+00:00
Noman Qureshi
20
Reputation points
1 answer
One of the answers was accepted by the question author.
how to extend using Init script on DBFS
is there a way for me to extend the life of using current set-up of init scripts on DBFS? reason is I am getting errors now and i have not migrated my init scripts to UC/Workspace or Cloud storage and nothing runs now.
asked 2024-04-29T16:34:14.41+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 99%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAJ%3C/text%3E%3C/svg%3E)
Amesh Jeram
20
Reputation points
accepted 2024-04-30T04:01:51.1633333+00:00
Amesh Jeram
20
Reputation points
1 answer
An error occurred when using odbc and managed identity to link azure databricks in the. net core
Environmental description: The app service has already been configured with managed identity, and the relevant permissions for this managed identity have been configured in Azure databricks. When the app service uses code to access Azure databricks, the…
asked 2024-04-23T05:55:31.6166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 79%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZZ%3C/text%3E%3C/svg%3E)
ZhiHong Zhang (BEYONDSOFT CONSULTING INC)
0
Reputation points
commented 2024-04-30T03:22:16.7133333+00:00
PRADEEPCHEEKATLA-MSFT
77,751
Reputation points Microsoft Employee
2 answers
LIST error
All, When I run "LIST 'abfss://containertables@sa.dfs.core.windows.net/';" against the EXTERNAL LOCATIONS I get the result in the first image. If I append __unitystorage and run "LIST…
asked 2023-06-01T02:30:04.6033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 24%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGR%3C/text%3E%3C/svg%3E)
Gopinath Rajee
646
Reputation points
answered 2024-04-29T22:27:04.44+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 74%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EML%3C/text%3E%3C/svg%3E)
Mingquan Liang [mliang-z]
0
Reputation points
1 answer
Azure Databricks Lakehouse Monitoring queries
Hi Team, I was exploring on Azure Databricks Lakehouse monitoring. I have few queries on this: When I am running a "refresh metrics" irrespective of an automated schedule or manual refresh, which compute does it run? There is no mechanism to…
asked 2024-04-28T11:50:10.0333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 35%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESG%3C/text%3E%3C/svg%3E)
Sudipta Goswami
0
Reputation points
commented 2024-04-29T14:29:05.6666667+00:00
Vinodh247-1375
11,216
Reputation points
2 answers
execution plan in databricks
How to check execution plan in data bricks select * from cte
asked 2024-04-15T11:33:12.4566667+00:00
Vineet S
165
Reputation points
commented 2024-04-29T13:02:36.4433333+00:00
Amira Bedhiafi
15,676
Reputation points
1 answer
How to read nested xlsx files in Azure Data Factory
How we can read these types of nested xlsx files in ADF
asked 2024-04-27T03:51:29.27+00:00
Abhishek Tripathi
0
Reputation points
answered 2024-04-29T12:03:01.8666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Harishga
3,505
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
How to query Databricks audit logs?
Hi, I would like to ask where the Databricks Audit Log files are stored on the DBFS. And is there any way that I can query log files? Thanks.
asked 2023-02-15T05:23:20.95+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 5%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMS%3C/text%3E%3C/svg%3E)
Mohammad Saber
591
Reputation points
answered 2024-04-29T07:02:48.6366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 54%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Madhura Maddipatla
0
Reputation points
1 answer
Databricks jobs are failing with the insufficient privileges error
When running the jobs in databrick user is getting the insufficient privilege error, but user do have all the privileges like select, read_metadata, modify, create and owner as well on the respective database.
asked 2024-04-24T19:28:13.1033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(51.2, 24%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chethana B S
0
Reputation points
commented 2024-04-29T03:14:19.0533333+00:00
PRADEEPCHEEKATLA-MSFT
77,751
Reputation points Microsoft Employee
1 answer
How can I delete my Azure Databricks workspace when the workspace resource has already been removed?
As the title says I have an Azure Databricks Workspace where I have deleted the Workspace resource in Azure. However, the workspace still appears in the Account Console. From here I can log into the workspace but not create clusters etc. When I click on…
asked 2024-04-18T11:20:48.4533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 10%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKR%3C/text%3E%3C/svg%3E)
Kim Reinhart Jensen (KJEN)
0
Reputation points
commented 2024-04-29T02:52:08.18+00:00
PRADEEPCHEEKATLA-MSFT
77,751
Reputation points Microsoft Employee
2 answers
I am getting error while displaying the dataframe in azure databricks
df.show() is working but display(df) is not working df=spark.read.csv("abfss://demo@anshadls2001.dfs.core.windows.net/data/",header= "True") display(df) (1) Spark Jobs Failed to store the result. Try rerunning the command.Failed to…
asked 2024-01-20T04:50:28.2+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 87%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
Pawan Singh
0
Reputation points
answered 2024-04-26T21:25:36.8266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 76%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMT%3C/text%3E%3C/svg%3E)
Mojtaba Taslimitehrani
0
Reputation points
1 answer
Sync the tables from one azure databricks workspace to other databricks workspace or to adls gen2
Hello Team, We have two UC enabled databricks workspace. And we have to sync tables created in one Azure databricks workspace to other databricks workspace using PAT/any other reliable way or to adls gen2 account. Request you to let me know is there a…
asked 2024-04-11T18:50:09.1566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 24%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAG%3C/text%3E%3C/svg%3E)
Ashwini Gaikwad
65
Reputation points
commented 2024-04-26T18:25:52.01+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BhargavaGunnam-MSFT
26,306
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Raising the limit in Azure Databricks identities
Context: The Azure Databricks documentation states that "You can have a maximum of 10,000 combined users and service principals and 5,000 groups in an account. Each workspace can have a maximum of 10,000 combined users and service principals and…
asked 2024-04-25T10:41:16.95+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 28.999999999999996%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKS%3C/text%3E%3C/svg%3E)
Kim Stig Hansen
20
Reputation points
accepted 2024-04-26T07:31:55.23+00:00
Kim Stig Hansen
20
Reputation points
1 answer
One of the answers was accepted by the question author.
How to use AWS Databricks note book in ADF..?
Hi Team, I want to create Linked service in ADF with AWS Databricks notebook. Can you guys help me with list of steps that i need follow. Regards, Naveen.
asked 2023-10-30T03:44:41.43+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 77%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENK%3C/text%3E%3C/svg%3E)
Naveen Kumar Rachapalle (EXT/DIGITAL)
25
Reputation points
commented 2024-04-25T10:09:37.6533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 66%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
Syed Ahamed S
0
Reputation points
1 answer
Custom text single label classification - Model API Consumption within Databricks
Hello together, i trained a model within Azure Language Studio , Custom text single label classification - and i want to consume Model API within Databricks Notebook. I get always below given error, kindly asking for your help. Thanks. Error: HTTP…
asked 2024-04-18T11:00:16.9066667+00:00
Aziz Öztürk
20
Reputation points
commented 2024-04-25T05:47:24.94+00:00
PRADEEPCHEEKATLA-MSFT
77,751
Reputation points Microsoft Employee
1 answer
How to fix timeout error when clreating compute cluster on azure databricks.
Azure error message: [id: InstanceId(e8b82690e17a4624a152afb20dce5339), status: INSTANCE_LAUNCHING, workerEnvId:WorkerEnvId(workerenv-3367469989580530), lastStatusChangeTime: 1713836219115, groupIdOpt Some(0),requestIdOpt…
asked 2024-04-23T02:31:21.1133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 21%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
GEOFREY GETUBA
0
Reputation points
commented 2024-04-25T05:39:28.3766667+00:00
PRADEEPCHEEKATLA-MSFT
77,751
Reputation points Microsoft Employee
2 answers
One of the answers was accepted by the question author.
How to ingest CDC data from SQL DATA SYNC to Event hub?
Hi we have a scenario to implement real-time CDC from SQL DATA SYNC to Azure SQL DB. Which tools are good for near real-time processes and prices as well. By using data bricks we can merge stream data into delta lake. But How to ingest real-time…
asked 2020-11-10T12:28:43.067+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 54%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETG%3C/text%3E%3C/svg%3E)
THIMMAIAH GARI,PRASHANTH,,
201
Reputation points
edited an answer 2024-04-24T06:51:50.8266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 91%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
Shane Blake
0
Reputation points