Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,334 questions
2 answers
Azure Data Lake Gen2 Disaster Recovery/Storage account failover Capabilities?
I am trying to wrap my brain around Azure Data Lake Gen 2 DR/Storage account failover capabilities and I feel as though I am getting conflicting information. On the Microsoft Documentation -…
asked 2020-06-05T16:28:28.463+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 66%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Adam Sebetich
1
Reputation point
commented 2023-12-12T13:47:27.41+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 40%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
Venkatachalapathy, Rajkumar SBOBNG-ITZ/IF
6
Reputation points
2 answers
remove chars from integer columns in databricks delta
Hi Expert, how to remove chars from column in databricks delta when datatype is not null col1 1 2 3 er ge e
asked 2023-12-08T12:02:08.32+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 75%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Shambhu Rai
1,406
Reputation points
commented 2023-12-10T17:34:15.5333333+00:00
Shambhu Rai
1,406
Reputation points
1 answer
Load files in appending in databricks
Hi Expert, I have 2 files in blob storage . file1.csv, file2.csv how will i load it in a view using appending order using *.csv extension using temp view file_list = dbutils.fs.ls("/mnt/mount/test/file")
asked 2023-11-29T05:48:38.4166667+00:00
Shambhu Rai
1,406
Reputation points
commented 2023-11-30T12:32:49.06+00:00
Shambhu Rai
1,406
Reputation points
1 answer
jdfc url of blob storage
Hi , i wanted to push the files to blob storgate using portman ... how to connect using jdbc connection i tried below but unable to push it.. can i get jdbc file for…
asked 2023-11-06T12:32:08.7566667+00:00
Shambhu Rai
1,406
Reputation points
commented 2023-11-15T07:05:29.57+00:00
PRADEEPCHEEKATLA-MSFT
76,511
Reputation points Microsoft Employee
0 answers
Datalake account_partitioned table row count is verly low
We have azure datalake with builtin pool In that we have builtin account_Partitioned table which use to have similar no or rows as per non partitioned table but now from today only showing 10% of total rows We are in UK West region and something…
asked 2023-11-06T18:07:39.4833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(96, 78%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Musarrat Aziz
5
Reputation points
edited the question 2023-11-15T07:01:44.1266667+00:00
PRADEEPCHEEKATLA-MSFT
76,511
Reputation points Microsoft Employee
1 answer
Correction on Microsoft PL-300 Practice Assessment Questions
This is to bring to the notice of the Admin in charge of Practice Test Questions under Pl-300 Power Bi Exam. There is a confusion in one of the questions and answers. Please find attached snap shot and correct the answers Thank you
asked 2023-10-24T10:51:44.65+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(284.8, 49%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAO%3C/text%3E%3C/svg%3E)
Ayotomide Ogunsakin
0
Reputation points
edited the question 2023-11-06T07:46:50.6233333+00:00
Sumarigo-MSFT
43,401
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Data lake zones and containers, what is the best practices of file name
Hi All, follow up Best practices from cloud Adoption framework, can somebody tell standers for file Name Data LakeLayersContainer numberContainer nameFile Name Standard1Raw1Landing?1Raw2Conformance?2Enriched1Standardized?2Curated2Data…
asked 2023-10-16T07:58:21.98+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 20%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMV%3C/text%3E%3C/svg%3E)
manish verma
421
Reputation points
accepted 2023-10-20T13:10:33.5533333+00:00
manish verma
421
Reputation points
1 answer
Can we pass partition key as a parameter in a generic/dynamic azure dataflow.
Operation on target Load Entity failed: Operation on target Load Entity failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: com.microsoft.dataflow.Issues: DF-TX-114 - Key partitioning does…
asked 2023-10-05T08:43:06.7+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 16%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBU%3C/text%3E%3C/svg%3E)
Batta, Uha Kanakavalli
0
Reputation points
commented 2023-10-12T18:26:57.77+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 38%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EQ%3C/text%3E%3C/svg%3E)
QuantumCache
20,026
Reputation points
1 answer
One of the answers was accepted by the question author.
Help me decide between Azure data lake analytics and Azure Databricks
Both Azure Data Lake Analytics and Azure Databricks can be used for batch processing. Can you help me understand for which case we can use each one?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 64%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EW%3C/text%3E%3C/svg%3E)
2 answers
One of the answers was accepted by the question author.
RowsWritten from data flow in log analytics
I was recently able to get the rowsCopied and rowsRead from a pipeline copy activity in log analytics....I'm trying to get the rowsWritten from the pipeline dataflow activity but I'm not getting any input. Is this even logged in log analytics? and if…
asked 2021-09-28T07:13:57.18+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 72%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
Adeleke Abimbola
231
Reputation points
commented 2023-09-11T15:15:13.5833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(9.6, 66%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
Pratibha
0
Reputation points
1 answer
ADF converts NUMBER datatype to Decimal, String (if precision > 28) while copying data type from ORACLE to ADLS as Parquet
While extracting data from Oracle on premise to Azure Data Lake as Parquet files via SHIR, we notice the below points: NUMBER Datatype in Oracle gets converted to Decimal, String (if precision > 28) in parquet file Kindly share some light on the…
asked 2023-07-31T04:53:47.64+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 59%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Swapnil Sarkar
0
Reputation points
commented 2023-08-19T07:30:36.56+00:00
AnnuKumari-MSFT
30,361
Reputation points Microsoft Employee
1 answer
Azure Data Lake Analytics Retirement - Frequently Asked Questions
On 29 February 2024, we will retire Azure Data Lake Analytics and this service will no longer be available. Before that date, please migrate your services that were deployed using Azure Data Lake Analytics to another Azure data processing product of…
asked 2022-03-17T19:20:07.157+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 6%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
Michael Rys [MSFT]
11
Reputation points Microsoft Employee
edited an answer 2023-08-18T18:41:45.55+00:00
Michael Rys [MSFT]
11
Reputation points Microsoft Employee
1 answer
How to backup ADLS for ZRS
Hi Experts, ADLS doesny support backup out of box. I want to backup the adls files for ZRS (zone level redudancy). Can anyone please suggest what is the best way?
asked 2023-07-12T14:19:03.9933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 39%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Saha
0
Reputation points
commented 2023-08-07T23:02:22.1766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 9%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KranthiPakala-MSFT
46,422
Reputation points Microsoft Employee
2 answers
sql query on blob storage
Hi Expert, how to write sql query on azure blob storage files using databricks notebook without using azure synapse
asked 2023-07-27T20:32:12.95+00:00
Shambhu Rai
1,406
Reputation points
commented 2023-08-07T07:10:15.0333333+00:00
PRADEEPCHEEKATLA-MSFT
76,511
Reputation points Microsoft Employee
0 answers
How to fix api-version error when submitting jobs by using the Azure CLI?
I wanted to submit jobs by using the Azure CLI, but I found that "az dla job submit" and other "az dla" command lines cannot run successfully, an error was reported: api-version '2016-11-01' is invalid. Valid api-version:…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 64%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESZ%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
cluster performance in ideal time
Hi Expert, how to avoid the ideal time/sleeping time scenarios on databricks cluster so that it would run smoothly even though there is no traffic. and we should not start all the time and should run continuously
asked 2023-07-26T07:29:24.2666667+00:00
Shambhu Rai
1,406
Reputation points
accepted 2023-07-26T20:34:05.46+00:00
Shambhu Rai
1,406
Reputation points
2 answers
Data Transfer in real time between on-prem and cloud
Hello, i basically need my on-prem MS SQL db to be synced with cloud MS SQL DB at all times. As of now, i am thinking of using Azure Data factory but i fear it cannot perform near realtime data transfer. Other option is not use kafka to move data, but…
asked 2023-01-22T18:41:59.0333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 48%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERM%3C/text%3E%3C/svg%3E)
Rajshekar Mani
6
Reputation points
commented 2023-07-20T06:07:44.49+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 76%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EI2%3C/text%3E%3C/svg%3E)
IndikaWickramasinghe-2424
0
Reputation points
1 answer
storage account to data transfer for azure data lake gen2
storage account to data transfer for azure data lake gen2 No use Azure Data Factory
asked 2023-07-14T06:51:15.1633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 99%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMP%3C/text%3E%3C/svg%3E)
Manickavasagam P
0
Reputation points
commented 2023-07-17T16:23:20.8033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BhargavaGunnam-MSFT
25,876
Reputation points Microsoft Employee
1 answer
Unable to unlink existing company id and link new company email id to receive discounts
Hello, earlier I had linked my company A email id to receive discounts. Now I have moved to new company - Company B , and want to link my certification profile, however, when I try to do it nothing happens, and old company keeps on appearing. With my new…
asked 2023-04-24T09:58:09.2033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 48%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
rajni kant
0
Reputation points
edited the question 2023-04-26T03:38:58.29+00:00
PRADEEPCHEEKATLA-MSFT
76,511
Reputation points Microsoft Employee
1 answer
Migration of ADLS gen1 to gen2 - U-SQL scripts to ADLS Gen2
We are in the process of migrating ADLS Gen1 to ADLS Gen2. We understand that ADLA (Azure Data Lake Analytics) is not supported on ADLS Gen2. Can you please support us in how we can migrate U-SQL scripts to ADLS Gen2? Is there a quick workaround for…
asked 2023-04-11T11:55:35.81+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 73%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
Renny Sreekumar
0
Reputation points
commented 2023-04-12T15:36:33.64+00:00
BhargavaGunnam-MSFT
25,876
Reputation points Microsoft Employee