Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,348 questions
3 answers
I want to delete storage account in azure before deleting what are the things we need to check in the azure
we need to understand if we are good to clean up the old logs from that storage account
asked 2024-02-28T11:32:40.8033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(48, 16%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETS%3C/text%3E%3C/svg%3E)
Tamil Selvan M
0
Reputation points
commented 2024-03-07T05:04:03.8466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 81%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Amrinder Singh
2,195
Reputation points Microsoft Employee
0 answers
How to run one synapse pipeline across tenants from different domain (entraID)
Hi, I have synapse pipeline with some spark notebooks with ETL transformations. I need to run this pipeline and save data to different ADLS. All of them are in different subscriptions - some of them are having different entraID than my workspace where I…
asked 2024-02-19T17:35:00.5633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 91%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPM%3C/text%3E%3C/svg%3E)
Peter Michalik
20
Reputation points
edited a comment 2024-03-06T08:41:52.18+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
5,840
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
it's must using SAS token for copy file between blob and blob?
we want to use azcopy tool to copy file between storage account's blob, and use service principal as login identity, when we try to run azcopy copy command without SAS token of storage account ,the job is copy files failed. it's must using SAS token…
asked 2024-02-27T08:52:40.4233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 46%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHW%3C/text%3E%3C/svg%3E)
Huang, Winston-HR
280
Reputation points
accepted 2024-03-06T05:36:14.9133333+00:00
Huang, Winston-HR
280
Reputation points
2 answers
One of the answers was accepted by the question author.
In ADLS, when using the DataLakeServiceClient, does every user have access to view the ACL for each directory?
Hello, I'm working on an application that needs to replicate ADLS access controls which are configured by Access Control LIsts. I'm using the python SDK with the InteractiveBrowserCredential class to authenticate a user through their Azure account. The…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(41.6, 64%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
1 answer
Encountering errors when attempting to write files from Azure Synapse Analytics notebook to Azure Data Lake Storage (ADLS) Gen2, while having a private endpoint configured with the DFS URL (OSError: [Errno 5] Input/output error)
Issue: Encountering errors when attempting to write files from Azure Synapse Analytics notebook to Azure Data Lake Storage (ADLS) Gen2, while having a private endpoint configured with the DFS URL. Requirement: Files need to be downloaded from a…
asked 2024-02-23T15:43:25.0566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 61%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Ramachandran, Kannan
0
Reputation points
commented 2024-03-05T07:54:21.3666667+00:00
ShaikMaheer-MSFT
37,896
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Copy files from Sharepoint into Azure Data Lake Store Gen 2 using ADF linked services ?
I have to copy files from Sharepoint into Azure Data Lake Store Gen 2 using ADF linked services
asked 2024-02-28T13:23:10.42+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 96%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJK%3C/text%3E%3C/svg%3E)
Jaydeep Kumbhar
20
Reputation points
edited the question 2024-03-05T07:09:08.5433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAM%3C/text%3E%3C/svg%3E)
AllenXu-MSFT
16,091
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Retirement Announcement - Azure Data Lake Storage Gen1
Hello Azure Customers in the community, Azure Data Lake Storage Gen1 will be retired on Feb 29, 2024. Azure Data Lake Storage Gen2 offers a richer set of data management capabilities, lower cost, and integration with newer analytics offerings such as…
asked 2021-02-19T23:25:12.753+00:00
bharathn-msft
5,086
Reputation points Microsoft Employee
edited an answer 2024-03-04T17:39:10.2033333+00:00
bharathn-msft
5,086
Reputation points Microsoft Employee
1 answer
write a file to azure storage account file share
I want to write data from azure synapse analytics to Azure SMB File share eg I have a dataframe df and I want to write this data to a container #df This is my…
asked 2024-03-04T10:25:34.66+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 69%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Aditya Singh
105
Reputation points
commented 2024-03-04T12:04:11.23+00:00
Deepanshu katara
4,980
Reputation points
2 answers
One of the answers was accepted by the question author.
Storage event trigger is running twice when a single blob is created
Hi All, I created a storage event trigger on a pipeline, but noticed that the trigger runs twice when a single file lands in the container , when two file lands the pipeline is triggered 3 times, i expect the pipeline to run once when a single file…
asked 2024-02-16T08:11:41.2966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 28.999999999999996%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EOA%3C/text%3E%3C/svg%3E)
Osamwonyi Alekwe
40
Reputation points
accepted 2024-03-04T07:38:24.3033333+00:00
Osamwonyi Alekwe
40
Reputation points
2 answers
One of the answers was accepted by the question author.
I have a dataframe in Synapse Apache spark pool and I want to create a table for that dataframe in my dedicated sql pool
I have a dataframe in Synapse Apache spark pool and I want to create a table for that dataframe in my dedicated sql pool.
asked 2024-03-01T04:20:45.7966667+00:00
Aditya Singh
105
Reputation points
accepted 2024-03-04T06:27:15.3966667+00:00
Aditya Singh
105
Reputation points
3 answers
Which subnet should be added to enable network communication interoperability
I want to use azcopy to transfer files from storage account A to storage account B. Where storage account A is in subnet a, and storage account B in subnet b, Azcopy is deployed in VM, which is in subnet c. What confuses me is when adding a network…
asked 2024-02-23T08:28:32.6666667+00:00
Huang, Winston-HR
280
Reputation points
answered 2024-03-02T02:29:34.7466667+00:00
Sumarigo-MSFT
43,801
Reputation points Microsoft Employee
1 answer
Copying Complete SharePoint Library to Azure Data Lake Storage in Azure Data Factory
How can I copy the entire SharePoint library, including its folders, subfolders (nested folders), and all files, to Azure Data Lake Storage (ADLS) in Azure Data Factory? Is this possible?
asked 2024-03-01T11:50:49.09+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 75%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
A Aathithya
25
Reputation points
answered 2024-03-01T14:17:32.69+00:00
Amira Bedhiafi
15,676
Reputation points
1 answer
One of the answers was accepted by the question author.
Power Apps: Can you connect to ADLS Gen2
Can I connect PowerApps to ADLS Gen2? Is there any limitation?
asked 2021-10-01T03:42:45.497+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 41%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECV%3C/text%3E%3C/svg%3E)
Chilla, Vishwanatha
51
Reputation points
commented 2024-03-01T10:52:40.0166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 90%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EML%3C/text%3E%3C/svg%3E)
Mark Lui
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Using Synapse spark pool how can a read a table which is in Dedicated SQL POOL
Using Synapse spark pool how can a read a table which is in Dedicated SQL POOL I have a table in the sql dedicated pool This is my spark pool
asked 2024-02-29T11:08:05.3666667+00:00
Aditya Singh
105
Reputation points
commented 2024-03-01T04:16:37.4533333+00:00
Aditya Singh
105
Reputation points
0 answers
How to load delta files from ADLS to SQL in ADF
Hi have requirement to load delta files from adls container to sql but delta files are partitioning so load all latest partitioned files using adf how to achive this.
asked 2024-02-27T05:20:34.0166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 20%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPK%3C/text%3E%3C/svg%3E)
Prapul Kumar Dongari
116
Reputation points
commented 2024-02-29T10:27:29.6066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 12%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
Suba Balaji
11,186
Reputation points
1 answer
One of the answers was accepted by the question author.
How to convert the content md5 value (base64) from getmetadata activity to hex to match locally generated MD5 value?
I developed a code to get the content MD5 value from Getmetadata activity of a file from my azure data lake storage account. Content MD5 value thus generated is in base64 format. I am receiving the MD5 value from a linux system which is in hexadecimal…
asked 2024-02-19T14:51:34.4466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 69%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDS%3C/text%3E%3C/svg%3E)
D Sai Sowmya
20
Reputation points
accepted 2024-02-28T17:13:20.5066667+00:00
D Sai Sowmya
20
Reputation points
0 answers
Slow performance reading parquet files from Data Lake Gen2
I'm using Azure Data Lake Gen2 to store data in parquet file format. I have split the data using partitions by year, month and day to benefit from the filtering functionality. I'm reading the data from python directly (no Spark or Synapse here) using the…
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,348 questions
Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
966 questions
asked 2024-02-15T14:36:23.51+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 30%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJL%3C/text%3E%3C/svg%3E)
Jorge Lopez
31
Reputation points
commented 2024-02-27T15:59:46.84+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 98%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KarishmaTiwari-MSFT
18,522
Reputation points Microsoft Employee
1 answer
Gen1 Compatibility layer
we can see in the documentation the support for the Gen1 compatibility layer will end on feb 29 th .will there be a extension for the Gen1 comptability layer
asked 2024-02-20T12:10:21.0666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 71%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
sagi rachana
0
Reputation points
commented 2024-02-26T06:44:29.1066667+00:00
Amrinder Singh
2,195
Reputation points Microsoft Employee
2 answers
One of the answers was accepted by the question author.
Can't create a datalake storage account Gen2 when creating an azure synapse resource
When trying to create a new data lake gen2 storage account (while creating a azure Synapse resource), I keep on getting an error message that the account name can't be verified and that I need to try again. I've waited for a couple of hours and have…
asked 2023-11-16T17:15:29.1633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 46%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EUF%3C/text%3E%3C/svg%3E)
u512353 (Fuentes Alfaro, Patricia)
25
Reputation points
commented 2024-02-25T20:35:42.4066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(284.8, 72%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJV%3C/text%3E%3C/svg%3E)
Jackson Vieira
0
Reputation points
2 answers
How to generate a static IP for ADF to connect to Third party SFTP server
Hello All, I want to connect to third party SFTP server and donwload the files in AzureContainer. In order to connect to the SFTP I need to provide them with a IP address (Not range) so they can whitelist it. I will be using ADF SFTP connector to…
asked 2024-01-27T00:03:56.6566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 69%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPP%3C/text%3E%3C/svg%3E)
Pankaj Palta
0
Reputation points
commented 2024-02-23T19:55:53.92+00:00
Pankaj Palta
0
Reputation points