Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,426 questions
1 answer
Azure datalake creation methods
Hi friends, we need to create a datalake (ADLS) with bronze, silver, and gold layers. One of our colleagues suggested it is a better idea to create ADLS through programming, but we plan to create it manually. Are there any advantages of each approach …
asked 2024-07-24T10:24:14.9566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 15%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anshal
2,246
Reputation points
answered 2024-07-24T12:28:47.9933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(80, 13%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBB%3C/text%3E%3C/svg%3E)
Babafemi Bulugbe
3,215
Reputation points MVP
1 answer
One of the answers was accepted by the question author.
ADLS Gen2 Query Acceleration
Hello, As a follow up to this thread: Query acceleration using parquet does not work with double fields - Microsoft Q&A I would like to know if and when Microsoft plans to enable the query acceleration feature for Parquet files as well for ADLS Gen2…
asked 2024-07-17T05:15:46.93+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 99%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED7%3C/text%3E%3C/svg%3E)
DanielMeierotto-7637
20
Reputation points
accepted 2024-07-24T09:27:59.3333333+00:00
DanielMeierotto-7637
20
Reputation points
2 answers
Getting the size of parquet files from azure blob storage
I have a blob container abcd The folder structure is like below: abcd/Folder1/Folder a, Folder b…..Folder z Inside a particular Folder a/v1/full/20230505/part12344.parquet Similarly Folder b/v1/full/20230505/part9385795.parquet Scenario is I need to get…
asked 2024-07-09T11:57:50.94+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 89%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKJ%3C/text%3E%3C/svg%3E)
KEERTHANA JAYADEVAN
66
Reputation points
commented 2024-07-24T08:31:27.1433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
Nehruji R
4,691
Reputation points Microsoft Vendor
1 answer
ADLS connections with third party
Hi friends, we have ingested data from all the data sources and stored it in ADLS Gen2 but after that, there is a virtual layer in the form of Denodo. This virtualization tool only has views and once views are executed, it fetches the data from multiple…
1 answer
One of the answers was accepted by the question author.
Can I connect JDBC to Azure Data Lake Storage Gen2?
[background] I want to use the following Athena Connector with Azure Data Lake Storage Gen2 to query Azure Storage from Amazon Athena on AWS. https://docs.aws.amazon.com/athena/latest/ug/connectors-adls-gen2.html According to the…
asked 2022-09-26T23:56:05.703+00:00
Takahito Iwasa
4,841
Reputation points MVP
commented 2024-07-23T10:23:28.3733333+00:00
PREMRAJ S
0
Reputation points
2 answers
One of the answers was accepted by the question author.
Azure data lake storage and strategy
Hi Friends, We need to store huge data in a data lake of around 150TB. I have the following questions: We might not store the whole 150TB data at one point, but if we do and have one datalake storage would performance issues be caused? What are the…
1 answer
Can't query partitioned tables - Azure Synapse Link for Dataverse
Hi, we have a problem with accessing partitioned tables in Azure Synapse Workspace that were previously created by ingesting tables from Power Platform via Azure Synapse Link for Dataverse. Snapshot folder and .csv files are normally created but can't be…
asked 2023-03-20T16:05:06.8433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(208, 75%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELM%3C/text%3E%3C/svg%3E)
Leo Milos
10
Reputation points
commented 2024-07-23T00:51:57.4366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 17%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGM%3C/text%3E%3C/svg%3E)
Gary Melhaff - US
6
Reputation points
2 answers
SAS token generation by Databricks to access CSV files from ADLS container folder
Hi Team, There are some csv files zips inside the ADLS container folder. These zip files need to be downloaded for data correction. Downloading the file requires SAS token embedded with zip file path. Databricks has been used to generate the token and…
asked 2024-07-17T05:08:24.3033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 91%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Subhadip Roy
21
Reputation points
commented 2024-07-22T08:11:32.9266667+00:00
Subhadip Roy
21
Reputation points
1 answer
One of the answers was accepted by the question author.
Unable to connect Fivetran with public IP to ADLS Gen2 in Vnet with Private end point
Hi, I'm encountering an issue while trying to connect Fivetran to ADLS Gen2 in the UK South region. Here are the details: I have configured ADLS Gen2 as a destination in Fivetran and completed the prerequisites, including creating an SPN with appropriate…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 15%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
2 answers
One of the answers was accepted by the question author.
Azure data lake storage and evaluation
Hi friends, what is the meaning of metrics operations applied in data lake storage calculations? And there are a bunch of all metrics to calculate that are confusing. Any easy way to understand?
1 answer
ADLS Storage access forbidden 403 in synapse
Hello, I tried to run the sql code under synpase platform but seems like i dont have access to it, but i could access them when running activity like 'copy data etc. What should i do next to open the access?
asked 2024-07-18T12:47:04.3666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 2%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENW%3C/text%3E%3C/svg%3E)
Nick Wong
170
Reputation points
answered 2024-07-18T17:52:00.45+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 81%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Amrinder Singh
4,435
Reputation points Microsoft Employee
1 answer
ADLS Gen2 Access Logs via Diagnostic Settings
Hi, I was wondering if there is any way to enable access logs like read, write, delete on Azure Data Lake Storage Gen2 (ADLS Gen2). For Azure Blob Storage we achieved this via Diagnostic Settings, but to me it looks like this doesn't cover any operations…
asked 2024-07-17T15:08:17.9566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 89%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECA%3C/text%3E%3C/svg%3E)
Cedric Ahlers
15
Reputation points
commented 2024-07-18T12:30:41.7333333+00:00
Amrinder Singh
4,435
Reputation points Microsoft Employee
2 answers
To Read Delta tables from ADLS Gen2 from Azure Analysis Services (AAS)
I'm not able to Read Delta tables from ADLS Gen2 from Azure Analysis Services (AAS), even though the MS Azure page here https://learn.microsoft.com/en-us/azure/analysis-services/analysis-services-datasource#tab1400a has given an option to read delta…
asked 2024-07-01T08:29:19.1566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 86%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
Aravind K Pachaiyappan
0
Reputation points
commented 2024-07-18T12:20:02.0933333+00:00
Nehruji R
4,691
Reputation points Microsoft Vendor
1 answer
Encryption Data on transit - from oracle to Azure Data lake
Hello - We have this implementation setup: use Synapse analytics Pipelines to extract data from Oracle OCI database Copy data to Azure data Lake Storage Transform data from ADLS Gen 2 to Azure Sql Database We want to make sure that : Data…
asked 2024-07-02T07:19:25.15+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 88%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EXM%3C/text%3E%3C/svg%3E)
Xhevahir Mehalla
160
Reputation points
commented 2024-07-18T07:06:51.0866667+00:00
Xhevahir Mehalla
160
Reputation points
2 answers
One of the answers was accepted by the question author.
How to fix the copy activity error while copying data from databricks delta table to datalake in csv format
There are some error tables in Databricks delta table . Those tables need to extracted as csv and load in azure data lake , inside the folder of the container. Staging has been enabled in the copy activity since it is 2 step process. Approx row count of…
asked 2024-07-11T07:54:48.48+00:00
Subhadip Roy
21
Reputation points
accepted 2024-07-17T06:49:53.1066667+00:00
Subhadip Roy
21
Reputation points
2 answers
One of the answers was accepted by the question author.
how to read excel file data (stored in adls gen2 ) in Azure Data Factory pipeline
I have source excel file which is coming as showing in the format . Please help how to read excel file with that specified format . Please help . Thank You.
asked 2024-07-15T06:39:57.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 88%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBS%3C/text%3E%3C/svg%3E)
Bathini, Shirish
20
Reputation points
accepted 2024-07-16T13:58:19.0633333+00:00
Bathini, Shirish
20
Reputation points
1 answer
Custom Logs.
I have a CSV file (say "empfile.csv") in my data lake which is then loaded into Azure Synapse's dedicated SQL pool. Now, My data exists at both Data Lake and SQL pool. Suppose, I update some field values in this data (say "Manager"…
asked 2024-07-15T08:16:44.56+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 79%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAT%3C/text%3E%3C/svg%3E)
Apurva Tiwari
40
Reputation points
commented 2024-07-16T08:55:05.3466667+00:00
AnnuKumari-MSFT
32,161
Reputation points Microsoft Employee
2 answers
One of the answers was accepted by the question author.
Where do i find Azure HTTP status 429 response-code-examples
Where do i find Azure HTTP status 429 response-code-examples for: Azure Service BUS Azure Event Hub Azure Event Grid Azure Blob Storage Microsoft Fabric Azure Blob Storage und MS Fabric benutzt Azure Datalake Storage gen2 API
asked 2024-07-12T14:27:28.4266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 35%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Richard Geary
20
Reputation points
commented 2024-07-15T08:20:01.7666667+00:00
Richard Geary
20
Reputation points
2 answers
How to copy files from storage account to another storage account with the same last modified date using ADF
Hi Team, The request is that, when we are running ADF copy job to copy data from Landing storage account to Prod Storage account the last modified date coming as present copy time, but we want to copy the last modified date, what we have in landing…
asked 2024-06-20T15:58:52.0233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(25.6, 2%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
Mahasiva, Ravi (CSW)
60
Reputation points
commented 2024-07-15T07:37:43.9133333+00:00
Nehruji R
4,691
Reputation points Microsoft Vendor
2 answers
This request is not authorized to perform this operation using this permission.", 403, HEAD Synapse connect to adls
I am trying to select data from ADLS Gen2 storage delta table and keep receiving this error. I added the synapse service principal as storage blob data contributor and ACLs to container with no luck. Firewall is set to enable all networks as well. Please…
asked 2024-06-22T20:29:54.0233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 87%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDW%3C/text%3E%3C/svg%3E)
Daniel Woodhouse
0
Reputation points
commented 2024-07-15T07:32:42.4733333+00:00
Nehruji R
4,691
Reputation points Microsoft Vendor