Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,425 questions
0 answers
DP-203 Labs
Is there any way to do all the labs for DP-203 without costing alot of money AND without taking like 10 minutes to install something? For each lab, the ./setup.ps1 file takes like 10 or 15 minutes to load. I'd like to not have to delete the resources…
Azure Data Lake Storage
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,671 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,071 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,134 questions
Azure Training
Azure Training
Azure: A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.Training: Instruction to develop new skills.
1,295 questions
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 23%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBC%3C/text%3E%3C/svg%3E)
2 answers
SAS token generation by Databricks to access CSV files from ADLS container folder
Hi Team, There are some csv files zips inside the ADLS container folder. These zip files need to be downloaded for data correction. Downloading the file requires SAS token embedded with zip file path. Databricks has been used to generate the token and…
asked 2024-07-17T05:08:24.3033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 91%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Subhadip Roy
21
Reputation points
commented 2024-07-20T10:59:04.45+00:00
Vinodh247
13,301
Reputation points
2 answers
One of the answers was accepted by the question author.
Azure data lake storage and evaluation
Hi friends, what is the meaning of metrics operations applied in data lake storage calculations? And there are a bunch of all metrics to calculate that are confusing. Any easy way to understand?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 15%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
1 answer
ADLS Storage access forbidden 403 in synapse
Hello, I tried to run the sql code under synpase platform but seems like i dont have access to it, but i could access them when running activity like 'copy data etc. What should i do next to open the access?
asked 2024-07-18T12:47:04.3666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 2%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENW%3C/text%3E%3C/svg%3E)
Nick Wong
170
Reputation points
answered 2024-07-18T17:52:00.45+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 81%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Amrinder Singh
4,345
Reputation points Microsoft Employee
1 answer
ADLS Gen2 Access Logs via Diagnostic Settings
Hi, I was wondering if there is any way to enable access logs like read, write, delete on Azure Data Lake Storage Gen2 (ADLS Gen2). For Azure Blob Storage we achieved this via Diagnostic Settings, but to me it looks like this doesn't cover any operations…
asked 2024-07-17T15:08:17.9566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 89%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECA%3C/text%3E%3C/svg%3E)
Cedric Ahlers
15
Reputation points
commented 2024-07-18T12:30:41.7333333+00:00
Amrinder Singh
4,345
Reputation points Microsoft Employee
2 answers
To Read Delta tables from ADLS Gen2 from Azure Analysis Services (AAS)
I'm not able to Read Delta tables from ADLS Gen2 from Azure Analysis Services (AAS), even though the MS Azure page here https://learn.microsoft.com/en-us/azure/analysis-services/analysis-services-datasource#tab1400a has given an option to read delta…
asked 2024-07-01T08:29:19.1566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 86%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
Aravind K Pachaiyappan
0
Reputation points
commented 2024-07-18T12:20:02.0933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
Nehruji R
4,531
Reputation points Microsoft Vendor
2 answers
Getting the size of parquet files from azure blob storage
I have a blob container abcd The folder structure is like below: abcd/Folder1/Folder a, Folder b…..Folder z Inside a particular Folder a/v1/full/20230505/part12344.parquet Similarly Folder b/v1/full/20230505/part9385795.parquet Scenario is I need to get…
asked 2024-07-09T11:57:50.94+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 89%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKJ%3C/text%3E%3C/svg%3E)
KEERTHANA JAYADEVAN
66
Reputation points
commented 2024-07-18T12:13:50.3+00:00
Amrinder Singh
4,345
Reputation points Microsoft Employee
2 answers
Azure data lake storage and strategy
Hi Friends, We need to store huge data in a data lake of around 150TB. I have the following questions: We might not store the whole 150TB data at one point, but if we do and have one datalake storage would performance issues be caused? What are the…
1 answer
Encryption Data on transit - from oracle to Azure Data lake
Hello - We have this implementation setup: use Synapse analytics Pipelines to extract data from Oracle OCI database Copy data to Azure data Lake Storage Transform data from ADLS Gen 2 to Azure Sql Database We want to make sure that : Data…
asked 2024-07-02T07:19:25.15+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 88%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EXM%3C/text%3E%3C/svg%3E)
Xhevahir Mehalla
140
Reputation points
commented 2024-07-18T07:06:51.0866667+00:00
Xhevahir Mehalla
140
Reputation points
0 answers
ADLS Gen2 Query Acceleration
Hello, As a follow up to this thread: Query acceleration using parquet does not work with double fields - Microsoft Q&A I would like to know if and when Microsoft plans to enable the query acceleration feature for Parquet files as well for ADLS Gen2…
asked 2024-07-17T05:15:46.93+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 99%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED7%3C/text%3E%3C/svg%3E)
DanielMeierotto-7637
0
Reputation points
commented 2024-07-17T10:20:03+00:00
Sumarigo-MSFT
45,406
Reputation points Microsoft Employee
2 answers
One of the answers was accepted by the question author.
How to fix the copy activity error while copying data from databricks delta table to datalake in csv format
There are some error tables in Databricks delta table . Those tables need to extracted as csv and load in azure data lake , inside the folder of the container. Staging has been enabled in the copy activity since it is 2 step process. Approx row count of…
asked 2024-07-11T07:54:48.48+00:00
Subhadip Roy
21
Reputation points
accepted 2024-07-17T06:49:53.1066667+00:00
Subhadip Roy
21
Reputation points
2 answers
One of the answers was accepted by the question author.
how to read excel file data (stored in adls gen2 ) in Azure Data Factory pipeline
I have source excel file which is coming as showing in the format . Please help how to read excel file with that specified format . Please help . Thank You.
asked 2024-07-15T06:39:57.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 88%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBS%3C/text%3E%3C/svg%3E)
Bathini, Shirish
20
Reputation points
accepted 2024-07-16T13:58:19.0633333+00:00
Bathini, Shirish
20
Reputation points
1 answer
Custom Logs.
I have a CSV file (say "empfile.csv") in my data lake which is then loaded into Azure Synapse's dedicated SQL pool. Now, My data exists at both Data Lake and SQL pool. Suppose, I update some field values in this data (say "Manager"…
asked 2024-07-15T08:16:44.56+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 79%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAT%3C/text%3E%3C/svg%3E)
Apurva Tiwari
40
Reputation points
commented 2024-07-16T08:55:05.3466667+00:00
AnnuKumari-MSFT
32,011
Reputation points Microsoft Employee
2 answers
One of the answers was accepted by the question author.
Where do i find Azure HTTP status 429 response-code-examples
Where do i find Azure HTTP status 429 response-code-examples for: Azure Service BUS Azure Event Hub Azure Event Grid Azure Blob Storage Microsoft Fabric Azure Blob Storage und MS Fabric benutzt Azure Datalake Storage gen2 API
asked 2024-07-12T14:27:28.4266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 35%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Richard Geary
20
Reputation points
commented 2024-07-15T08:20:01.7666667+00:00
Richard Geary
20
Reputation points
2 answers
How to copy files from storage account to another storage account with the same last modified date using ADF
Hi Team, The request is that, when we are running ADF copy job to copy data from Landing storage account to Prod Storage account the last modified date coming as present copy time, but we want to copy the last modified date, what we have in landing…
asked 2024-06-20T15:58:52.0233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(25.6, 2%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
Mahasiva, Ravi (CSW)
60
Reputation points
commented 2024-07-15T07:37:43.9133333+00:00
Nehruji R
4,531
Reputation points Microsoft Vendor
2 answers
This request is not authorized to perform this operation using this permission.", 403, HEAD Synapse connect to adls
I am trying to select data from ADLS Gen2 storage delta table and keep receiving this error. I added the synapse service principal as storage blob data contributor and ACLs to container with no luck. Firewall is set to enable all networks as well. Please…
asked 2024-06-22T20:29:54.0233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 87%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDW%3C/text%3E%3C/svg%3E)
Daniel Woodhouse
0
Reputation points
commented 2024-07-15T07:32:42.4733333+00:00
Nehruji R
4,531
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
Want to migrate from one Synapse workspace to another Synapse workspace
Hi team, Want to migrate from one Synapse workspace to another Synapse workspace. One being Dev environment, another one test environment. Please provide leads. Regards, NagaSri
asked 2021-10-18T15:17:49.867+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 80%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EN%3C/text%3E%3C/svg%3E)
Naga
66
Reputation points
answered 2024-07-12T15:07:38.5833333+00:00
Mehdi Belkhiria
0
Reputation points Microsoft Employee
1 answer
whitelist the serverless data plane subnets in the cloud region
I am following below instructions from the documentation to whitelist the serverless data plane subnets in the cloud region of your Databricks workspace. But unable to find ARM resource ID of the serverless compute subnet details …
asked 2024-07-10T05:35:31.7966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(25.6, 17%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Santhosh Singh (ext)
0
Reputation points
commented 2024-07-12T05:30:06.1266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
12,615
Reputation points Microsoft Vendor
1 answer
Unable to connect Fivetran with public IP to ADLS Gen2 in Vnet with Private end point
Hi, I'm encountering an issue while trying to connect Fivetran to ADLS Gen2 in the UK South region. Here are the details: I have configured ADLS Gen2 as a destination in Fivetran and completed the prerequisites, including creating an SPN with appropriate…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 15%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
1 answer
Accessing Fabric Lakehouse via Logic App
Has anyone had any luck connecting a logic app to a lakehouse in Microsoft Fabric? Some additional background: We have a data pipeline that is ultimately generating a summary of some actions taken within the pipeline. This summary is stored as an Excel…
asked 2024-07-10T02:35:07.8866667+00:00
JC
10
Reputation points
answered 2024-07-10T06:54:33.51+00:00
Sumarigo-MSFT
45,406
Reputation points Microsoft Employee