Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,351 questions
1 answer
One of the answers was accepted by the question author.
is azure data lake analytics supports onlu storage Gen1
is azure data lake analytics account only support data lake storage Gen1? Can blob storage be use for data analytics account? data analytics account locations are very limited. if storage account is in different location than analytics account, will…
asked 2020-06-09T14:50:27.063+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 42%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESD%3C/text%3E%3C/svg%3E)
Seuli Das Majumder
101
Reputation points
commented 2022-03-17T19:58:39.99+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 6%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
Michael Rys [MSFT]
11
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Data Lake site identification
Hi again, I am extracting sql tables and loading to data factory (data lake). I am extracting the same tables from multiple locations. When I query the datalake, I need to be able to identify which data references each location. It is about 30…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 17%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
2 answers
ADF: How to connect to Azure data lake Gen 1 to access csv files and delete the row from the file.
ADF: How to connect to Azure data lake Gen 1 to access csv files and delete the row from the file. while i tried to create Data flow with source as Azure data lake Gen1 the connection itself is failing with the below error: Azure datalakestorage…
asked 2022-03-14T14:24:22.963+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 33%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESC%3C/text%3E%3C/svg%3E)
Shilpa Chilamkurthy
1
Reputation point
answered 2022-03-17T08:07:16.577+00:00
AaronHughes
391
Reputation points
1 answer
Azure Blob Encoding issue
Hi Folks, I have a client dropping their CSV file into Azure Blob but when I download the file, everything is good but when I use the edit view, it has wierd symbol. When I use a power automate to get the file content, I also do have wierd symbol Any…
asked 2022-03-07T16:20:25.447+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 11%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMP%3C/text%3E%3C/svg%3E)
Martin Pesant
11
Reputation points
commented 2022-03-16T10:24:23.157+00:00
PRADEEPCHEEKATLA-MSFT
77,901
Reputation points Microsoft Employee
2 answers
Azure Data Factory Power Query error: "We could not evaluate this query due to invalid or missing credentials"
I'm trying to add a Power Query resource to Azure Data Factory v2 (ADF). The dataset that I'm consuming is Parquet. I can successfully preview the data from within the dataset. As well, I can open and read the .parquet file from it's location in Azure…
asked 2022-03-02T22:07:35.563+00:00
John Palmer
66
Reputation points
commented 2022-03-14T17:14:06.827+00:00
AnnuKumari-MSFT
31,146
Reputation points Microsoft Employee
2 answers
One of the answers was accepted by the question author.
Data lake Gen2 Linked service for connecting to a container
Hi Team, I would like to create a Data lake Gen2 Linked service for direct connection to a container. Storage account has multiple container's and want to get the LS connected to one particular container alone. I.e. Attached a screenshot of a…
asked 2022-03-09T08:40:18.54+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(41.6, 24%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Rinshad Koissan
51
Reputation points
commented 2022-03-10T03:24:11.68+00:00
PRADEEPCHEEKATLA-MSFT
77,901
Reputation points Microsoft Employee
0 answers
How to dynamically rename files to data lake while copying from source (File share) with sub folder structure using ADF?
Hi Team, How to dynamically rename files to data lake while copying from source (File share) with sub folder structure using ADF? Folder L01 > Folder L11 > Folder L21 > Folder L31 > Folder L41 > File_A.csv Folder L01 >…
asked 2022-02-22T05:33:38.34+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 31%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFD%3C/text%3E%3C/svg%3E)
Frank D'Souza
1
Reputation point
commented 2022-03-09T19:44:08.727+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 26%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HimanshuSinha-msft
19,381
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
ADLS Gen2 Storage RBAC for user from differnt AAD tenant
HI, I have a service which is used by users from different AAD tenant like user1@a.onmicorosoft.com and user2@b.onmicrosoft.com. my blob storage is in my app AAD tenant. I want to assign permissions to users from different AAD tenant(to their…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 22%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGV%3C/text%3E%3C/svg%3E)
2 answers
How to connect Business Central to Azure Data Factory
I posted this question on Re: [MicrosoftDocs/azure-docs] Business Central Connector (Issue #88574) But that didn’t help. Is it actually possible to connect Business Central to ADF?
asked 2022-02-26T12:37:04.963+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 8%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anonymous
answered 2022-03-09T16:45:40.367+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 98%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBZ%3C/text%3E%3C/svg%3E)
Ben Zhai
6
Reputation points
1 answer
One of the answers was accepted by the question author.
loading each object in an array vs the page
Greetings, I am just now getting started with Data Factory and am a little confused. Below is a sample of the JSON from the REST API that I am pulling. I have the pipeline working to paginate and save blobs, but it only saves each page and not each…
asked 2022-03-04T17:20:55.563+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 89%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJP%3C/text%3E%3C/svg%3E)
Joshua P. Thomas
21
Reputation points
accepted 2022-03-09T16:33:56.847+00:00
Joshua P. Thomas
21
Reputation points
1 answer
One of the answers was accepted by the question author.
Failure happened on 'Source' side. ErrorCode=UserErrorOdbcOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException
I was trying to load data from on-prem data lake to azure gen2 using copy activity through SHIR. Facing this issue. Failure happened on 'Source' side.…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 77%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV%3C/text%3E%3C/svg%3E)
2 answers
Getting unauthorized response with DataLake Gen 2 link
I'm having trouble with some simple data queries. I'm using an account key to authorize connections to Data Lake storage Gen 2. First I set the account key (I know not best practice storing key in notebook but I'm just trying to get something…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 93%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 24%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
1 answer
Error while using COPY INTO command to load data
Getting this error 'Not able to validate external location because The remote server returned an error: (403) Forbidden.' while using copy into command to load data from adls gen 2 path parquet to synapse internal table using SAS key .i…
asked 2022-02-16T12:56:49.603+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 55.00000000000001%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EG%3C/text%3E%3C/svg%3E)
GSMATTA
1
Reputation point
commented 2022-03-04T07:01:48.543+00:00
ShaikMaheer-MSFT
37,896
Reputation points Microsoft Employee
1 answer
ADF Copy Data REST API , putting JSON response to SQL Table
Hi, I am trying to call an API using REST connector in COPY DATA . However when I use SQL Table in SINK and run pipeline, the JSON is not getting stored in the table instead the NULL value is being inserted into the table. I have not defined any…
asked 2022-02-04T04:42:50.03+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 44%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
Mav
81
Reputation points
commented 2022-03-04T06:32:35.947+00:00
ShaikMaheer-MSFT
37,896
Reputation points Microsoft Employee
1 answer
Using Azure Data Factory to send TSV files to a Snowflake stage, and it is escaping certain characters incorrectly.
I am using a Azure Data Factory pipeline to move compressed TSV files from an on-prem file share to a ADLS share that acts as a Snowflake stage, so we can move it to a Snowflake table. The problem is when the combination of a backslash and a newline…
asked 2022-02-03T19:25:28.937+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(313.6, 71%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELS%3C/text%3E%3C/svg%3E)
Leif Starbuck
1
Reputation point
commented 2022-03-04T06:30:46.627+00:00
ShaikMaheer-MSFT
37,896
Reputation points Microsoft Employee
1 answer
data lake and schemas
Since data lakes store data in its raw form from the original data source how do you manage providing data to users to create reports when they're accustomed to snowflake and star schemas?
asked 2022-02-03T00:41:32.57+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 70%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETT%3C/text%3E%3C/svg%3E)
Thirston Third
21
Reputation points
commented 2022-03-04T06:28:18.597+00:00
ShaikMaheer-MSFT
37,896
Reputation points Microsoft Employee
1 answer
Azure data factory pipeline copy activity failure
I am getting " The new name (table name) is already in use as a object name and would a duplicate that is not permitted" in Azure data factory pipeline using copy activity. This pipeline will get triggered when a code is executed and a file is…
asked 2022-02-21T08:06:04.46+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(259.20000000000005, 86%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Arunkumar R
1
Reputation point
commented 2022-03-04T05:58:04.427+00:00
Arunkumar R
1
Reputation point
0 answers
Trying to use power query on ADF error: "We could not evaluate this query due to invalid or missing credentials"
Hi I am trying to use power query on azure data factory and I am receiving the following error when I add a data source from our azure data lake: "We could not evaluate this query due to invalid or missing credentials" (see image below). …
asked 2021-11-18T12:46:54.15+00:00
Anonymous
commented 2022-03-02T22:14:27.727+00:00
John Palmer
66
Reputation points
0 answers
Error managing tables dataverse to data lake gen 2
Hi, I have used Azure Synapse Link for Dataverse to sync some tables over to a data lake storage. For many tables this works fine, but for some of the "Append only" tables, the isDelete column is missing. If I remove some of the tables and…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(32, 68%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
1 answer
How to dynamically map and copy each XML data response (via API/HTTP) in Azure Data Factory ??
Hello, We have a Web API source that returns responses(data) in XML format. We need to extract data for multiple objects from this source/method and hence using HTTP Web API as datasource/dataset/linkedservice with XML format. Here's a sample response…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 72%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)