Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,350 questions
2 answers
One of the answers was accepted by the question author.
Design strategy for Data lake
Hi friends, we need to design an Azure data lake from scratch for a solution that has complex, multiple data sources from Databases, Rest APIs, and custom applications, and the Data lake solution should be scalable and high-performance in terms of data…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 15%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
asked 2024-05-01T06:57:01.2766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(179.20000000000002, 53%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Alpha
0
Reputation points
commented 2024-05-06T07:02:36.8333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 81%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Amrinder Singh
2,200
Reputation points Microsoft Employee
1 answer
401 Unauthorized "Audience validation failed" from ADLS endpoint
We are using service principal credentials to authenticate using OAuth2 token. Fetching the access token step is successful however, request getting failed when hitting ADLS endpoint . Error response : Exact error message :…
asked 2024-05-03T11:11:59.4933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 2%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Shubhank
0
Reputation points
answered 2024-05-06T06:59:35.34+00:00
Amrinder Singh
2,200
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Data size of databricks delta tables
It has been observed that the size of delta tables are much less as compared to when checked the underlying delta files in the storage account. Suppose a databricks delta table raw.deltaTableA has size of 2MB if we check the size of underlying delta…
asked 2024-05-02T09:39:01.4133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 23%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENK%3C/text%3E%3C/svg%3E)
NIKHIL KUMAR
101
Reputation points
accepted 2024-05-05T14:27:03.56+00:00
NIKHIL KUMAR
101
Reputation points
6 answers
Error trying to validate storage account name while creating a new synapse workspace
I am unable to create a new data lake storage (gen2) account name in the basic tab for create Azure Synapse Workspace. The error I get is - " There was an error trying to validate storage account name. please try again ". The error message is…
asked 2022-08-03T07:01:54.587+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 2%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
Jozure
446
Reputation points
commented 2024-05-04T05:39:05.8733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 54%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAH%3C/text%3E%3C/svg%3E)
Ahmed Helal
0
Reputation points
2 answers
One of the answers was accepted by the question author.
Blob Capacity by Tier
Hi I was wondering if there is a way to get the blob or usage capacity for storage accounts broken down by tier through rest api?
asked 2024-05-01T19:58:31.9633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 64%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Morris, Michael A
40
Reputation points
accepted 2024-05-02T20:45:55.0533333+00:00
Morris, Michael A
40
Reputation points
0 answers
How to store set variable json array output into csv file in data factory
I used the Lookup activity under for each loop and used Store Procedure which gave me some record as an output if the condition is fails So that output I have stored into the Set Variable and then those values I need to store into csv file whose file…
asked 2024-04-26T18:01:30.62+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 61%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
Mohit Bokade
0
Reputation points
commented 2024-05-02T13:44:22.6766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
10,000
Reputation points Microsoft Vendor
1 answer
Azure Synapse Link with F&O missing tables
Hello, I'm facing this problem with synapse link, I did: Connect F&O environment with power apps. Established an incremental link with data lake. I can see D365 Finance and Operation in the list of tables in manage tables of the link. My problem…
asked 2024-03-19T08:57:53.9533333+00:00
Ameer Jaber
0
Reputation points
edited a comment 2024-05-02T10:10:16.0533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 3%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Rupa Kamble (Accenture Solutions Private Li)
0
Reputation points Microsoft Vendor
2 answers
Unable to create Storage Event trigger in Synapse Data Factory
I want to create a pipeline triggered by an event in my storage account : when a blod is created inside.nWhen I want to publish that event trigger in synapse, I have this error : The client 'd4d9f262-75fa-4138-845c-019afa12cf7a' with object id…
asked 2024-04-30T12:22:49.8033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 83%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJS%3C/text%3E%3C/svg%3E)
Junior Steve KAMDEM DJOKO
0
Reputation points
answered 2024-05-02T09:11:16.7+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 10%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAP%3C/text%3E%3C/svg%3E)
Anand Prakash Yadav
6,085
Reputation points Microsoft Vendor
2 answers
Copy LOB based data from DB2 to Azure Data Lake Gen2 storage using ADF
Hi team, I was trying to work on a scenario to copy data from a table in db2 database having a column with datatype as LOB(CLOB) to Azure Data Lake Gen2 storage container using Azure Data Factory. However, its taking a considerable amount of time( a day)…
asked 2024-04-21T03:40:26.3+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 48%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDM%3C/text%3E%3C/svg%3E)
Debajit Mitra
0
Reputation points
commented 2024-05-01T10:02:15.8666667+00:00
Amira Bedhiafi
15,751
Reputation points
3 answers
Filtering data for last 24 months in Mapping Data Flows
How can I filter data using Mapping Data Flows for the last 24 months, starting from the max date and going back? The date column is in the mm/dd/yyyy format.
asked 2024-03-19T01:29:58.9966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 41%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
ADITYA SINGH
0
Reputation points
edited an answer 2024-05-01T08:15:23.97+00:00
Pinaki Ghatak
2,400
Reputation points Microsoft Employee
0 answers
How to fix error Job failed due to reason: com.microsoft.dataflow.broker.InvalidPayload
Operation on target Country failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: com.microsoft.dataflow.broker.InvalidPayloadException: Fail to validate with reason: Retry Request, fail to…
asked 2024-04-29T13:20:24.6633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 17%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDB%3C/text%3E%3C/svg%3E)
Dhruva B
50
Reputation points
commented 2024-04-30T15:54:52.89+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BhargavaGunnam-MSFT
26,316
Reputation points Microsoft Employee
3 answers
How to send a mail notification for a failed pipeline in Azure Synapse Analytics?
How can I send a notification email to a specific email address without using a logic app when one of my Synapse Analytics pipelines fails? I would like to include the error message in the email notification.
asked 2024-03-20T15:48:58.53+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 69%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Aditya Singh
105
Reputation points
commented 2024-04-30T15:30:48.9533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 38%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERC%3C/text%3E%3C/svg%3E)
Roberto Cuevas
0
Reputation points
1 answer
How to maintain same folder structure as source while sink the processed file
I have a requirement to process the JSON to parquet on daily basis. I have folder A,B,C needs to sink the file to another container with same structure as A,B,C for example if I'm processing a file from folder A it should sink to output container folder…
asked 2024-04-26T08:36:30.47+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 11%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Venkatesh Srinivasan
0
Reputation points
answered 2024-04-30T11:36:19.9033333+00:00
Smaran Thoomu
10,000
Reputation points Microsoft Vendor
3 answers
One of the answers was accepted by the question author.
Azure blob storage gen2
I need to restrict uploading files to Blob storage at 18 MB size and allow only .txt extension
asked 2024-04-26T10:54:48.57+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(284.8, 83%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECS%3C/text%3E%3C/svg%3E)
Chandru Sati
20
Reputation points
accepted 2024-04-29T01:54:10.3433333+00:00
Chandru Sati
20
Reputation points
1 answer
How can i create a linked service in ADF for Sharepoint online?
I want to extract files from sharepoint to ADL by only using ADF. I followed few steps Step1: Azure Active Directory -> Registered new app -> created new secret key I have the Tenant ID, ClientID(App ID), Secret Key Step2: Sharepoint online ->…
asked 2024-04-27T21:58:38.2233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 73%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Siddartha Reddy Jammula
0
Reputation points
edited an answer 2024-04-28T02:56:38.9766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 98%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KarishmaTiwari-MSFT
18,527
Reputation points Microsoft Employee
1 answer
Azure logic app , extract file from sharepoint to ADLS
Hi, I am working on a file extraction from sharepint to ADLS and the file size is 1 GB. I need to copy the file to ADLS. I am currently using standard logic app as the file size is more than the default size. After that i tried adding the file size in…
asked 2024-04-26T20:06:43.52+00:00
Ritu Harivansh Prajapati
0
Reputation points
answered 2024-04-27T04:20:28.6433333+00:00
Abhishek Tripathi
0
Reputation points
0 answers
Download data from cosmos by v-studio
When I download data from cosmos from azure data lake storage of visual studio, it shows an error: 流 URL 无效
asked 2024-04-16T12:07:24.16+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(169.60000000000002, 2%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EXW%3C/text%3E%3C/svg%3E)
Xinlong Wang
0
Reputation points Microsoft Employee
commented 2024-04-26T21:33:55.55+00:00
KarishmaTiwari-MSFT
18,527
Reputation points Microsoft Employee
1 answer
Copy Data activity fails when copying from ADLS to ADLS
I'm currently attempting to use a Copy Data activity to copy a .json file over from what we treat as a "staging area" of our data over to our Datalake all within Azure Datalake Storage. The files start out in .json format but are copied over…
asked 2024-04-24T14:31:33.2366667+00:00
Christopher Ackerman
11
Reputation points
answered 2024-04-25T06:42:19.9933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
Nehruji R
2,051
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Can't deploy python function with the module azure-identity or azure-storage-filedatalake
I'm trying to deploy my function that I've been working on locally, whenever I add the module "azure-storage-filedatalake" or the module "azure-identity" my function doesn't appear on the function app anymore, but when I remove this…
asked 2024-04-18T09:55:12.23+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 33%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAB%3C/text%3E%3C/svg%3E)
Ayoub Ben Yamoune
20
Reputation points
accepted 2024-04-24T08:27:54.5833333+00:00
Ayoub Ben Yamoune
20
Reputation points