Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,480 questions
1 answer
How cn i restrict someone from not downloading any data from Fabric Lakehouse or Fabric warehouse
As a data admin, i want to control data access for a user in your Microsoft Fabric Warehouse. The goal is to allow this user, who has a Contributor role, to view data directly in the workspace without being able to download it as a file. This scenario…
asked 2024-11-05T05:14:23.56+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 11%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJS%3C/text%3E%3C/svg%3E)
Jain, Shreyans
0
Reputation points
answered 2024-11-05T06:49:32.66+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 2%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHB%3C/text%3E%3C/svg%3E)
Hari Babu Vattepally
325
Reputation points Microsoft Vendor
0 answers
How cn i restrict someone from not downloading any data from Fabric Lakehouse or Fabric warehouse
As a data admin, i want to control data access for a user in your Microsoft Fabric Warehouse. The goal is to allow this user, who has a Contributor role, to view data directly in the workspace without being able to download it as a file. This scenario…
asked 2024-11-05T05:24:13.73+00:00
Jain, Shreyans
0
Reputation points
edited the question 2024-11-05T06:18:19.8333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 47%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
Mounika Reddy Anumandla
665
Reputation points Microsoft Vendor
1 answer
Data lake schema enforcement
Hello, In Data Lake data is processed or ingested as schema on read and that is data is read in it format that it comes from the source. But I read an article that says schema enforcement makes data lakes high-performance and data readable. Please…
asked 2024-11-03T14:25:43.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
340
Reputation points
commented 2024-11-05T05:11:48.32+00:00
Hari Babu Vattepally
325
Reputation points Microsoft Vendor
1 answer
Copy activity failed because you have multiple concurrent copy activities runs writing to the same file
Hi All, I am migrating 1000 of SQL tables from on premise SQL server to Azure Blob storage. I am using ADF for each and copy activity to do so. However, while processing the tables in parallel/concurrent, I am getting below error - Failure happened…
asked 2024-10-24T11:00:55.3666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 27%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Magarde, A (Ankit)
0
Reputation points
commented 2024-11-04T14:47:01.6266667+00:00
AnnuKumari-MSFT
33,241
Reputation points Microsoft Employee
3 answers
One of the answers was accepted by the question author.
PATCH method not allowed for Storage Services REST API "Path - Update"
I try to use the Set Access Control option described in https://learn.microsoft.com/en-us/rest/api/storageservices/data-lake-storage-gen2?view=rest-storageservices-datalakestoragegen2-2019-12-12. Although I was able to run other methods (as Path - List)…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 49%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
2 answers
One of the answers was accepted by the question author.
Azure data lake folder structure
Hi, the link provided in the below thread is not working, any way to find the information or a new URL/link: FAQs About Organizing a Data…
asked 2024-09-19T08:43:49.7+00:00
azure_learner
340
Reputation points
accepted 2024-11-03T13:53:16.2866667+00:00
azure_learner
340
Reputation points
2 answers
Azure Data Factory CopyData from DB on-premise to Azure using SHIR
Hi, I had set up Self Hosted Integration Runtime on my on-premise VM with azure cloud using Azure Data Factory. The connection is already been tested and checked green on both integration runtime and linked services. My goal is to be able to copy data…
asked 2023-06-07T09:02:10.1233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 47%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYC%3C/text%3E%3C/svg%3E)
Yudha Cipta Maulana
0
Reputation points
commented 2024-10-30T03:18:06.5833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 24%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESL%3C/text%3E%3C/svg%3E)
Shobna Lata
20
Reputation points
2 answers
One of the answers was accepted by the question author.
Why Data asset is not supported when try to create a AutoML job?
Hi, I successuffly created a data asset (folder_URI type) with uploaded imges in an Azure Blob storage (registered storage source), but when I try to create a job in Azure AutoML the Data asset shows (not supported). Any idea what is the issue? Thanks
asked 2023-01-12T23:11:21.94+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 89%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGB%3C/text%3E%3C/svg%3E)
Gonzalez Barba, Jorge
20
Reputation points
commented 2024-10-28T15:07:41.1633333+00:00
Gonzalez Barba, Jorge
20
Reputation points
1 answer
Is there a way to use query acceleration on .avro files
I have a data lake storage resource in which I am storing .avro files. I now have a need to transfer the data in those files to a database. I've managed to make that work. But I would love to optimize it by only downloading data I need. I read this…
asked 2024-10-24T10:20:14.7966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 31%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFM%3C/text%3E%3C/svg%3E)
Filip Mattsson
0
Reputation points
answered 2024-10-24T18:46:25.45+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 6%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKD%3C/text%3E%3C/svg%3E)
Keshavulu Dasari
1,065
Reputation points Microsoft Vendor
1 answer
About right naming convention on paths and blobs
Hi, I'm used to design ADLS partition in the following format: source={source}/table={table}/date={date}/content.csv When a table in any source doesn't have data in certain date, that partition won't exists. In the other hand, not every table will…
2 answers
One of the answers was accepted by the question author.
Azure Datalake query accelaration error XML specified is not syntactically valid.\nRequestId:e3204a59-901e-005f-7612-d2ee5f000000\nTime:2021-11-05T06:58:32.8652708Z\r\nStatus: 400 (XML specified is not syntactically valid.
I am using query acceleration to access and filter data lake- https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-query-acceleration-how-to?tabs=dotnet%2Cazure-cli I followed the same steps as per the document to the letter. but…
asked 2021-11-05T07:07:13.077+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 17%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
DevBuster007
186
Reputation points
commented 2024-10-17T09:58:52.64+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 99%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJA%3C/text%3E%3C/svg%3E)
James Anthony
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Power BI JSON file 1000 row limit
Hi all, I am facing issues with Power BI only pulling in 1000 rows. I have exported JSON data into azure data lake storage gen 2, which then our business analyst was able to pull in the files and create separate tables in a pbix file. The issue he has…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 17%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
Why is the activity log stored in the adls gen2 container not a valid json
Why is the activity log stored in the adls gen2 container not a valid json? It's line by line, and the outer layer is missing '[]' and comma separation
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 53%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZ%3C/text%3E%3C/svg%3E)
1 answer
partitioning of Data lake
I have just the below link on data partitioning of Azure data lake : https://learn.microsoft.com/en-us/azure/architecture/best-practices/data-partitioning But it is too brief and does not cover this important topic extensively. I request you to please…
asked 2024-10-08T10:09:02.7433333+00:00
azure_learner
340
Reputation points
commented 2024-10-15T18:56:04.92+00:00
Keshavulu Dasari
1,065
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
DataLakeServiceClient.DeleteFileSystem() method on non-ADLSv2 upgraded storage accounts
The .NET codebase I work on interacts with some storage accounts that are upgraded to ADLSv2 and some that are not. I noticed that the DataLakeServiceClient.DeleteFileSystem() method works in both cases. Can I get a confirmation that the method's…
asked 2024-10-08T00:52:39.42+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 39%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Max Morrison
30
Reputation points Microsoft Employee
accepted 2024-10-15T00:58:33.5466667+00:00
Max Morrison
30
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Azure datalake and data consistency
Hi Experts, Azure Data Lake Storage (ADLS) does not natively provide full ACID (Atomicity, Consistency, Isolation, Durability) transaction support unlike traditional relational databases designed to support ACID transactions. This raises the following…
asked 2024-10-12T09:44:01.3866667+00:00
azure_learner
340
Reputation points
accepted 2024-10-13T11:24:21.8266667+00:00
azure_learner
340
Reputation points
1 answer
Best way to grant SQL DB (serverless sql pool) access to end user.
What is the best way to grant access to users for SQL DB(serverless sql pool) build on top of synapse azure lake db. How should SQL DB (views or external tables) access the lake database (storage container). Should we use RBAC, Managed Identity, ACLS,…
asked 2024-10-08T00:04:18.47+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(240, 44%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAF%3C/text%3E%3C/svg%3E)
Abbas Furniturewala
0
Reputation points
commented 2024-10-10T07:05:46.4733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
11,125
Reputation points Microsoft Vendor
1 answer
Data lake performance and improvement
The data lake performance depends on critical factors such as partitioning on date and if available in data region etc. I have already partitioned the data lake in a hierarchal structure LandingZone Subject area YYYY MM …
asked 2024-10-07T16:48:20.0033333+00:00
azure_learner
340
Reputation points
answered 2024-10-08T08:13:19.6033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 31%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVK%3C/text%3E%3C/svg%3E)
Vinod Kumar Reddy Chilupuri
655
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
can abac be applied to a data lake?
Can abac be applied to a data lake storage account or does it have to be rbac with potential ACL?
asked 2024-09-13T04:26:59.0133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 22%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EML%3C/text%3E%3C/svg%3E)
Michael Liveri
40
Reputation points
accepted 2024-10-03T23:50:18.7066667+00:00
Michael Liveri
40
Reputation points
2 answers
Copy Activity very slow for Azure data factory
Hi All, we are having issue where copy activity from Azure Table storage to data factory is taking over 50mins on daily basis. it used to take 15-20mins until a week back. What's the source of the slow copy activity? Azure Table storage What's the sink…
asked 2024-09-13T03:50:38.0833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 74%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMS%3C/text%3E%3C/svg%3E)
Mayuri Shah
0
Reputation points
commented 2024-10-03T05:41:48.32+00:00
AnnuKumari-MSFT
33,241
Reputation points Microsoft Employee