Azure SQL Database
An Azure relational database service.
5,730 questions
1 answer
Copy Dataverse data into Azure SQL using Synapse Link
Im using Azure Synapse pipeline to copy data from Dataverse to Azure SQL DB. For large tables, it is only copying 10 million records and it is not inserting after. I have about 15 mil records. I dont see any settings limitation anywhere on the pipeline.…
asked 2024-10-23T15:49:18.2433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 55.00000000000001%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
Murali Ramasamy
5
Reputation points
edited a comment 2024-10-30T05:22:00.5933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
11,125
Reputation points Microsoft Vendor
2 answers
Azure Data Factory CopyData from DB on-premise to Azure using SHIR
Hi, I had set up Self Hosted Integration Runtime on my on-premise VM with azure cloud using Azure Data Factory. The connection is already been tested and checked green on both integration runtime and linked services. My goal is to be able to copy data…
asked 2023-06-07T09:02:10.1233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 47%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYC%3C/text%3E%3C/svg%3E)
Yudha Cipta Maulana
0
Reputation points
commented 2024-10-30T03:18:06.5833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 24%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESL%3C/text%3E%3C/svg%3E)
Shobna Lata
20
Reputation points
1 answer
Copy activity failed because you have multiple concurrent copy activities runs writing to the same file
Hi All, I am migrating 1000 of SQL tables from on premise SQL server to Azure Blob storage. I am using ADF for each and copy activity to do so. However, while processing the tables in parallel/concurrent, I am getting below error - Failure happened…
asked 2024-10-24T11:00:55.3666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 27%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Magarde, A (Ankit)
0
Reputation points
commented 2024-10-29T08:24:12.72+00:00
AnnuKumari-MSFT
33,241
Reputation points Microsoft Employee
3 answers
PATCH method not allowed for Storage Services REST API "Path - Update"
I try to use the Set Access Control option described in https://learn.microsoft.com/en-us/rest/api/storageservices/data-lake-storage-gen2?view=rest-storageservices-datalakestoragegen2-2019-12-12. Although I was able to run other methods (as Path - List)…
asked 2024-10-22T09:31:41.02+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 49%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Scowl
5
Reputation points
answered 2024-10-29T00:36:14.3533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 98%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KarishmaTiwari-MSFT
20,127
Reputation points Microsoft Employee
2 answers
One of the answers was accepted by the question author.
Why Data asset is not supported when try to create a AutoML job?
Hi, I successuffly created a data asset (folder_URI type) with uploaded imges in an Azure Blob storage (registered storage source), but when I try to create a job in Azure AutoML the Data asset shows (not supported). Any idea what is the issue? Thanks
asked 2023-01-12T23:11:21.94+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 89%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGB%3C/text%3E%3C/svg%3E)
Gonzalez Barba, Jorge
20
Reputation points
commented 2024-10-28T15:07:41.1633333+00:00
Gonzalez Barba, Jorge
20
Reputation points
1 answer
Is there a way to use query acceleration on .avro files
I have a data lake storage resource in which I am storing .avro files. I now have a need to transfer the data in those files to a database. I've managed to make that work. But I would love to optimize it by only downloading data I need. I read this…
asked 2024-10-24T10:20:14.7966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 31%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFM%3C/text%3E%3C/svg%3E)
Filip Mattsson
0
Reputation points
answered 2024-10-24T18:46:25.45+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 6%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKD%3C/text%3E%3C/svg%3E)
Keshavulu Dasari
1,065
Reputation points Microsoft Vendor
1 answer
About right naming convention on paths and blobs
Hi, I'm used to design ADLS partition in the following format: source={source}/table={table}/date={date}/content.csv When a table in any source doesn't have data in certain date, that partition won't exists. In the other hand, not every table will…
2 answers
One of the answers was accepted by the question author.
Azure Datalake query accelaration error XML specified is not syntactically valid.\nRequestId:e3204a59-901e-005f-7612-d2ee5f000000\nTime:2021-11-05T06:58:32.8652708Z\r\nStatus: 400 (XML specified is not syntactically valid.
I am using query acceleration to access and filter data lake- https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-query-acceleration-how-to?tabs=dotnet%2Cazure-cli I followed the same steps as per the document to the letter. but…
asked 2021-11-05T07:07:13.077+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 17%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
DevBuster007
186
Reputation points
commented 2024-10-17T09:58:52.64+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 99%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJA%3C/text%3E%3C/svg%3E)
James Anthony
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Power BI JSON file 1000 row limit
Hi all, I am facing issues with Power BI only pulling in 1000 rows. I have exported JSON data into azure data lake storage gen 2, which then our business analyst was able to pull in the files and create separate tables in a pbix file. The issue he has…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 17%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
Why is the activity log stored in the adls gen2 container not a valid json
Why is the activity log stored in the adls gen2 container not a valid json? It's line by line, and the outer layer is missing '[]' and comma separation
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 53%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZ%3C/text%3E%3C/svg%3E)
1 answer
partitioning of Data lake
I have just the below link on data partitioning of Azure data lake : https://learn.microsoft.com/en-us/azure/architecture/best-practices/data-partitioning But it is too brief and does not cover this important topic extensively. I request you to please…
asked 2024-10-08T10:09:02.7433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
300
Reputation points
commented 2024-10-15T18:56:04.92+00:00
Keshavulu Dasari
1,065
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
DataLakeServiceClient.DeleteFileSystem() method on non-ADLSv2 upgraded storage accounts
The .NET codebase I work on interacts with some storage accounts that are upgraded to ADLSv2 and some that are not. I noticed that the DataLakeServiceClient.DeleteFileSystem() method works in both cases. Can I get a confirmation that the method's…
asked 2024-10-08T00:52:39.42+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 39%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Max Morrison
30
Reputation points Microsoft Employee
accepted 2024-10-15T00:58:33.5466667+00:00
Max Morrison
30
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Azure datalake and data consistency
Hi Experts, Azure Data Lake Storage (ADLS) does not natively provide full ACID (Atomicity, Consistency, Isolation, Durability) transaction support unlike traditional relational databases designed to support ACID transactions. This raises the following…
asked 2024-10-12T09:44:01.3866667+00:00
azure_learner
300
Reputation points
accepted 2024-10-13T11:24:21.8266667+00:00
azure_learner
300
Reputation points
1 answer
Best way to grant SQL DB (serverless sql pool) access to end user.
What is the best way to grant access to users for SQL DB(serverless sql pool) build on top of synapse azure lake db. How should SQL DB (views or external tables) access the lake database (storage container). Should we use RBAC, Managed Identity, ACLS,…
asked 2024-10-08T00:04:18.47+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(240, 44%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAF%3C/text%3E%3C/svg%3E)
Abbas Furniturewala
0
Reputation points
commented 2024-10-10T07:05:46.4733333+00:00
phemanth
11,125
Reputation points Microsoft Vendor
1 answer
Data lake performance and improvement
The data lake performance depends on critical factors such as partitioning on date and if available in data region etc. I have already partitioned the data lake in a hierarchal structure LandingZone Subject area YYYY MM …
asked 2024-10-07T16:48:20.0033333+00:00
azure_learner
300
Reputation points
answered 2024-10-08T08:13:19.6033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 31%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVK%3C/text%3E%3C/svg%3E)
Vinod Kumar Reddy Chilupuri
575
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
can abac be applied to a data lake?
Can abac be applied to a data lake storage account or does it have to be rbac with potential ACL?
asked 2024-09-13T04:26:59.0133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 22%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EML%3C/text%3E%3C/svg%3E)
Michael Liveri
40
Reputation points
accepted 2024-10-03T23:50:18.7066667+00:00
Michael Liveri
40
Reputation points
2 answers
Azure data lake folder structure
Hi, the link provided in the below thread is not working, any way to find the information or a new URL/link: FAQs About Organizing a Data…
asked 2024-09-19T08:43:49.7+00:00
azure_learner
300
Reputation points
commented 2024-10-03T09:16:32.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
Nehruji R
8,066
Reputation points Microsoft Vendor
2 answers
Copy Activity very slow for Azure data factory
Hi All, we are having issue where copy activity from Azure Table storage to data factory is taking over 50mins on daily basis. it used to take 15-20mins until a week back. What's the source of the slow copy activity? Azure Table storage What's the sink…
asked 2024-09-13T03:50:38.0833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 74%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMS%3C/text%3E%3C/svg%3E)
Mayuri Shah
0
Reputation points
commented 2024-10-03T05:41:48.32+00:00
AnnuKumari-MSFT
33,241
Reputation points Microsoft Employee
0 answers
Filepath() ,metadata functions in serverless view on delta format
Hi , Earlier i was creating serverless views with FORMAT='PARQUET' using openrowset wildcard paths ("/synapsetesing/test/*.parquet"), so i was able to use metadata functions like filepath etc , now when i create views on FORMAT='DELTA' ,…
asked 2021-11-08T07:20:29.68+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 8%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anonymous
commented 2024-10-03T01:27:05.81+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 0%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGK%3C/text%3E%3C/svg%3E)
Gilbert, Keith
0
Reputation points
1 answer
One of the answers was accepted by the question author.
ADF pipeline to read the data from UC table to adls gen2 account
Hello Team, We have a requirement to create Azure Datafactory pipeline to read the data from UC table, access on the table is provided ( to Azure Datafactory Managed Identity) and copy the data into adls gen2. Is there a way or article to implement this?…
asked 2024-04-11T19:05:05.9733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(51.2, 77%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAG%3C/text%3E%3C/svg%3E)
Ashwini Gaikwad
130
Reputation points
edited a comment 2024-10-01T08:27:53.21+00:00
Kuldeep Singh
16
Reputation points