Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,807 questions
0 answers
Azure Data factory
Failure happened on 'Source' side. ErrorCode=SqlOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=A database operation failed with the following error: 'A transport-level error has occurred when receiving results…
asked 2024-11-03T11:12:13.94+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 52%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAB%3C/text%3E%3C/svg%3E)
Amarnath Bakthavachalam
0
Reputation points
asked 2024-11-03T11:12:13.94+00:00
Amarnath Bakthavachalam
0
Reputation points
0 answers
Issue while creating Azur Data Factory
I am new to Azure. When I am trying to create Azure data factory with sandboxes free trial. Not getting created and error message popping up is "Validation failed. Required information is missing or not valid". The error occurs in resource…
asked 2024-11-03T11:04:54.0233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 79%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVG%3C/text%3E%3C/svg%3E)
Vishal Gupta
0
Reputation points
asked 2024-11-03T11:04:54.0233333+00:00
Vishal Gupta
0
Reputation points
2 answers
datafactory copy activity with dynamic mapping
Hello, I am trying to modify my JSON pipeline file to use a translator instead of static mapping ... When I statically define the values like: translator": { "type": "TabularTranslator", …
asked 2024-11-01T16:55:39.2366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(160, 51%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJG%3C/text%3E%3C/svg%3E)
Jorge García
0
Reputation points
answered 2024-11-03T09:43:22.27+00:00
Jorge García
0
Reputation points
0 answers
trying to write data from adlsgen2 to synapse sql dedicated pool but getting data type error
I am trying to write data into a table from adls gen2 using a copy activity. when creating the table, i have to use varchar(max). varchar(8000) is not an option as my data is too big for that. when creating the table using dedicated sql pool, i am able…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 39%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
1 answer
ADF is billing high
We are in the middle of the development of an Azure project using ADF and other Azure services. The trigger scheduled and event base are used profusely by my colleagues as part of testing and it is causing billing issues. My question is does only…
asked 2024-11-02T16:54:32.6233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
300
Reputation points
answered 2024-11-02T17:21:00.17+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 38%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMP%3C/text%3E%3C/svg%3E)
Marcin Policht
25,290
Reputation points MVP
1 answer
Can't sink from Data Factory to Blob Storage after updating storage account public access to 'Enabled from selected virtual networks and IP addresses'.
I have an Azure Data Factory pipeline that is no longer able to access my storage account after I have updated the public network configuration to 'Enabled from selected virtual networks and IP addresses' in my storage account. I have followed the steps…
asked 2024-11-01T22:25:53.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 17%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJF%3C/text%3E%3C/svg%3E)
Joe Fried (TATA CONSULTANCY SERVICES LTD)
0
Reputation points Microsoft Vendor
edited an answer 2024-11-02T13:42:29.8766667+00:00
Amira Bedhiafi
25,866
Reputation points
2 answers
How to assign Variable Value to Parameter
Hi, we need to create a "batch_no" in the format YYYYMMDDHHMMSS within our pipeline as a human-readable job identifier , since system variables like pipeline run ID or trigger ID aren’t easily readable. The "batch_no" should remain…
asked 2024-11-02T02:51:24.05+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 43%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAZ%3C/text%3E%3C/svg%3E)
Amy Zhang
25
Reputation points
answered 2024-11-02T13:38:43.09+00:00
Amira Bedhiafi
25,866
Reputation points
2 answers
Unable to search for Purview assets in Data Factory
Hi, When I search for the Purview assets in Data Factory, the search keeps happening without yielding any result. However, I am able to search directly in the Purview portal. Please let me know a resolution.
asked 2024-10-25T18:37:48.55+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 70%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
Ravikiran Srinivasulu
10
Reputation points
commented 2024-11-02T05:21:29.05+00:00
Ravikiran Srinivasulu
10
Reputation points
1 answer
Error from Lookup component in ADF not invoking subsequent component
We have a component running a stored procedure. This particular one returned too much data, which is throwing this error: ErrorCode=UserErrorComputePluginFailure,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The size of…
asked 2024-10-29T16:08:06.9533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(208, 73%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHB%3C/text%3E%3C/svg%3E)
Huckleby, Barbara A
0
Reputation points
commented 2024-11-02T04:44:52.76+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
16,640
Reputation points Microsoft Vendor
0 answers
In a private subscription I am unable to connect a power query activity from ADF to an Azure SQL DB
I had a powerbi report with semantic model that was exceeding our P2 premium node on refresh. I am looking to migrate the power query steps to ADF and sink it to a SQL Data Mart instead for consumption by the report. Since the Power Query activity only…
asked 2024-11-01T22:10:05.71+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 77%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJB%3C/text%3E%3C/svg%3E)
Jonathan Burbano
0
Reputation points
edited a comment 2024-11-02T01:28:21.0866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
945
Reputation points Microsoft Vendor
1 answer
Occassional Issue with passing parameters between activities
We are having a spotty issue where once in every few pipeline runs an activity will not fill in the parameters even though we run the same code 50 times and 49 of those times works. There doesn't seem to be any difference that we can find. Has there been…
asked 2024-10-31T19:01:41.81+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 20%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMF%3C/text%3E%3C/svg%3E)
Mike Fatal
0
Reputation points
edited an answer 2024-11-01T20:41:28.14+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Bhargava-MSFT
31,116
Reputation points Microsoft Employee
3 answers
Missing Records in Azure Log Analytics
Hi All, Logs of activities, pipelines, and triggers are gathered within the Synapse environment for monitoring purposes. It has been observed that some pipelines ran successfully, as seen in the Monitor section of Synapse, but there are missing records…
asked 2024-10-21T13:55:53.4933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 54%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJP%3C/text%3E%3C/svg%3E)
Jakub Pytlik
0
Reputation points
commented 2024-11-01T20:39:42.9233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 20%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPR%3C/text%3E%3C/svg%3E)
Pranay Reddy Madireddy
365
Reputation points Microsoft Vendor
1 answer
DataFactory DataFlow Activity ManageVnet dataflow cluster is not started
Hi All, we are unable to start the azure data factory data flow cluster . we are using Mange Vent IR in itally north region when we start the cluster in Debug Mode we get message-The startup of the cluster is taking longer than expected , we recommend…
asked 2024-10-24T12:33:08.52+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 20%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMV%3C/text%3E%3C/svg%3E)
manish verma
461
Reputation points
commented 2024-11-01T16:56:09.1766667+00:00
manish verma
461
Reputation points
1 answer
Can ADF load data from Kusto to Cosmos?
Can you please tell me can ADF load data from Kusto to Cosmos? If it can, how to do it? Please give me some guidance, I am a new hand. Thank you very much!
asked 2024-10-31T09:36:51.56+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 87%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYC%3C/text%3E%3C/svg%3E)
Yifan Cui
0
Reputation points Microsoft Employee
commented 2024-11-01T15:27:32.2766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
2,525
Reputation points Microsoft Vendor
2 answers
Unable to create dynamic linked service for Microsoft Access database
Hi Team I am trying to create Linked service for Microsoft Access database using dynamic connection string where i am passing connection string value through parameter, test connection working fine but I am getting following error while saving the linked…
asked 2024-10-28T17:43:18.2+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 11%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDK%3C/text%3E%3C/svg%3E)
Dinesh Kumar
15
Reputation points
answered 2024-11-01T13:08:12.7533333+00:00
Pinaki Ghatak
4,535
Reputation points Microsoft Employee
2 answers
Error: 11402 -The value of the property '' is invalid: 'The remote name could not be resolved.
Hello Team, I am using the private endpoints in Azure Data Factory and trying to connect On premises database and getting the error : The value of the property '' is invalid: 'The remote name could not be resolved: 'ServerName''. The remote name could…
asked 2024-05-13T14:13:42.9766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 13%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Rohit Kulkarni
691
Reputation points
commented 2024-11-01T12:15:15.51+00:00
LiborOndruek-3549
21
Reputation points
1 answer
Como recuperar minha conta no Power BI Service
Como recuperar minha conta no Power BI Service sem precisar colocar senha no Authenticator? pois tenho arquivos que preciso recuperar. Fiz uma conta corporativa no Microsoft Fabric free: : kessiafarias@000000, e criei uma conta no Power BI service com…
asked 2024-10-31T12:34:48.5+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 13%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELS%3C/text%3E%3C/svg%3E)
lucas silva
0
Reputation points
answered 2024-11-01T09:18:46.3866667+00:00
Chandra Boorla
2,525
Reputation points Microsoft Vendor
2 answers
How to save complete web activity output to ADLS Gen2 in JSON format? Azure Synapse Studio
I want to save the output of my Web activity (which hits an Api and then fetches data from that Api in Json format) into blob storage account container. Also, my web activity is within a foreach - so multiple Api links are being hit and multiple files…
asked 2024-10-24T05:42:30.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 20%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Mitashi Malkani
0
Reputation points
commented 2024-11-01T07:01:02.5866667+00:00
Chandra Boorla
2,525
Reputation points Microsoft Vendor
1 answer
copy tables from sql server on-prem to .parquet format in delta lake
Currently our copy process for 10 tables is running in pipeline in ADF, our source is ON-PREM SQL Server and target is in BLOB Storage GEN2. it copies data(tables) in parquet file format in HOT storage Account --- StorageV2 (general purpose v2) but I…
asked 2024-10-29T19:03:53.5233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 10%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZH%3C/text%3E%3C/svg%3E)
Zaeemul Haque
25
Reputation points
commented 2024-11-01T06:55:45.1833333+00:00
Chandra Boorla
2,525
Reputation points Microsoft Vendor
0 answers
SQLFailedToConnect: Pipeline copy activity fails when set to trigger but runs fine in debug mode.
I am trying to copy data from onprem SQl db to Azure gen2 storage account using ADF. I have configured SHIR which is connected to ADF. I have created a linked server to the on-prem sql db which connects successfully. I have created a simple copy activity…
asked 2024-10-30T03:43:39.9466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 24%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESL%3C/text%3E%3C/svg%3E)
Shobna Lata
20
Reputation points
commented 2024-11-01T05:34:22.2433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 4%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
ShaktiSingh-MSFT
15,336
Reputation points