Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,820 questions
1 answer
One of the answers was accepted by the question author.
Incorrect parameter mapping in a dataflow
Hi, I have faced the problem with using definition of parameter. In a DataFlow in source stage, I use a storage procedure with parameters as an input. If I directly put a parameter name in a value, it looks OK (because in expression builder he recognises…
asked 2024-11-05T10:02:30.5466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 13%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGE%3C/text%3E%3C/svg%3E)
Gużewski, Jacek
115
Reputation points
accepted 2024-11-06T05:44:26.6333333+00:00
Gużewski, Jacek
115
Reputation points
0 answers
The specified network name is no longer available - error code - 22852
Hello Team, Today I faced an error, when i am trying to do test connection. Please resolved as soon as possible. I need to resolve please help me to resolve. 
asked 2024-09-20T12:10:29.63+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 55.00000000000001%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBS%3C/text%3E%3C/svg%3E)
Balwant Singh Chouhan
50
Reputation points
commented 2024-11-06T05:01:48.3366667+00:00
Balwant Singh Chouhan
50
Reputation points
1 answer
Why is the 7days timeout not reflected as setting when calling Azure Function inside Azure Data Factory?
I have called to trigger an Azure Function inside the Azure Data Factory. I have set the timeout to be 7 days, the minimum timeout is to be 10min. However, it turns out to have a timeout after 230 seconds. I saw the documentation mentioning the 230…
asked 2024-11-06T02:53:59+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(249.60000000000002, 50%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECK%3C/text%3E%3C/svg%3E)
Choi, (Kerwin)
0
Reputation points
answered 2024-11-06T04:42:49.97+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 40%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKA%3C/text%3E%3C/svg%3E)
Khadeer Ali
575
Reputation points Microsoft Vendor
1 answer
Automating Azure Data Factory Sync with Git Branch After Python SDK Modifications
Hello, I've modified my datafactory through python SDK and added some trigger. Now my publish branch is out of sync with the main branch. I found the theoretical solution here:…
asked 2024-11-05T14:41:08.2666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 80%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Moritz Moeller
0
Reputation points
commented 2024-11-06T01:01:43.1766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
2,605
Reputation points Microsoft Vendor
1 answer
Could not load resource. Please ensure no mistakes in the JSON and that referenced resources exist.
Dear experts, I successfully created a data flow 2 weeks ago, and until then, I was able to edit it.. However, this Monday morning, I noticed that I can no longer edit it. When I try to open the data flow, I encounter the following message: “Could not…
asked 2024-11-04T09:26:28.8233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 39%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EF%3C/text%3E%3C/svg%3E)
Fouad
0
Reputation points
answered 2024-11-05T23:40:54.4+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
950
Reputation points Microsoft Vendor
2 answers
Salesforce V2 Link Service doens't work. error message: a task was canceled Source=mscorlib
I am migrating the ADF salesforce connector to the new V2 connector and now testing it. When testing the connection for the link service, it is good. However, it failed in the copy activity. with message: Failure happened on 'Source' side.…
asked 2024-05-16T13:35:58.54+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 81%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EC5%3C/text%3E%3C/svg%3E)
ChoiLiamhehimhis-5365
25
Reputation points
answered 2024-11-05T21:34:12.07+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 46%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJS%3C/text%3E%3C/svg%3E)
Jeff S
0
Reputation points
1 answer
How to perform copy activity with new ServiceNow connector in ADF. I am getting below error while perform copy activity
I am changing old legacy ServiceNow connector to new ServiceNow connector, In new ServiceNow connector we don't have any option to select table or query. for that I have selected table name in datasets. create one simple copy activity and try to copy…
asked 2024-11-04T13:32:27.0166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 83%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHT%3C/text%3E%3C/svg%3E)
Hari Tiwari, Tieto
0
Reputation points
commented 2024-11-05T19:45:56.3466667+00:00
Ganesh Gurram
950
Reputation points Microsoft Vendor
1 answer
Is integration runtime shared across dataflow runs?
Hi everyone, We have started to use Azure Data Factory recently and we are trying to find the best way to utilize Integration Runtimes. We are using Azure Public IR without managed VN, but we are defining region on our own for performance purposes…
asked 2024-11-05T11:02:07.35+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(124.80000000000001, 37%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPM%3C/text%3E%3C/svg%3E)
Pawel Minkina
0
Reputation points
answered 2024-11-05T18:15:54.5266667+00:00
Amira Bedhiafi
26,101
Reputation points
2 answers
Generate date rows in ADF data flow
I'm just starting to learn Azure Data Factory as we have a need to migrate our Alteryx workflows. I've created a data flow within data factory which summarizes data to return a max and min date column per recruiter, as per below. What I now need to do…
asked 2024-11-04T14:00:20.39+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 43%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERH%3C/text%3E%3C/svg%3E)
Richard Hawkins (AXIS)
0
Reputation points
commented 2024-11-05T18:07:53.9433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
16,725
Reputation points Microsoft Vendor
1 answer
How to check text in row 1 of Excel file for about 10 column headers?
I'm using Azure Data Factory (ADF) on the portal. I use the latest Chrome browser to get to ADF. I'm not an admin and have no admin rights. I have an Excel file I want to read. I'm still fairly new to ADF. I'm guessing this could be done in a data flow.…
asked 2024-11-05T12:36:13.51+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 77%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECR%3C/text%3E%3C/svg%3E)
Chuck Roberts
110
Reputation points
answered 2024-11-05T17:37:20.27+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
11,125
Reputation points Microsoft Vendor
0 answers
In a private subscription I am unable to connect a power query activity from ADF to an Azure SQL DB
I had a powerbi report with semantic model that was exceeding our P2 premium node on refresh. I am looking to migrate the power query steps to ADF and sink it to a SQL Data Mart instead for consumption by the report. Since the Power Query activity only…
asked 2024-11-01T22:10:05.71+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 77%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJB%3C/text%3E%3C/svg%3E)
Jonathan Burbano
0
Reputation points
commented 2024-11-05T17:27:15.4133333+00:00
Ganesh Gurram
950
Reputation points Microsoft Vendor
1 answer
Request Timeout on ADF Web Activity to Synapse
Hello everyone! I have a set of pipelines in Azure Data Factory (ADF) that trigger Synapse via a Web activity POST request. This setup worked until today around 11 AM CET, but now it’s throwing the following error: Error calling the endpoint. Response…
asked 2024-11-05T15:51:24.8766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 34%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEM%3C/text%3E%3C/svg%3E)
Erik Martins Tonon
0
Reputation points
commented 2024-11-05T17:05:53.6433333+00:00
AnnuKumari-MSFT
33,316
Reputation points Microsoft Employee
2 answers
DataFactory DataFlow Activity ManageVnet dataflow cluster is not started
Hi All, we are unable to start the azure data factory data flow cluster . we are using Mange Vent IR in itally north region when we start the cluster in Debug Mode we get message-The startup of the cluster is taking longer than expected , we recommend…
asked 2024-10-24T12:33:08.52+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 20%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMV%3C/text%3E%3C/svg%3E)
manish verma
461
Reputation points
commented 2024-11-05T16:43:41.55+00:00
phemanth
11,125
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
Where to find Web task HTTP Response headers in data factory?
I am calling an API via a web task in Data factory. The api returns critical information in the response headers, which i can see if i call the API via postman, but does not seem to be returned via the web task in data factory. How can i see the…
asked 2022-07-06T15:08:11.12+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 79%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EN%3C/text%3E%3C/svg%3E)
na
141
Reputation points
commented 2024-11-05T16:01:15.74+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 51%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
Ritesh Sharma
0
Reputation points
3 answers
Missing Records in Azure Log Analytics
Hi All, Logs of activities, pipelines, and triggers are gathered within the Synapse environment for monitoring purposes. It has been observed that some pipelines ran successfully, as seen in the Monitor section of Synapse, but there are missing records…
asked 2024-10-21T13:55:53.4933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 54%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJP%3C/text%3E%3C/svg%3E)
Jakub Pytlik
0
Reputation points
commented 2024-11-05T15:42:27.4066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 20%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPR%3C/text%3E%3C/svg%3E)
Pranay Reddy Madireddy
440
Reputation points Microsoft Vendor
0 answers
SAP CDC connector in ADF failing
Hi All, We are trying to extract data from a SAP table to Synapse thru SAP CDC connector in a ADF pipeline. For one of the tables we are getting error as below. Please let us know if it is an issue on Microsoft side. *Job failed due to reason: at Source…
asked 2024-11-04T09:48:17.13+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 33%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENS%3C/text%3E%3C/svg%3E)
Nair, Srikanth
0
Reputation points
commented 2024-11-05T14:13:02.5366667+00:00
Smaran Thoomu
16,725
Reputation points Microsoft Vendor
1 answer
Improving Performance of Copy Activity from Cosmos DB to Cosmos DB Resource
Hello, I am currently trying to using the copy activity in Azure Data Factory to migrate data from one cosmos DB container to another cosmos DB container (both NoSQL). Both the source and sink have 50k throughput. I am trying to move 1.7 TB of data and…
asked 2024-10-30T15:04:38.0166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 77%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERD%3C/text%3E%3C/svg%3E)
Ram Damodaran
1
Reputation point Microsoft Employee
commented 2024-11-05T14:06:20.57+00:00
Smaran Thoomu
16,725
Reputation points Microsoft Vendor
1 answer
Can't sink from Data Factory to Blob Storage after updating storage account public access to 'Enabled from selected virtual networks and IP addresses'.
I have an Azure Data Factory pipeline that is no longer able to access my storage account after I have updated the public network configuration to 'Enabled from selected virtual networks and IP addresses' in my storage account. I have followed the steps…
asked 2024-11-01T22:25:53.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 17%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJF%3C/text%3E%3C/svg%3E)
Joe Fried (TATA CONSULTANCY SERVICES LTD)
0
Reputation points Microsoft Vendor
commented 2024-11-05T14:05:28.22+00:00
Smaran Thoomu
16,725
Reputation points Microsoft Vendor
2 answers
How to assign Variable Value to Parameter
Hi, we need to create a "batch_no" in the format YYYYMMDDHHMMSS within our pipeline as a human-readable job identifier , since system variables like pipeline run ID or trigger ID aren’t easily readable. The "batch_no" should remain…
asked 2024-11-02T02:51:24.05+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 43%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAZ%3C/text%3E%3C/svg%3E)
Amy Zhang
25
Reputation points
commented 2024-11-05T14:01:24.23+00:00
Smaran Thoomu
16,725
Reputation points Microsoft Vendor
1 answer
How to give Sharepoint folder granular access to a Service Principal ?
Our team is working to get data from Sharepoint to Azure blob storage. We decided to use ADF as logic apps has size limitations. The doc referred is Microsoft Learn To complete this solution, we need to give the sharepoint site full access to the…
asked 2024-10-24T09:06:37.0233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(51.2, 62%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKS%3C/text%3E%3C/svg%3E)
Khadraoui, Said
0
Reputation points
commented 2024-11-05T13:50:58.04+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 33%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
Alvaro Avila
80
Reputation points