Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,443 questions
1 answer
ADF Copy Activity slow after downgrade IR to 5.45.8999.1
Hi, We recently downgraded IR to 5.45.8999.1 to get around the issue related to 5.48.9106.2 that was causing 'NpgsqlTypes.NpgsqlInterval' errors. While we no longer having the 'NpgsqlTypes.NpgsqlInterval' errors with 5.45.8999.1, our copy activity…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(160, 40%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EC8%3C/text%3E%3C/svg%3E)
0 answers
I need to access the service principal key for an sp created and managed by MS Entra
We are trying to connect to Snowflake using a Data Factory because we want to copy data from Snowflake to an Azure SQL DB. We have created an azure blob storage account and and a linked service. We have created SF integration and stage. During the allow…
asked 2025-04-24T19:48:47.6833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 12%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESP%3C/text%3E%3C/svg%3E)
Scott Pelton (PowerBI)
0
Reputation points
commented 2025-04-25T23:07:23.4666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
11,745
Reputation points Microsoft External Staff
0 answers
Microsoft Fabric Capacity Paused
We have paused our Fabric capacity; will we still incur fabric monthly capacity cost?
asked 2025-04-25T17:50:52.0533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 51%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Mir Majeed
0
Reputation points
commented 2025-04-25T18:06:05.7066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 99%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJN%3C/text%3E%3C/svg%3E)
J N S S Kasyap
1,620
Reputation points Microsoft External Staff
0 answers
I am trying to pull Sharepoint list into SDP using Pipeline but getting error
I am trying to pull Sharepoint list into SDP using Pipeline but getting error : List Name : 333 Backlog Tracker Requesting response from path 333 Backlog Tracker and query $top=10 failed with status code NotFound and message <?xml…
asked 2025-04-25T16:02:48.5566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 57.00000000000001%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELA%3C/text%3E%3C/svg%3E)
Lohot, Abhishek
0
Reputation points
edited a comment 2025-04-25T16:48:06.6533333+00:00
J N S S Kasyap
1,620
Reputation points Microsoft External Staff
1 answer
Issue with Large Numbers Changing in ADF Data Flow Source (CSV vs Excel)
Hi Microsoft Support, I'm experiencing a data accuracy issue when working with large numbers in Azure Data Factory (ADF) Data Flows. For example: In my file, the number is: 123456789012345000 But when I open the same value in DATAFLOW/, it becomes:…
asked 2025-04-23T17:26:13.67+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 55.00000000000001%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EIM%3C/text%3E%3C/svg%3E)
Ichhwak Malhare
0
Reputation points
commented 2025-04-25T15:52:07.0266667+00:00
J N S S Kasyap
1,620
Reputation points Microsoft External Staff
1 answer
I want to document ADF pipelines runs / triggers
I want to document ADF pipelines runs / triggers in excel or PowerBI (for report purpose) and note what the data is being used for and source and target data as well as trigger times.. how often and when does it run. Is there any way that I can get these…
asked 2025-04-22T20:40:49.32+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 77%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDS%3C/text%3E%3C/svg%3E)
Duyen Su
0
Reputation points
commented 2025-04-25T15:50:13.4933333+00:00
J N S S Kasyap
1,620
Reputation points Microsoft External Staff
1 answer
One of the answers was accepted by the question author.
Not able to write the data in cosmos DB
Hi , I am facing the issue in loading the data in cosmos db. I am able to test the connection and preview the data in ADF but not able to write the data and SPN has contributor role to cosmosdb through which I am trying to load. Not getting the error…
asked 2025-04-23T21:35:28.6233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 55.00000000000001%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVP%3C/text%3E%3C/svg%3E)
Venkateswarlu Pottapalli
20
Reputation points Microsoft External Staff
accepted 2025-04-25T15:28:09.5366667+00:00
Venkateswarlu Pottapalli
20
Reputation points Microsoft External Staff
1 answer
ADF connection to DB2 using SSL certificate
Hi, We have a azure data factory and we need to connect to on-prem DB2 (version 11.5.9) using SSL certificate through Self Hosted Integration Runtime. Our selfhosted integration runtime also in azure. I have created the linked service for db2 and…
asked 2025-04-25T12:26:35.89+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 37%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
Mani
25
Reputation points
answered 2025-04-25T14:56:33.68+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 50%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
Venkat Reddy Navari
1,190
Reputation points Microsoft External Staff
1 answer
AZF Dataflow error connection CosmosDB
I have created a linked service and the testconnection is succeeded But when I use this linked service in a dataflow and test the connection I get the following error: Error codeDFExecutorUserError DetailsStatus code 400,…
asked 2025-04-22T11:34:58.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(96, 28.000000000000004%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EED%3C/text%3E%3C/svg%3E)
Erica de Graaf-Borkent
0
Reputation points
commented 2025-04-25T13:02:15.2+00:00
Erica de Graaf-Borkent
0
Reputation points
2 answers
One of the answers was accepted by the question author.
Azure Data Factory (ADF) DRP - self hosted integration runtime question
Is it possible to assume that the following DRP setup will work: ADF is created and metadata are available in backup region SHIR is created in a primary ADF with 2 VMs in primary region and 2 VMs in backup region (2 VMs in backup region will be…
asked 2025-04-25T09:39:31.29+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 33%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMP%3C/text%3E%3C/svg%3E)
Marcin Parzy
5
Reputation points
commented 2025-04-25T12:07:10.52+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
22,425
Reputation points Microsoft External Staff
1 answer
One of the answers was accepted by the question author.
Moving data from Snowflake to Azure SQL DB using Copy Activity
Hi All, I am using COPY activity to move data from Snowflake to Azure SQL DB. First of all, this data movement is not allowed without intermediate staging of the data in ADLS. I would love to avoid this intermediate step. If this is how it will work then…
asked 2024-09-04T14:26:06.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 27%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
PK
25
Reputation points
commented 2025-04-25T10:50:28.1033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 68%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Ameya Sabnis
0
Reputation points
1 answer
How to extract incremental data from delta files without having any filter condition in data in dataflow adf
Hi Team, Could you please help us to find the solution for below requirement. we have delta files folder it have 40million records, There is no last modified date column to extract latest data, in dataflow how to extract the latest delta file data…
asked 2025-04-18T16:18:03.0533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 22%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
SaiSekhar, MahasivaRavi (Philadelphia)
140
Reputation points
answered 2025-04-25T10:00:58.4233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 30%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Aswin
392
Reputation points
1 answer
One of the answers was accepted by the question author.
Migration from Azure Data Factory to Microsoft Fabric – Compatibility of Data Flows and Activities
We have built our current data integration solution using Azure Data Factory (ADF), utilizing Copy Activity for data movement and Mapping Data Flows for transformations. Now, we are considering migrating to Microsoft Fabric. Given that Fabric uses…
asked 2025-04-25T05:33:08.61+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 16%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAP%3C/text%3E%3C/svg%3E)
Akshay Patel
130
Reputation points
commented 2025-04-25T08:04:12.8633333+00:00
Akshay Patel
130
Reputation points
2 answers
One of the answers was accepted by the question author.
Capture an Error in a For Each loop in ADF
Hi, I have a pipeline that includes a ForEach loop. On occasion the pipeline fails in the for each loop, but the only error message i get is "Activity failed because an inner activity failed". How do i get to, or capture the actual…
asked 2025-04-23T09:25:07.15+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 98%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGP%3C/text%3E%3C/svg%3E)
Gareth Parry
0
Reputation points
accepted 2025-04-25T07:43:25.63+00:00
Gareth Parry
0
Reputation points
1 answer
Unable to create Airflow Workflow orchestration manager in Azure Data Factory - ReferenceError: Cannot access 'Z' before initialization
I am getting this error when I try to create a new Apache Airflow Workflow orchestration manager in Azure Data Factory. Whatever values I choose for the configuration, the error is the same. I have tried using different browsers and new incognito…
asked 2025-04-23T14:10:12.3766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 16%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
Divyajey
0
Reputation points
commented 2025-04-25T07:34:34.1933333+00:00
Smaran Thoomu
22,425
Reputation points Microsoft External Staff
0 answers
Use Azure ADF CDC with source DB in private network
I believe that my use case must be quite common as I want to use ADF CDC feature on an SQL server that is deployed in Azure VM, in a virtual network with only a private IP. It seems like Azure CDC doesn't work with self-hosted int runtime, so I could…
asked 2025-04-25T04:44:02.7533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 19%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPH%3C/text%3E%3C/svg%3E)
Petr Hecko
0
Reputation points
commented 2025-04-25T07:09:44.31+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
6,200
Reputation points Microsoft External Staff
1 answer
Azure Fabric - Airflow feature in vnet internal mode
Hello, we have a requirement to deploy Apache Airflow on Azure and considering couple of options as part of this. Airflow on AKS Airflow on WebApp managed Airflow on fabric or ADF Our requirement is to run workflows (tasks as part of DAG) on…
asked 2025-03-24T05:35:09.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 55.00000000000001%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMT%3C/text%3E%3C/svg%3E)
Mohammed Thahif BK
346
Reputation points Microsoft Employee
commented 2025-04-24T21:57:01.22+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 88%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAL%3C/text%3E%3C/svg%3E)
Assaf L
287
Reputation points
2 answers
Azure Synapse Pipeline Trigger Won't Fire
I created a storage event trigger in Azure Synapse Analytics. In the "Data preview" pane, I am able to see that Azure detects files which match the trigger criteria. However, when I publish the trigger and upload a matching file, the trigger…
asked 2025-04-21T15:05:19.53+00:00
Scott McQueen
20
Reputation points
commented 2025-04-24T19:30:54.2+00:00
Chandra Boorla
11,745
Reputation points Microsoft External Staff
1 answer
One of the answers was accepted by the question author.
Sql server to Parquet file date missmatch using ADF copy data activity
Hi , I am doing copy data activity in adf Source - Sql server Sink - parquet file When doing copy Date field is changing From 0001-01-01 is changing to 0001-01-03 in parquet. It is incrementing the day by 2 in parquet file. …
asked 2022-06-13T07:02:07.37+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(41.6, 83%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGC%3C/text%3E%3C/svg%3E)
gajanan choure
56
Reputation points
commented 2025-04-24T17:44:34.16+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 89%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKS%3C/text%3E%3C/svg%3E)
Katarzyna Szmydki
0
Reputation points
1 answer
406 error while connecting to API from Rest source of Copy data activity but works from Web activity and Postman.
I am facing an issue where the API works from Web activity of Adf pipeline but doesn't work from Rest source of Copy data activity. It was working initially and stopped. Same API works from web activity and postman. My API works from Web activity but…
asked 2025-04-15T15:24:42.81+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 88%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMS%3C/text%3E%3C/svg%3E)
Mishra, Swastik
0
Reputation points
commented 2025-04-24T11:49:43.4633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 57.00000000000001%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDR%3C/text%3E%3C/svg%3E)
Dileep Raj Narayan Thumula
0
Reputation points Microsoft External Staff