Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
1 answer
Assert failure rows: Output to sink Option Reverts After Publish
I'm encountering a UI issue in Azure Data Factory while configuring a Data Flow activity inside a pipeline. When I uncheck the "Assert failure rows: Output to sink" option and publish the pipeline, the setting appears to be saved. However,…
Azure Data Factory
asked 2025-07-08T13:22:00.2866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 57.99999999999999%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYH%3C/text%3E%3C/svg%3E)
Yahya Hussaini
0
Reputation points
commented 2025-07-09T18:34:12.0866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 50%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
Venkat Reddy Navari
3,630
Reputation points Microsoft External Staff
Moderator
1 answer
One of the answers was accepted by the question author.
My tumbling window trigger in Azure data Factory is not running
My tumbling window trigger is not running. It does not have any dependencies. Please see below screen print, even though the scheduled start time has passed trigger status is "Started" it was supposed to start running at 15:00 EST 07/07.…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-08T05:41:56.42+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 99%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDG%3C/text%3E%3C/svg%3E)
Dattatreya Gandham
20
Reputation points
commented 2025-07-09T09:26:25.0233333+00:00
Amira Bedhiafi
34,651
Reputation points
Volunteer Moderator
1 answer
How to resolve Budget Export Error from FinOPS HUB Deployment ADF 'error':{'code':'NotFound','message':'The requested export with export name: finops-hub-monthly-costdetails could not be found.'}}
Hi Team, We have integrated FinOps Hub Toolkit tools into our subscription to leverage cost analytics and insights from our budget spending. However, we are seeing a problem with the FinOPS hub deployed ADF exporting budget data into Storage accounts. We…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-07T07:17:57.47+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 72%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMV%3C/text%3E%3C/svg%3E)
Manisankar Vr
0
Reputation points
commented 2025-07-09T08:00:46.0666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
25,175
Reputation points Microsoft External Staff
Moderator
1 answer
Azure Free trial
I started with an Azure free trial, but I didn't understand how to work with ADF at first, so I decided to save my credits for later. However, Microsoft has now declared that my one-month trial period is over, and they are asking me to switch to a paid…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-08T10:13:20.28+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 19%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGR%3C/text%3E%3C/svg%3E)
Girija Raut
0
Reputation points
commented 2025-07-09T07:45:19.35+00:00
Smaran Thoomu
25,175
Reputation points Microsoft External Staff
Moderator
2 answers
Continuous vs. Phase-Specific Pipelines for Retry and Reconciliation (ADF + Databricks)
In our healthcare data migration project, we are using a modular, control-table-driven architecture for One-Time, Catch-Up, and Real-Time phases. The pipeline processes are divided across Extract, Transform, and Load stages — primarily orchestrated via…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-07T15:29:47.1333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 56.00000000000001%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJC%3C/text%3E%3C/svg%3E)
Janice Chi
320
Reputation points
commented 2025-07-09T07:44:22.4033333+00:00
Smaran Thoomu
25,175
Reputation points Microsoft External Staff
Moderator
2 answers
One of the answers was accepted by the question author.
Design Error Handling In ADF
Hello, Please help me on how to design Error handling in ADF. I am moving files from Bronze ADLS Gen2 to Silver Azure SQL. So If data in the file contain error more that 20% of total records than reject entire file and move file to reject bucket, if less…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 79%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
1 answer
Unable to undo or commit changes in Fabric Git, also when trying to access Notebooks getting 404 error.
Everything was working fine until yesterday. We normally pause the fabric capacity at night and resume in the morning. When we resumed today morning, there were multiple file changes detected in Fabric Source Control. We tried to access some…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-07T18:40:58.1833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 47%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAD%3C/text%3E%3C/svg%3E)
Azure Dev
0
Reputation points
answered 2025-07-08T05:11:35.8433333+00:00
Smaran Thoomu
25,175
Reputation points Microsoft External Staff
Moderator
2 answers
One of the answers was accepted by the question author.
ADF Pipeline error: Request content size is too large
I have a pretty basic ADF pipeline that checks a subdirectory for a group of flat files and based on the name of the file, it will load data into a SQL staging table. The ForEach activity has 7 If Condition activities which will run based on the…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2021-05-25T15:08:39.773+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 35%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Matt Michalowski
31
Reputation points
commented 2025-07-07T20:26:55.7566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 23%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJP%3C/text%3E%3C/svg%3E)
Jim Pucci
0
Reputation points
3 answers
One of the answers was accepted by the question author.
ADF copy activity loading 2 different files into same variant table
so when am loading data from azure blob to snowflake variable table, its copying the content of 2 files into one variant table i.e. students 2 files in blob - sql.students, sql.subjects 2 variant table in snowflake - stage.sql_students_variant &…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-03T11:01:27.78+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(96, 77%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
65
Reputation points
accepted 2025-07-07T08:10:31.3166667+00:00
azure_learner
65
Reputation points
1 answer
recommended SHIR configuration to ingest 80TB from DB2 Snapshot to ADLS Gen2 via ADF within 3 days?
We are performing a large-scale one-time data migration project. Our source is an IBM DB2 snapshot (on-prem), and the target is Azure Data Lake Storage Gen2. The entire data volume is ~80TB, distributed across 800 tables. We are using Azure Data Factory…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-06-28T15:53:10.39+00:00
Janice Chi
320
Reputation points
commented 2025-07-07T06:32:53.5533333+00:00
Smaran Thoomu
25,175
Reputation points Microsoft External Staff
Moderator
1 answer
One of the answers was accepted by the question author.
ADF Copy Activity Generating negative acknowledgements on SFTP Server while execution.
The Copy Activity in Azure Data Factory (ADF) is configured to copy files from SFTP Server. This copy activity is triggering a negative acknowledgement (NACK) first on the SFTP server while downloading the files and after a few seconds file copy is…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-04T05:22:21.0766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 76%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Shashank Thogaru
20
Reputation points
accepted 2025-07-07T06:18:18.42+00:00
Shashank Thogaru
20
Reputation points
1 answer
How to Report File Metadata for 2000+ Files Uploaded by Multiple Users to Azure Blob Storage
Hi Team, I would like to know following questions. Please do needful. 1.How to Report File Metadata for 2000+ Files Uploaded by Multiple Users to Azure Blob Storage 2.How to connect Hadoop hive to databricks Regards, Prasad, 5017654887.
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-03T02:41:39.2233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(32, 57.99999999999999%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
Prasad Sandu
0
Reputation points
commented 2025-07-04T13:30:11.3066667+00:00
Venkat Reddy Navari
3,630
Reputation points Microsoft External Staff
Moderator
1 answer
How does Azure Data Factory handle data movement ?
How does Azure Data Factory handle data movement and transformation across hybrid environments, and what are the best practices for optimizing pipeline performance and cost efficiency
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-02T22:05:28.1966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 59%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHS%3C/text%3E%3C/svg%3E)
Himanshu Sinha
11
Reputation points
commented 2025-07-04T13:21:52.3633333+00:00
Smaran Thoomu
25,175
Reputation points Microsoft External Staff
Moderator
3 answers
How to iterate through Nested folders and read file after the file in adf
Hi, How can I iterate through folder and inside subfolder by folder and process one file after another without using regex in azure data factory
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-02T13:02:06.2633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 45%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERJ%3C/text%3E%3C/svg%3E)
Radha Jahnavi Pusarapu
0
Reputation points
commented 2025-07-04T12:00:48.7433333+00:00
Venkat Reddy Navari
3,630
Reputation points Microsoft External Staff
Moderator
1 answer
Seeking solution for intermittent "SqlFailedToConnect" errors when writing to Azure SQL Managed Instance from Azure Data Factory
I have multiple ADF pipelines that copy data from a variety of sources into our Azure SQL Managed Instance. All of these jobs, regardless of source, fail intermittently with a Sink error such…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-02T18:12:10.1166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(80, 47%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERL%3C/text%3E%3C/svg%3E)
Rob Lamb
0
Reputation points
commented 2025-07-04T11:51:21.78+00:00
Venkat Reddy Navari
3,630
Reputation points Microsoft External Staff
Moderator
3 answers
Error while running data factory pipeline in fabric
Hi Team, I am getting below error while executing data pipeline in fabric. Can you please help me on this how to resolve it. Failed to get User Auth access token. The error message is: Failed to get User Auth access token. The error message is:…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-03T14:28:23.6+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 9%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESG%3C/text%3E%3C/svg%3E)
Sharad Gawade
20
Reputation points
answered 2025-07-03T22:48:22.0466667+00:00
Smaran Thoomu
25,175
Reputation points Microsoft External Staff
Moderator
2 answers
Error: 11402 -The value of the property '' is invalid: 'The remote name could not be resolved:
I am setting up a Linked service for ODATA in ADF, for IntegrationRuntime I am selecting "AutoResolveIntegrationRuntime" but connection to ODATA resource is failing: The value of the property '' is invalid: 'The remote name could not be…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2024-05-08T05:48:44.8233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 3%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBA%3C/text%3E%3C/svg%3E)
Bhat, Asif
20
Reputation points
answered 2025-07-03T18:14:19.3266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 45%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBC%3C/text%3E%3C/svg%3E)
Bharath Chandran
0
Reputation points
1 answer
data extraction issues since upgrading to Odata 4.0 in ADF
Our data extraction process, using the Odata connector in ADF has shown problems since our client upgraded their service to Odata 4.0. The issues consist of not succesfully extracting an endpoint. Our client shows that their network gets overloaded even…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-01T09:34:47.71+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 21%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETB%3C/text%3E%3C/svg%3E)
Thijs Bakker
0
Reputation points
commented 2025-07-03T11:50:11.5366667+00:00
Venkat Reddy Navari
3,630
Reputation points Microsoft External Staff
Moderator
1 answer
One of the answers was accepted by the question author.
dataset showing warning even after updating linked service version from 1 to 2
ADF dataset showing warning even after updating linked service version from 1 to 2 also tried to refresh the page tried to reopen the ADF ports but still the same warning also the test connection is successful in linked service after updating…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-06-05T07:38:46.85+00:00
azure_learner
65
Reputation points
accepted 2025-07-03T10:50:52.7433333+00:00
azure_learner
65
Reputation points
1 answer
ServiceNow Incident table extraction with V2 Connectors
Using the new ServiceNowV2 connectors in Azure Data Factory, we are trying to extract the incident table. When setting up a copy activity, we receive the error message below. I have tried to limit the number of records by providing a filter to small…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
asked 2025-07-01T06:53:26.65+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 13%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJS%3C/text%3E%3C/svg%3E)
Joshoua Simon
0
Reputation points
commented 2025-07-03T10:16:27.89+00:00
Venkat Reddy Navari
3,630
Reputation points Microsoft External Staff
Moderator