Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
1 answer
Breaking Undocumented Change: Oracle Connector v2 for queries ending in a semi-colon | Works with v1 linked service
Breaking Undocumented Change: Oracle Connector v2 on ADF: When using "Query" option in Copy Activity, and if that query ends in a semicolon(;), it breaks during run time on v2 linked service, while this works fine on v1 linked service. …
Azure Data Factory
asked 2025-06-26T15:53:49.17+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(48, 21%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVT%3C/text%3E%3C/svg%3E)
VIjaya Teja Rebbapragada
0
Reputation points
commented 2025-06-27T19:20:41.0933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
14,510
Reputation points Microsoft External Staff
Moderator
1 answer
Help with Metadata-Driven ADF Pipeline & Column Mapping Case Sensitivity
Hi everyone, I'm building a metadata-driven Azure Data Factory (ADF) pipeline to dynamically ingest Excel sales files from multiple source systems (like SAP, Senior, etc.) into raw SQL tables. Pipeline Overview I have created a single pipeline…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-25T17:41:34.31+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 57.00000000000001%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYS%3C/text%3E%3C/svg%3E)
YERNAIDU SIRAPARAPU
45
Reputation points
commented 2025-06-27T19:18:15.3566667+00:00
Chandra Boorla
14,510
Reputation points Microsoft External Staff
Moderator
1 answer
OData Queries aren't working when using the Lookup action in Azure Data Factory
I'm trying to list all the files in a specific folder on a SharePoint site that have value in a column called "FakeModified" that is less than 2025-01-01. I am using the Lookup activity in Azure Data Factory with a SharePointOnlineListResource…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-25T16:41:04.47+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 20%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECW%3C/text%3E%3C/svg%3E)
Calvin Worst
0
Reputation points
commented 2025-06-27T16:59:56.2266667+00:00
Chandra Boorla
14,510
Reputation points Microsoft External Staff
Moderator
1 answer
Azure Data Factory: No 'Name' Key Appearing Under Additional Headers For REST APIs Data Copy Action
Hello, I'm trying to bring in data to SQL Server using a REST API. I've successfully completed the first stage, where I needed to call the Login endpoint to fetch the apiToken in order to request all other endpoints using the bearer authentication. I'm…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-24T17:19:53.7366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 54%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
sk900
0
Reputation points
commented 2025-06-27T16:55:44.53+00:00
Chandra Boorla
14,510
Reputation points Microsoft External Staff
Moderator
1 answer
Failure happened on 'Source' side. Copy activity Error in Fabric
Failure happened on 'Source' side. ErrorCode=UserErrorWriteFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The file operation is failed, upload file failed at path:…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-27T00:32:51.13+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(313.6, 93%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Sri Ram Katta
0
Reputation points
commented 2025-06-27T13:17:34.48+00:00
Sri Ram Katta
0
Reputation points
1 answer
Conditional Split in ADF Data Flow issue - no rows in sink
I have a Conditional Split transformation that works fine except one sink results empty although there are source rows matching the split condition. In my scenario I have - say - 4 conditions, two of which are supposed to end up in the same table sink.…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-26T14:28:01.3266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 50%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EUP%3C/text%3E%3C/svg%3E)
Ulrike Pick
0
Reputation points
edited a comment 2025-06-27T11:08:54.52+00:00
Ulrike Pick
0
Reputation points
1 answer
What are the recommended steps for setting up a Service Principal in Azure for enterprise use, and how should the IT team manage its lifecycle securely?
We’re working on setting up Service Principals (App Registrations) in Azure to support automated data workflows (e.g., Azure Data Factory accessing Azure SQL, ADLS, etc.). The goal is to enable secure, non-interactive authentication while following…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-27T09:33:08.0033333+00:00
YERNAIDU SIRAPARAPU
45
Reputation points
answered 2025-06-27T10:23:23.5466667+00:00
Nandan Hegde
36,146
Reputation points MVP
Volunteer Moderator
1 answer
Why the size of Data Read in ADF copy activity is much bigger than the table size from BigQuery?
I am currently using Azure Data Factory (ADF) to copy data from a BigQuery table to Azure Blob Storage. However, I noticed a discrepancy where the "Data Read" metric in the copy activity is significantly larger than the actual size of the…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-25T02:30:45.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 34%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESW%3C/text%3E%3C/svg%3E)
Sheldon Wong
0
Reputation points
commented 2025-06-27T09:14:25.8233333+00:00
Chandra Boorla
14,510
Reputation points Microsoft External Staff
Moderator
1 answer
Compatibility and Support for Amazon Corretto in Azure Data Factory
Hello Azure Support Team, We are exploring the possibility of replacing Oracle Java with Amazon Corretto (OpenJDK distribution) in our Azure Data Factory (ADF) environment. Before proceeding, we would like to clarify the following: Compatibility: Is…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-23T13:27:38.9433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(32, 88%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENP%3C/text%3E%3C/svg%3E)
Naveen Prasanth
0
Reputation points
commented 2025-06-27T08:59:11.3266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 50%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
Venkat Reddy Navari
2,885
Reputation points Microsoft External Staff
Moderator
2 answers
How does data factory handle guid
How would you design a data pipeline in Azure Data Factory to move data from an on-premises SQL Server to an Azure Data Lake, ensuring data security, fault tolerance, and performance optimization?
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-26T05:31:59.26+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 59%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHS%3C/text%3E%3C/svg%3E)
Himanshu Sinha
11
Reputation points
commented 2025-06-27T08:58:17.4+00:00
Chandra Boorla
14,510
Reputation points Microsoft External Staff
Moderator
1 answer
Best way to ingest data from SAP BSEG table
I need to extract the following columns from the BSEG table: BUKRS, BELNR, SHKZG, HKONT, DMBTR, SGTXT, PRCTR, GJAHR I’ve tried: Filtering by HKONT values Partitioning by GJAHR (fiscal year) Combining both filters However, all attempts are resulting…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-25T10:00:20.27+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(179.20000000000002, 53%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Srinivas, Shreya
0
Reputation points
commented 2025-06-27T08:56:50.91+00:00
Chandra Boorla
14,510
Reputation points Microsoft External Staff
Moderator
4 answers
Issue with Oracle Connector Ver. 2.0 in ADF writing to Parquet fles
As recommended I have updated our Oracle connectors in our ADF linked services from ver 1.0 to 2.0 since ver 1.0 will be deprecated by 31st July 2025. Since using ver 2.0 we are getting errors when saving files to parquet format. Below is the error we…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-06T01:44:17.1966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 24%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKD%3C/text%3E%3C/svg%3E)
Kelman David
25
Reputation points
commented 2025-06-27T08:52:46.6933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 38%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
Mohammed Ben M'hand
0
Reputation points
1 answer
Data Flow Sink Fails with DF-Executor-InternalServerError and Spark Executor Timeout in ADF
I have a Data Flow activity in Azure Data Factory named DF_StageAccountInfo that was working previously without any issues. However, it suddenly started consistently failing with the following error: Operation on target DF_StageAccountInfo failed:…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-25T06:02:51.6933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 23%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGA%3C/text%3E%3C/svg%3E)
Grace Ann Salvame
0
Reputation points
edited an answer 2025-06-27T07:09:20.8533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 80%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKB%3C/text%3E%3C/svg%3E)
Krupal Bandari
770
Reputation points Microsoft External Staff
Moderator
1 answer
'json' parameter is not valid after updating secret key in ADF web activity
Hello, We have a pipeline with two web activities: The first retrieves a secret key from the key vault. The second uses that secret key to obtain a bearer token. Recently, we updated the expired secret key retrieved in the first web activity. The…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-03-20T15:33:29.6633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 72%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAO%3C/text%3E%3C/svg%3E)
Anneka Osmun
0
Reputation points
commented 2025-06-26T20:14:09.6733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 94%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Shaik Sadiq Basha (MINDTREE LIMITED)
0
Reputation points Microsoft External Staff
1 answer
Delta vs Parquet Format – Design Clarification for DB2 to Azure SQL Hyperscale Migration
In our current project, we are migrating approximately 80 TB of data across ~800 tables from IBM DB2 (on-prem) to Azure SQL Hyperscale. The architecture involves: Source: IBM DB2 Landing Zone: ADLS Gen2 (CSV format via ADF SHIR) Transformation:…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-20T06:58:30.36+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 56.00000000000001%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJC%3C/text%3E%3C/svg%3E)
Janice Chi
140
Reputation points
commented 2025-06-26T16:51:03.18+00:00
Chandra Boorla
14,510
Reputation points Microsoft External Staff
Moderator
1 answer
Azure ADF Copy Activity Fails After running for 11 hours with Transport level error
Hi, I am trying to copy multiple tables using the Copy Activity in the ADF pipeline from the Azure Data Warehouse to the Fabric Warehouse. However, the majority of the copy activities fail after running for a couple of hours with the below error. I am…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-25T06:04:03.4833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 44%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESP%3C/text%3E%3C/svg%3E)
Suraj Pardeshi
0
Reputation points
commented 2025-06-26T10:41:12.1+00:00
Venkat Reddy Navari
2,885
Reputation points Microsoft External Staff
Moderator
1 answer
How do we read a secret value in ADF using Lookup Activity or any other activity
How do we read a secret value in ADF using Lookup Activity or any other activity
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-19T06:04:42.9233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(240, 59%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EIN%3C/text%3E%3C/svg%3E)
Immaneni, Nanda
25
Reputation points
commented 2025-06-26T06:32:04.62+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
24,110
Reputation points Microsoft External Staff
Moderator
1 answer
One of the answers was accepted by the question author.
Copy data not recognizing existing headers
I'm very new to DF, so I need help figuring out the most efficient way to set up a pipeline. I have to copy .zip files from an SFTP, store them in Blob storage, and extract .csv files with certain names from the .zip file to process and move into a…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 46%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
1 answer
for first-time load of Account data into Dynamics 365 using Azure Data Factory (ADF).
for first-time load of Account data into Dynamics 365 using Azure Data Factory (ADF). I have a CSV file as source which has Account information. I need to import the Accounts from the CSV file into Dynamics 365 Account table. One of the column is parent…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-19T14:11:57.5166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 65%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENP%3C/text%3E%3C/svg%3E)
Nagalakshmi Pradeep
0
Reputation points
commented 2025-06-25T18:58:24.5466667+00:00
Chandra Boorla
14,510
Reputation points Microsoft External Staff
Moderator
3 answers
One of the answers was accepted by the question author.
How to Implement a Metadata-Driven ADF Pipeline to Dynamically Process Latest Files from ADLS?
Hello Community, I need help designing a single, reusable, and metadata-driven Azure Data Factory (ADF) pipeline for a sales order ingestion use case involving multiple source systems like SAP and Senior. Business Scenario We receive Excel files from two…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
asked 2025-06-20T09:57:32.63+00:00
YERNAIDU SIRAPARAPU
45
Reputation points
answered 2025-06-25T17:49:19.57+00:00
YERNAIDU SIRAPARAPU
45
Reputation points