Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
1 answer
One of the answers was accepted by the question author.
ADF error 'The object has not yet been initialized'
Hello, We have a pipeline that loads multiple data objects from SAP CDC source which runs okay for the past few weeks. This error only pops up in the last few days. Error: "The object has not yet been initialized." We have asked SAP team, they…
Azure Data Factory
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 75%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFL%3C/text%3E%3C/svg%3E)
2 answers
One of the answers was accepted by the question author.
How to fix ClientIPNotAuthorized on ADF
How do I fix the ClientIDNotAuthorized error message? I am on VPN as well.
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-05-13T20:02:20.89+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 24%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Sherry
20
Reputation points
commented 2025-06-12T00:38:58.1266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 92%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPK%3C/text%3E%3C/svg%3E)
Pritam Kabiraj
235
Reputation points Microsoft External Staff
Moderator
1 answer
Copy activity convert NUMBER to DOUBLE format
Hi we use the ADF copy activity to copy data from an oracle source to azure gen2 container as parquet file. The source data contain data type NUMBER, but the copy activity converts it to DOUBLE format. Could you please provide a solution for this…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-06-09T19:59:31.38+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 32%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZY%3C/text%3E%3C/svg%3E)
Zhu, Yueli YZ [NC]
280
Reputation points
commented 2025-06-11T17:17:40.98+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
13,790
Reputation points Microsoft External Staff
Moderator
1 answer
How to Capture Row-Level Schema Failures and Redirect Bad Records in ADF During DB2 to ADLS Ingestion
We are ingesting data from an IBM DB2 source into ADLS Gen2 using Azure Data Factory (ADF) Copy activity. The Copy activity is set up with schema validation enabled. During ingestion, if a row violates schema (e.g., data type mismatch), the entire…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-06-09T14:59:03.35+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 56.00000000000001%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJC%3C/text%3E%3C/svg%3E)
Janice Chi
100
Reputation points
commented 2025-06-11T11:53:46.5433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 99%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJN%3C/text%3E%3C/svg%3E)
J N S S Kasyap
3,300
Reputation points Microsoft External Staff
Moderator
1 answer
Architecture design
Hi friends, thank you for your help as always. My follow up question to below: https://learn.microsoft.com/en-us/answers/questions/2282164/architecture Sorry for silly question. How to get to good hands on the diagraming tools such as Draw IO etc. , I…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-06-10T13:24:57.7333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
615
Reputation points
commented 2025-06-11T09:47:19.6466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
24,015
Reputation points Microsoft External Staff
Moderator
1 answer
One of the answers was accepted by the question author.
Migration of large tables
we have 800 tables in DB2 and only 25 tables have 90% of data and other 775 tables has10% of data then what will be the best approach while migrating the data first to adlsgen2 and then to hyperscale via DB2-- ADF(SHIR) - ADLSGEN2- DBR(minimal trans) -…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-05-27T07:13:27.2233333+00:00
Janice Chi
100
Reputation points
commented 2025-06-11T03:49:53.1566667+00:00
Smaran Thoomu
24,015
Reputation points Microsoft External Staff
Moderator
2 answers
One of the answers was accepted by the question author.
ADF Pipeline error: Request content size is too large
I have a pretty basic ADF pipeline that checks a subdirectory for a group of flat files and based on the name of the file, it will load data into a SQL staging table. The ForEach activity has 7 If Condition activities which will run based on the…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2021-05-25T15:08:39.773+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 35%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Matt Michalowski
31
Reputation points
answered 2025-06-10T22:11:58.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(80, 32%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKR%3C/text%3E%3C/svg%3E)
Kleber Rebello
0
Reputation points
2 answers
One of the answers was accepted by the question author.
Data Factory copy activity is stuck in a "queued" status and not completing the data load
The Data Factory copy activity is stuck in a "queued" status and not completing the data load the same copy activity was running without any issues...now it is goind on queued
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 26%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
Where to find updates on the Couchbase (Preview) and its release for public use?
I see that there is a preview for a Couchbase connection to copy data from Couchbase to Azure Data Factory. Is there an expected release date for a production connection to Couchbase? Specifically for the use case Copying data to a Couchbase…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-06-10T15:03:15.76+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 5%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
Knara, Marcin
20
Reputation points
accepted 2025-06-10T16:07:39.5866667+00:00
Knara, Marcin
20
Reputation points
1 answer
designing a partition-based ingestion strategy
We are currently designing a partition-based ingestion strategy from IBM DB2 (on-prem) to ADLS Gen2 using Azure Data Factory (ADF). Since DB2 is a database and does not have a file system, we cannot use ADF’s Get Metadata activity to estimate partition…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-06-10T14:07:07.7833333+00:00
Janice Chi
100
Reputation points
answered 2025-06-10T15:17:16.2166667+00:00
J N S S Kasyap
3,300
Reputation points Microsoft External Staff
Moderator
3 answers
JSON to CSV transformation
I would like to below given json data to given csv format using data factory if possible json: { "employee":"123", "bankdetail":[ { "banknumber" : "bank1" }, { …
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2023-12-19T17:53:46+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 8%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anonymous
answered 2025-06-10T10:45:35.75+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 79%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAP%3C/text%3E%3C/svg%3E)
Anil Peter
0
Reputation points
2 answers
One of the answers was accepted by the question author.
Unexpected $500 Billing for Azure Synapse
I activated the Azure Synapse service on August 1st using the default settings. However, I noticed that I am being billed $250 per day, which has accumulated to a total of $500 over the last two days. As a student, I am using this service solely for…
Azure Cost Management
Azure Cost Management
A Microsoft offering that enables tracking of cloud usage and expenditures for Azure and other cloud providers.
3,525 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,357 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2024-08-03T18:13:18+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(284.8, 88%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
Paramjeet Singh
20
Reputation points
commented 2025-06-10T08:37:04.43+00:00
BeresfordBear
0
Reputation points
2 answers
One of the answers was accepted by the question author.
Truncated digits for numerical values in Azure Data Factory pipelines
Hello, when I am receiving numerical values within json response from API endpoint, Azure Data Factory is applying scientific notations into numerical values. But in the same time, it's also truncating last digits, see the following example: *Value…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-05-27T08:00:14.6866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 90%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Stanislav Slovik - Administrator
20
Reputation points
commented 2025-06-10T07:20:39.24+00:00
Stanislav Slovik - Administrator
20
Reputation points
0 answers
Connecting to On-Prem SQL Server through Change Data Capture (ADF) Issue
So I have been trying to get CDC working for the past two days but can't seem to get it to connect to my on-premise SQL Server as it seems to have issues using self hosted integration runtime inside the CDC wizard. I am trying to do incremental loads…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-06-04T21:17:53.26+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 62%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJS%3C/text%3E%3C/svg%3E)
Jason Sherbondy
0
Reputation points
commented 2025-06-09T20:09:33.45+00:00
Chandra Boorla
13,790
Reputation points Microsoft External Staff
Moderator
1 answer
Multiple Control tables or single control table
We’re currently working on a large-scale healthcare data migration project where data from on-prem DB2 is being ingested into Azure through a combination of ADF, SHIR, and Databricks. The project supports multiple ingestion modes, including one-time…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-05-23T13:21:43.01+00:00
Janice Chi
100
Reputation points
commented 2025-06-09T18:15:22.5+00:00
Smaran Thoomu
24,015
Reputation points Microsoft External Staff
Moderator
1 answer
How should DIU and Degree of Parallelism be configured in ADF when partition sizes vary significantly?
In our project, we are performing large-scale ingestion using Azure Data Factory copy activities. The source data is partitioned with highly variable sizes—some as small as 500 MB and others as large as 3 TB, all coming from the same source system. We…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-06-09T15:18:20.58+00:00
Janice Chi
100
Reputation points
answered 2025-06-09T16:18:57.63+00:00
J N S S Kasyap
3,300
Reputation points Microsoft External Staff
Moderator
1 answer
ADF Linked Service SHIR - Oracle connector 2.0 & TNSNAMES.ORA
Trying to find example on how to configure Linked service using Oracle connector 2.0 to point to TNSNames.Ora file which is available on SHIR server. Documentation is not clear. Can you please send me a link/information on how to make connection using…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-05-30T22:02:37.9533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(19.2, 49%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETS%3C/text%3E%3C/svg%3E)
Thakur, Satyajit
0
Reputation points
commented 2025-06-09T14:51:50.6266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(19.2, 17%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECD%3C/text%3E%3C/svg%3E)
chaitanya dasari
0
Reputation points
1 answer
One of the answers was accepted by the question author.
ADF managed Airflow error : 'ReferenceError: Cannot access 'Z' before initialization'
Hi, I would like to obtain the confirmation that the following error, occurring when creating a new managed Airflow integration runtime in ADF, is caused by the ADF of being new/recent (created in early 2025) and not supporting anymore Airflow as a WOM,…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-06-06T17:27:49.5133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 89%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELL%3C/text%3E%3C/svg%3E)
Louis Lalonde
40
Reputation points
edited a comment 2025-06-09T12:13:22.63+00:00
Louis Lalonde
40
Reputation points
2 answers
One of the answers was accepted by the question author.
Can we compare the cost of DBR vs DLT via pricing calculator ?
Can we compare the cost of DBR vs DLT via pricing calculator ?
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-06-06T14:19:45.5933333+00:00
Janice Chi
100
Reputation points
accepted 2025-06-09T11:21:44.8733333+00:00
Janice Chi
100
Reputation points
2 answers
One of the answers was accepted by the question author.
Airflow Azure Data Factory VNet configuration
Hi, I'm looking to know if it's possible to inject Azure Data Factory (ADF) managed Apache Airflow in a VNet directly from the ADF studio configuration page ? If it's not possible, what is the recommended strategy to obtain this configuration…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,582 questions
asked 2025-06-02T17:55:20.2966667+00:00
Louis Lalonde
40
Reputation points
commented 2025-06-09T09:08:14.36+00:00
Amira Bedhiafi
32,756
Reputation points
Volunteer Moderator