Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,160 questions
0 answers
ADF v2 Integration Runtime for Network share paths
HI I am looking to access the Network shared fodler with the help of Intgeration Runtime setup which is consumed by my copy activity, i am not able to access the path of the network share even after providing the required credentials which has the…
asked 2020-06-03T19:52:06.92+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 88%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
ManojMathe
1
Reputation point
commented 2020-06-11T06:05:02.403+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 76%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECM%3C/text%3E%3C/svg%3E)
ChiragMishra-MSFT
956
Reputation points
1 answer
How to add a Joing Query for pipeline Source
Hi Team, Currently i have configure integration Runtime and link services, After then I have to setup a join query for a pipeline source area as per below snap, but our concern is when i create a source connection i'm only allow to select one table…
asked 2020-06-04T02:45:02.047+00:00
Mahesh Madhusanka
106
Reputation points
commented 2020-06-11T06:02:09.483+00:00
ChiragMishra-MSFT
956
Reputation points
4 answers
ADF Data Flow Execution Failed
Hi Team, We are facing issue with Data Flow execution for past 3 hours and not able to run any pipelines. Can you please check the issue? { "errorCode": "4501", "message": "Failed to fetch execution state: '',…
asked 2020-06-04T11:27:17.22+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 88%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Sathish Kumar Sermadurai
16
Reputation points
commented 2020-06-10T16:50:32.903+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 8%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDF%3C/text%3E%3C/svg%3E)
Dan Faustine
1
Reputation point
3 answers
One of the answers was accepted by the question author.
Azure DataFactory : Set up Self Hosted IR
Hi Folks, I cannot set up a self hosted IR in my local machine. The purpose of this set up is to copy data from a SFTP server via a specific server to azure datalake. I have a admin credentials on my machine, however i get an error…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 9%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EK%3C/text%3E%3C/svg%3E)
2 answers
Connectivity between Azure data factory and Azure data catalog
Hi Team, We have a use case to use the azure data catalog Metadata information in Azure data factory. We have gone through the documentation however could not find any information on this. Could you help me if the feature is already available in…
asked 2020-06-09T07:29:32.307+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(124.80000000000001, 38%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJR%3C/text%3E%3C/svg%3E)
Jyothi Ragi
1
Reputation point
answered 2020-06-09T07:39:22.73+00:00
PRADEEPCHEEKATLA-MSFT
85,511
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Change node size in Azure Data Factory (Spark)
Hello! How do I change the node Size in Azure Data Factory for HD Insight? I Want to create an Standard_D13_v2. But I didn't find a tutorial for do it:
asked 2020-06-06T00:51:51.307+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 42%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJC%3C/text%3E%3C/svg%3E)
Joaquin Chemile
41
Reputation points
commented 2020-06-09T05:34:06.25+00:00
PRADEEPCHEEKATLA-MSFT
85,511
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Azure Data Flow : Location
Azure Data Flow : is it supported in Location Australia South East? Looks like it is supported in Australia East. My Data Factory is built in Australia South East. To use data flows should I clone to different location? Thanks
asked 2020-06-08T21:30:50.48+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 94%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
Alan Anscombe
51
Reputation points
accepted 2020-06-08T22:58:48.367+00:00
Alan Anscombe
51
Reputation points
2 answers
One of the answers was accepted by the question author.
Delay between Activities in Data Factory
We have pipeline that has been running for 6 months in Data Factory V2. Last night there was a 48 minute delay from between two activities , no queued time. Does any know if there was maintenance on Data Factory V2 in West Central US on 5/27/2020 9:40…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 32%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EB%3C/text%3E%3C/svg%3E)
3 answers
Setting Pane in Mapping Data FLow in Azure Data Factory
Hi Experts , Can anyone advise me why am getting 'Settings pane'/'Data Preview Pane 'for configuring activity in Mapping Data FLows in bottom right hand corner of the screen ?. It was ok yesterday but seems from today am getting this View . Can…
asked 2020-06-02T06:54:42.487+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(284.8, 42%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENK%3C/text%3E%3C/svg%3E)
n k
106
Reputation points
commented 2020-06-08T20:33:34.937+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 9%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KranthiPakala-MSFT
46,442
Reputation points Microsoft Employee
1 answer
AZURE SQL MI Error: Stale/aborted version cleanup was aborted for database id '10' due to database shutdown.
Hi, I am trying to restore SQL server .bak file to Azure SQL Managed Instance database through SSMS but getting an error Msg 22003, Level 16, State 1, Line 18 Stale/aborted version cleanup was aborted for database id '10' due to…
asked 2020-06-05T18:47:16.897+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 57.00000000000001%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH0%3C/text%3E%3C/svg%3E)
HemantSudehely-0557
266
Reputation points
commented 2020-06-08T18:03:41.503+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 93%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENS%3C/text%3E%3C/svg%3E)
Navtej Singh Saini
4,221
Reputation points Microsoft Employee
0 answers
toString does not accept array types
Hi All, Dataflows which have been working in the production environment have now started failing due to this error (DF-FN-006 - toString does not accept array types) This seems to be a recent change and I have quite a lot Mapping Dataflows which…
asked 2020-06-02T21:06:13.117+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 86%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
Rajneel Sharma
1
Reputation point
commented 2020-06-04T21:38:20.647+00:00
KranthiPakala-MSFT
46,442
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Use single self hosted IR to source on-Prem sql database and target Azure VM sql server data server
I have required to perform Azure Data Factory Copy activity. My source is the On-Prem SQL server database and target sink activity to the Azure VM SQL server data server. I did try by creating 2 self-hosted IR(one for source and one for target) but…
asked 2020-06-03T16:40:42.713+00:00
HemantSudehely-0557
266
Reputation points
accepted 2020-06-04T13:38:37.177+00:00
HemantSudehely-0557
266
Reputation points
1 answer
One of the answers was accepted by the question author.
Azure Data Factory extracting Activities from Dynamics 365
I have a ADF extracting 'activity' (activitypointer) from Dynamics 365 Online. The ADF runs for 60+ minutes and fails with the below error for the copy activity. The sink is a gen2 data lake. I have 'AutoResolveIntegrationRuntime' for this data factory.…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(51.2, 87%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESD%3C/text%3E%3C/svg%3E)
1 answer
ADF extract data from an on-premises oracle database into azure SQL database in real time
We have lots of web application on-premises, and its database include: Oracle, SQL Server, MySQL and SAP HANA. For Oracle DB, is it possible that ADF can extract data in read time? When there is any update in Oracle database, ADF can SYNC these…
asked 2020-06-02T02:02:42.407+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(272, 53%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EWF%3C/text%3E%3C/svg%3E)
Wenjian Feng
41
Reputation points
commented 2020-06-03T08:43:10.39+00:00
Wenjian Feng
41
Reputation points
1 answer
How to push data from Azure Data lake to remote file server (network file folder)
Hello Experts, I need your guidance on how to push files from Azure Data lake to remote file server(network file folder). Our requirement is to send files from Azure Data Lake Gen2 to Remote File Server( Network File Folder), once the data pushed…
asked 2020-06-01T08:56:36.84+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 78%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EF%3C/text%3E%3C/svg%3E)
fbmsdn511
1
Reputation point
commented 2020-06-03T01:24:27.297+00:00
KranthiPakala-MSFT
46,442
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Error on dynamic content at Set Variable Activity
When trying reading the output for a Web Actvity I get the following error message: {"code":"BadRequest","message":"ErrorCode=InvalidTemplate, ErrorMessage=The expression…
asked 2020-05-29T17:17:22.803+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(259.20000000000005, 92%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELB%3C/text%3E%3C/svg%3E)
Lawrence Borbón
21
Reputation points
commented 2020-06-01T16:32:45.913+00:00
Lawrence Borbón
21
Reputation points
1 answer
Unable to open the Author & Monitor page from my ADF subscription
I'm unable to open the Author & Monitor page from my ADF subscription. my Subscription id is ########-####-####-####-############ I've tried to use different browsers. I've enabled my cookies on my browser, but I continue to see a blank screen…
asked 2020-05-27T20:17:00.183+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 64%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Adnan Rahman
1
Reputation point
commented 2020-05-28T20:31:03.27+00:00
KranthiPakala-MSFT
46,442
Reputation points Microsoft Employee
0 answers
Azure Data Factory - Copy activity from ADLS to Cosmos Mongo DB
Hi, While running an Azure Data factory pipeline which copies data from AZure ADLS Gen storage 1 location to Azure Cosmos Mongo DB, I am receiving the below exception. Request to please help with regards to resolution. Thanks, Guru EXCEPTION : …
asked 2020-05-22T16:04:01.897+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(240, 36%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBG%3C/text%3E%3C/svg%3E)
Banaji, Gurumohan
1
Reputation point
commented 2020-05-28T05:53:49.447+00:00
Mike Ubezzi
2,776
Reputation points
1 answer
Copy Activity in Azure data factory do not copy multi line text in sql table maintaining the line breaks.
I am using copy data activity from CSV to SQL. In CSV I have a field with multiline string in csv for e.g. Refer attachment But when I run the pipeline in SQL I an seeing the data in a single line like 1 Eastlawn Alley 8528 Coolidge Trail 0133…
asked 2020-05-04T07:12:21.883+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 94%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGK%3C/text%3E%3C/svg%3E)
Govinda Kumar
1
Reputation point
answered 2020-05-08T13:58:21.857+00:00
Vaibhav Chaudhari
38,681
Reputation points
1 answer
Summary of pipelines
I want to document all the details of a particular pipeline in my ADF. Ex: which linked services are being used in my parent pipeline and child pipeline or which datasets are being used. Is there any option to summarize these details in Azure portal?
asked 2020-02-11T16:40:05.99+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 74%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMU%3C/text%3E%3C/svg%3E)
Manideep Unnamadala
1
Reputation point
answered 2020-02-11T22:30:04.157+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 81%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMT%3C/text%3E%3C/svg%3E)
Marilee Turscak-MSFT
36,336
Reputation points Microsoft Employee