Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,845 questions
1 answer
One of the answers was accepted by the question author.
Data factory API Trigger runs by order
The Sample Request and Body in the examples does not show how to sort the order. How do you get the latest succeeded run ? https://learn.microsoft.com/en-us/rest/api/datafactory/triggerruns/querybyfactory Managed to filter the successfull runs…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 70%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EIH%3C/text%3E%3C/svg%3E)
1 answer
Give all (inclusing new) datafactories access to my keyvault?
I have several datafactories and there will be more. There is one Azure Keyvault. Is there a way to give all datafactories access to this keyvault without specifying each one individually? Through a securitygroup or such?
asked 2020-09-04T13:57:20.503+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 30%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBH%3C/text%3E%3C/svg%3E)
BIP | Henro Veijer
1
Reputation point
commented 2020-09-25T23:35:58.383+00:00
JamesTran-MSFT
36,491
Reputation points Microsoft Employee
1 answer
Azure Log Analytics recursion - Get parent (or parent's parent) pipeline info for Activity in Data Factory
I have a number of data factories hooked up to a log analytics workspace. Currently I am creating a outage dashboard that looks at failed activities. I would like to show the initiating pipeline in this report for activities that failed. However, that…
asked 2020-09-18T16:09:04.547+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 45%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJA%3C/text%3E%3C/svg%3E)
John Aherne
516
Reputation points
answered 2020-09-25T23:25:35.737+00:00
olufemia-MSFT
2,861
Reputation points
1 answer
Missing Database
I can see my database in SSMS but I am unable to see it in Data Factory. Unfortunately it is the database that I need to get data into. It is in the same subscription, resource group, and on the same server with the same permissions.
asked 2020-09-17T13:41:32.093+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 25%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELT%3C/text%3E%3C/svg%3E)
Lee Taylor
31
Reputation points
commented 2020-09-25T19:40:39.81+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 26%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HimanshuSinha-msft
19,386
Reputation points Microsoft Employee
2 answers
ADF Copy activity using PolyBase fails to load "Date" type to Synapse
When loading Date type from a parquet file to Synapse with "Date" type via ADF Copy with PolyBase, the ADF throws the following error: { "errorCode": "2200", "message":…
asked 2020-09-24T14:46:46.05+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 15%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Robert Kostecki
6
Reputation points
answered 2020-09-25T17:15:07.9+00:00
Robert Kostecki
6
Reputation points
3 answers
One of the answers was accepted by the question author.
Integration runtime offline - The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.
Our self-hosted integration runtime has been successfully running for over a year but as of yesterday, it has been offline (error from log posted below). It's installed using the default settings (no proxy) and we have confirmed outbound access on :443…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 0%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
Azure Data factory - Conversion from Duration to execution hours unclear
Hi, I am following the conversion from pipeline duration to execution hour and I am not able to folow what is happening. I am using a Self hosted IR so DIU is not applicable. Example; How does 00:01:32 become 0.5333 execution hours? See attached…
asked 2020-09-18T07:42:54.927+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 52%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
markharing@bisqq.nl
61
Reputation points
accepted 2020-09-25T08:40:18.303+00:00
markharing@bisqq.nl
61
Reputation points
1 answer
One of the answers was accepted by the question author.
Performance, Pros and Cons of using Logic Apps or Azure Data Factory for Orchestration of huge ADW and Performance, Pros and Cons of using ADFV2 Mapping Data Flows
Please share Performance, Pros and Cons of using Logic Apps or Azure Data Factory for Orchestration for huge ADW and Performance, Pros and Cons of using ADFV2 Mapping Data Flows
asked 2020-09-11T07:30:47.203+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 84%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFP%3C/text%3E%3C/svg%3E)
Faiaz Pasha Shaik
21
Reputation points Microsoft Employee
accepted 2020-09-25T08:21:21.553+00:00
Faiaz Pasha Shaik
21
Reputation points Microsoft Employee
4 answers
One of the answers was accepted by the question author.
Excel Mapping Data Flow - Schema is set as text but value has been parsed
I am trying to use the Excel Mapping Data Flow (inline) and there is another problem in the way that the values are being parsed. I have a column which contains a whole integer value and the schema type is importing as text but the value has been…
asked 2020-08-15T10:34:00.977+00:00
Mathew Partridge
221
Reputation points
accepted 2020-09-25T06:58:52.443+00:00
Mathew Partridge
221
Reputation points
0 answers
Azure Data Factory - Pipelines stuck in Queued status
There are no Azure Data Factory Pipelines in progress, but all scheduled Pipeline executions are getting created in a status of "Queued" and not ever moving to "In Progress". Support suggested yesterday it could be related to this…
asked 2020-07-09T11:55:35.65+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 40%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJH%3C/text%3E%3C/svg%3E)
Jason Hendrix
1
Reputation point
commented 2020-09-25T00:50:57.343+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 70%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
MartinJaffer-MSFT
26,041
Reputation points
4 answers
One of the answers was accepted by the question author.
Azure Datafactory passing parameter to a datasource on a dataflow
Hi, I'm trying to pass a parameter from a pipeline into a dataflow, which has some datasources, and these datasources has parameters that i want to fulfill. This is an overview from my pipeline, and there is too the parameters that i want to…
asked 2020-09-17T13:28:03.867+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(208, 70%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHS%3C/text%3E%3C/svg%3E)
Higor Silva Grilo
66
Reputation points
accepted 2020-09-24T17:35:48.87+00:00
Higor Silva Grilo
66
Reputation points
1 answer
One of the answers was accepted by the question author.
DataSource receive parameters from Dataflow
Hi, I'm having some problems making the DataSource in receive the parameters that comes from the Dataflow that uses the DataSource. This is my parameters configuration on the DataSource side: Wich i use this code to get the parameter…
asked 2020-09-23T19:38:17.15+00:00
Higor Silva Grilo
66
Reputation points
accepted 2020-09-24T14:39:46.847+00:00
Higor Silva Grilo
66
Reputation points
1 answer
Security Recommendation for Azure SSIS IR in ADF
Can someone please let me know what security recommendation are there for Azure SSiS IR in Azure data factory ?
asked 2020-09-11T19:43:09.673+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 61%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAV%3C/text%3E%3C/svg%3E)
Akash Verma
21
Reputation points
commented 2020-09-24T05:08:10.73+00:00
PRADEEPCHEEKATLA-MSFT
81,391
Reputation points Microsoft Employee
1 answer
How to show the task tab in Azure Data Studio
My connection tab looks like this, it's hard to switch to another connect or SQL file How can I restore it like the preset view ,like this Thanks a lot
asked 2020-09-16T06:04:14.007+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 91%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJT%3C/text%3E%3C/svg%3E)
Jiawen Tang
1
Reputation point
commented 2020-09-24T05:07:31.773+00:00
PRADEEPCHEEKATLA-MSFT
81,391
Reputation points Microsoft Employee
1 answer
Store procedure with output param in lookup
Hi, I am using Lookup activity to execute store procedure with an output parameter. But I have noticed if the store procedure doing Insert, then "message": "Failure happened on 'Source' side.…
asked 2020-09-23T12:52:55.24+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(19.2, 98%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
rajesh kumar
1
Reputation point
answered 2020-09-23T18:07:53.133+00:00
HimanshuSinha-msft
19,386
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Data Factory copy data to another container
Hi Everyone, Could you help me please In Data Factory I created Copy Data pipelines from one Storage account (Containers) to another Storage account (Containers), copy working fine , could tell me please how I can configured to keep every time new…
asked 2020-09-17T23:23:48.12+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 85%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAC%3C/text%3E%3C/svg%3E)
Alex Corovin
46
Reputation points
accepted 2020-09-23T00:16:21.977+00:00
Alex Corovin
46
Reputation points
1 answer
Copy data from multiple Azure DB's to one Azure DB with Data Factory
I'm not sure that this is possible for this service or not. I have many databases of the same schema that I want to copy data on a monthly data into one single database. I need to be able to use a Table that contains a list of these databases and…
asked 2020-09-17T16:16:24.283+00:00
Lee Taylor
31
Reputation points
commented 2020-09-22T20:18:20.293+00:00
MartinJaffer-MSFT
26,041
Reputation points
1 answer
One of the answers was accepted by the question author.
Does data factory Copy Activity check if file at source has finished writing ?
Hello, I am using a Copy Activity to copy recent files (past 15 minutes by checking LastModifiedDate in the copy activity) from SFTP to ADLS Gen 2. At the moment, to make sure that the files the activity copies are complete in the source and not…
asked 2020-09-22T08:48:33.7+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 79%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEB%3C/text%3E%3C/svg%3E)
Emilien BOUCAUD
21
Reputation points
accepted 2020-09-22T12:21:30.48+00:00
Emilien BOUCAUD
21
Reputation points
2 answers
Data Factory V2 Storage Account not Refreshing - Because it thinks its blob Storage
I have created a Storage Account set to be Azure Data Lake V2 by setting the hierarchical namepace. And I used Azure Storage Explorer to add some files into the blob Container (Which is an Azure datalake gen2) I'm now creating a Data Lake V2 Linked…
asked 2020-07-31T10:10:17.587+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 27%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDE%3C/text%3E%3C/svg%3E)
Debbie Edwards
521
Reputation points
answered 2020-09-22T10:55:12.323+00:00
Debbie Edwards
521
Reputation points
1 answer
One of the answers was accepted by the question author.
Azure Data factory - Reading nvarchar column from excel source xlsx in azure data factory but numeric data is not being read properly!!
Column col1 has both numeric and character data so reading it as string in excel source(xlsx). Excel data ex. col1 (General Format) DFSDF 23 12345678 Expecting to read the data as it is in azure data factory ,but its altering the numeric data and…
asked 2020-09-10T08:22:01.303+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(284.8, 83%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGS%3C/text%3E%3C/svg%3E)
Gandhikumar, Suganthi
31
Reputation points
accepted 2020-09-22T05:35:06.88+00:00
Gandhikumar, Suganthi
31
Reputation points