Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,914 questions
1 answer
Why does Data Factory Salesforce Connector not pull correct column formats?
We had a data factory pipeline connecting to salesforce using integration runtime 4.9 and everything worked perfect. We upgraded our integration runtime to 5.1 because of a sql server connector issue and now our salesforce connector does not work…
asked 2024-06-18T12:37:25.0066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 55.00000000000001%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBD%3C/text%3E%3C/svg%3E)
Blasko, Dan W
0
Reputation points
answered 2024-06-19T14:48:27.39+00:00
Adam VEGH
0
Reputation points
1 answer
Getting the SharePoint List's Modied By and Created By in the Azure Data Factory Lookup query
I am trying to receive from the SharePoint List's columns with names of the 'Created By' and 'Modified By' users. As a default, Lookup activity (and Copy data which I am using) in the ADF doesn't return these fields, only CreatedById and ModifiedById.…
asked 2021-09-10T10:17:03.123+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(32, 57.99999999999999%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFL%3C/text%3E%3C/svg%3E)
Filip Lisiecki
21
Reputation points
edited a comment 2024-06-19T14:23:21.5466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 50%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Suraj Sasidaran
0
Reputation points
1 answer
[CosmosToKusto] fails due to System.MissingMethodException,Message=Method not found
CosmosToKusto worked fine before but recently all jobs failed with below error. What should I do to fix it? Thanks for your help! Failure type User configuration issue Details 'Type=System.MissingMethodException,Message=Method not found:…
asked 2024-06-19T12:10:07.3766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 28.000000000000004%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYZ%3C/text%3E%3C/svg%3E)
Yi Zeng (CAFE)
0
Reputation points Microsoft Employee
commented 2024-06-19T13:56:42.7233333+00:00
Yi Zeng (CAFE)
0
Reputation points Microsoft Employee
0 answers
How can I retain null values for decimal type columns when copying data from Snowflake source to Azure Blob Store Excel sink?
I am using the ADF copy activity to transfer decimal data type columns with null values from Snowflake source to Azure Blob Storage CSV/Parquet file sinks. However, the null values are getting converted to empty values. Is it possible to retain these…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 73%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EK%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
Not Able to see underline Columns for Joining in Data Flow Activity in ADF
Hi Community, I am using Data Flow Activity to transfer the data from Azure SQL DB to Azure Synapse (Dedicated Sql Pool) But I am not able to see underline Columns for Joining in Data Flow Activity in ADF.. Pls advise.. When I preview data, I can see…
asked 2024-06-18T11:43:37.7966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 24%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
Ashish Asthana
40
Reputation points
commented 2024-06-19T12:56:50.9666667+00:00
Amira Bedhiafi
18,021
Reputation points
3 answers
How to transform sharepoint list items into an excel format
I'm trying to fetch the columns inside a sharepoint list using below MS Graph API:- sites/site-id/lists/list-id/items?expand=fields Below is the output that I'm getting:- { "@odata.context":…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 79%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
1 answer
Error Combining SharePoint Files into Lakehouse Table
Hi Team, I am attempting to combine all files in the SharePoint folder and load them into a fabric lakehouse table. I successfully completed a similar task on another Data Flow, but this one is failing unexpectedly. Can you please assist with this issue?…
asked 2024-06-19T11:53:46.71+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(240, 32%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EG%3C/text%3E%3C/svg%3E)
Glasier
340
Reputation points
answered 2024-06-19T12:00:56.38+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
11,535
Reputation points Microsoft Vendor
0 answers
How to connect a SAP SLT hosted on GCP with Azure Data factory
We have a SAP SLT box hosted on GCP. The data needs to be moved to Azure Lakehouse architecture. Can this be connected using ADF via SHIR setup? Will SHIR be able to connect the SAP SLT hosted on GCP with Azure?
asked 2024-06-19T09:43:28.4866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 49%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGJ%3C/text%3E%3C/svg%3E)
Ganesan, Jothy
0
Reputation points
commented 2024-06-19T11:51:12.8633333+00:00
Ganesan, Jothy
0
Reputation points
1 answer
How to convert string to datetime in copy activity
I have delimited text files and I am trying to copy that into my sql db. I have some date columns and it is empty in certain cases, so my copy activity is throwing the following error, how to fix this ErrorCode=TypeConversionFailure,Exception occurred…
asked 2024-06-19T02:17:44.7233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 57.00000000000001%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHR%3C/text%3E%3C/svg%3E)
Hermoine Ron
0
Reputation points
edited a comment 2024-06-19T11:45:21.4133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Harishga
5,110
Reputation points Microsoft Vendor
0 answers
Message=The attribute 'status' with value '100000006' cannot be converted from original type 'System.String' to target type 'Microsoft.Xrm.Sdk.OptionSetValue'.
I want to update CRM data with the same value 100000006, but use copy data additional column failed. @array(json('{"name":"status","value":100000006}'))
asked 2024-06-18T03:39:13.7166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(284.8, 44%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECF%3C/text%3E%3C/svg%3E)
Chen, Flora-F1
0
Reputation points
commented 2024-06-19T11:24:50.27+00:00
Smaran Thoomu
11,535
Reputation points Microsoft Vendor
3 answers
Cannot connect to SQL Database. No process is on the other end of the pipe
I am trying to create a Linked Service on ADF for SHIR (this is up and connected to ADF) and when i fill in all details and click Test Connection. I get this error message: Cannot connect to SQL Database. Please contact SQL server team for further…
asked 2024-06-16T07:17:35.5833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 22%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVD%3C/text%3E%3C/svg%3E)
Vijaykumar Dinni
0
Reputation points
commented 2024-06-19T11:05:59.8333333+00:00
Smaran Thoomu
11,535
Reputation points Microsoft Vendor
4 answers
One of the answers was accepted by the question author.
Why ForEach activity declaring success in Data Factory taking too long?
We have a ForEach activity running in an Azure Data Factory pipeline to copy data from source to destination. But ForEach activity taking long time to success where as all other inner activity is completed. We are experience this issue from past few…
asked 2024-06-15T08:28:26.0933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(48, 99%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDB%3C/text%3E%3C/svg%3E)
Dharmendra Barad
20
Reputation points
accepted 2024-06-19T11:04:10.9833333+00:00
Dharmendra Barad
20
Reputation points
0 answers
How to use wildcard for XML Files in Copy Data
I have a Azure Data Lake Storage Gen2 with following folder and file structure .\source .\source\SystemA_20240618.xml .\source\SystemB_20240618.xml .\source\SystemA_20240619.xml .\source\SystemB_20240619.xml I need to process only the files…
0 answers
Static Code Analysis for ADF or Synapse Pipelines in Azure
Hi team, I would like to know is there any use case possible to perform static code analysis of our ADF or Synapse pipelines residing in Azure DevOps repository? Thanks, Debajit Mitra
asked 2024-06-14T03:42:29.93+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 48%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDM%3C/text%3E%3C/svg%3E)
Debajit Mitra
0
Reputation points
commented 2024-06-19T10:36:14.2+00:00
Smaran Thoomu
11,535
Reputation points Microsoft Vendor
1 answer
Error de Conexión REST: TrustFailure y Certificado Inválido en Azure Data Factory
Description: Hello everyone, I'm facing an issue while trying to connect a REST service in Azure Data Factory. The configuration is as follows: Base URL: https://yam.3psi.com.pe/wsextdatsee/api/operacioncliente?datpet=20383929050_1_Lgo2023$1 …
asked 2024-06-12T16:34:44.1533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 97%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EY%3C/text%3E%3C/svg%3E)
Yamaha_motors
0
Reputation points
edited an answer 2024-06-19T10:19:56.6866667+00:00
Smaran Thoomu
11,535
Reputation points Microsoft Vendor
2 answers
Accessing a SQL Server without going via VM
Good day Members I have connect using an IPSec to a on prem SQL DB(this is the only way). I can connect to the SQL DB in a VM. I have been using PowerBI and SSMS in the VM. I want to sink the data into an Azure SQL and use it without going via VM. I…
asked 2024-06-19T07:54:17.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 23%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
StudentLearner
0
Reputation points
commented 2024-06-19T10:15:41.0866667+00:00
StudentLearner
0
Reputation points
1 answer
Convertion of decoded string response from SOAP API to Integer
I am trying to get count of rows through SOAP API. The response is binary 64-bit encoded which is then decoded and replaced with spaces to get only the row count. Problems: Unable to convert decoded binary string row count into integer. Used…
asked 2024-06-17T23:46:09.68+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 65%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZR%3C/text%3E%3C/svg%3E)
Zain Rasheed
0
Reputation points
commented 2024-06-19T10:10:07.1133333+00:00
Harishga
5,110
Reputation points Microsoft Vendor
0 answers
Azure Data Factory - Jira Connector Schema Change / missing data
Hi All, We have been using ADF to ingest Jira into our data lake for several months now. The connector has worked with no issues and we have been able to pull all issues and all the custom fields, etc from Jira. Our Monday refresh failed due to a schema…
asked 2024-06-19T05:59:12.65+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 62%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
Jose
0
Reputation points
commented 2024-06-19T10:08:40.58+00:00
Smaran Thoomu
11,535
Reputation points Microsoft Vendor
0 answers
Issue with Salesforcev2Connector
Hi team, I have a question regarding this article. First I have created a Linked Service, DataSet and copy activity in the pipeline which copy from SalesForce to Azure SQL database through…
asked 2024-06-12T09:48:44.23+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(249.60000000000002, 81%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMJ%3C/text%3E%3C/svg%3E)
Michał Janczewski
0
Reputation points
commented 2024-06-19T08:51:19.1233333+00:00
Michał Janczewski
0
Reputation points
2 answers
One of the answers was accepted by the question author.
How to give a Sharepoint folder as a destination in Microsoft Fabric CopyData activity
In Microsoft Fabric, I am trying to setup a Copy Data activity that executes a stored procedure from Azure SQL Database and outputs a csv file to a Sharepoint Folder. I have copied the link from the sharepoint folder into the Connection field. The Test…
asked 2024-06-18T15:20:10.2733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 73%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGR%3C/text%3E%3C/svg%3E)
George Rustean
0
Reputation points
accepted 2024-06-19T08:44:58.0833333+00:00
George Rustean
0
Reputation points