Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,891 questions
0 answers
Dynamic mapping adf copy activity
Hi I am trying to do a dynamic mapping between file to SQL Table in ADF Copy activity. In the mapping section, I am adding the following dynamic content , for just one column for testing purpose. But it gives error "Expression of type: 'String'…
asked 2024-06-10T03:37:59.9866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(179.20000000000002, 73%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKN%3C/text%3E%3C/svg%3E)
Krishnamohan Nadimpalli
401
Reputation points
commented 2024-06-12T05:59:32.34+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
7,345
Reputation points Microsoft Vendor
0 answers
Dictionary error in ADF
Trying to copy data from SQL server to Snowflake (new connection) and it fails with the below error. 'Type=System.Collections.Generic.KeyNotFoundException,Message=The given key was not present in the dictionary.,Source=mscorlib,' With Snowflake legacy…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 54%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
1 answer
I get "Error in loading user data" whenever I try to access any of my resources
I was working on Azure when the network flipped and since then I can't access any resource although the network is back and stable
asked 2024-06-10T11:46:52.49+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 82%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Sara
0
Reputation points
commented 2024-06-11T20:42:04.5033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BhargavaGunnam-MSFT
28,111
Reputation points Microsoft Employee
0 answers
CopyFromSourcetoADLS, Error: The Self-hosted Integration Runtime 'integrationRuntime-edw' is offline,
Operation on target ForEach1 failed: Activity failed because an inner activity failed; Inner activity name: CopyFromSourcetoADLS, Error: The Self-hosted Integration Runtime 'integrationRuntime-edw' is offline, last connect time is '06/07/2024…
asked 2024-06-07T12:30:28.25+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(182.40000000000003, 55.00000000000001%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVM%3C/text%3E%3C/svg%3E)
Vineyard, Mr. Scotty
1
Reputation point
commented 2024-06-11T20:39:16.29+00:00
BhargavaGunnam-MSFT
28,111
Reputation points Microsoft Employee
1 answer
getmetadata activity based on an azure storage fileshare
Hello, I am trying to create a new "getmetadata" activity, based on a "file share" that resides in an "azure storage", it seems as it is not supported for a getmetada but not sure if there is any other option to list the…
asked 2024-06-07T12:21:32.8566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 18%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJZ%3C/text%3E%3C/svg%3E)
Jon Z
20
Reputation points
commented 2024-06-11T20:37:16.35+00:00
BhargavaGunnam-MSFT
28,111
Reputation points Microsoft Employee
1 answer
ADF Free trail
In free trail version of ADF while we are creating the SQL database it is showing some chargeable amount for that I need to pay any amount?
asked 2024-06-07T10:26:32.02+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 22%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKG%3C/text%3E%3C/svg%3E)
Kappaganthu Gurunadham
0
Reputation points
commented 2024-06-11T20:35:58.3+00:00
BhargavaGunnam-MSFT
28,111
Reputation points Microsoft Employee
1 answer
ADF Webactivity response XML accessing next page
XML response is giving next page url token, how to accomplish this is ADF using looping.
asked 2024-06-05T13:03:05.3466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 88%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
sankon437
21
Reputation points
commented 2024-06-11T20:21:52.4833333+00:00
BhargavaGunnam-MSFT
28,111
Reputation points Microsoft Employee
2 answers
Integration Runtime Oracle database connection error ERROR [08S01] : [Socket closed, Connection Dead] | ERROR [HYT00] : [Timeout expired] | ERROR [HY000] : [Network Operation Timed Out]
While connecting to an Oracle database from Azure Data Factory using Integratio Runtime we recieve the following error when executing pipelines that either does "Lookup" functions with typical "select * from ..." queries or in…
asked 2021-11-03T13:46:38.093+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 21%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERA%3C/text%3E%3C/svg%3E)
Ruben Austefjord
6
Reputation points
commented 2024-06-11T19:46:58.6833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(160, 1%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
SH
0
Reputation points
0 answers
ADLS Gen2 failed for forbidden: Storage operation 'CreateFile' on container ' ' and path ' ' get failed with 'Operation returned an invalid status code 'Forbidden''
Hi team, getting the below error while running the adf job, could you please check. ErrorCode=AdlsGen2ForbiddenError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 failed for forbidden: Storage operation 'CreateFile'…
asked 2024-06-11T15:57:03.4033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(25.6, 2%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
Mahasiva, Ravi (CSW)
60
Reputation points
commented 2024-06-11T19:40:32.26+00:00
BhargavaGunnam-MSFT
28,111
Reputation points Microsoft Employee
2 answers
One of the answers was accepted by the question author.
New salesforce connector doesn't support SOQL syntax
trying to convert to new salesforce connector and getting 'unbound' error when trying to return all fields for an object such as select fields(all) from profile where id = 'xxx'. using version 60.0 in linked service Seems like only the select…
asked 2024-06-11T16:23:08.8133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 66%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBA%3C/text%3E%3C/svg%3E)
Boilard, Andre
46
Reputation points
answered 2024-06-11T18:44:22.61+00:00
BhargavaGunnam-MSFT
28,111
Reputation points Microsoft Employee
2 answers
Error in connecting Azure SQL Managed instance
Hi All, I am using copy activity to copy a table from On-Prem SQL Server (Source) to Azure SQL Managed Instance (Sink). I am getting the error as…
asked 2024-06-04T07:25:57.9366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 34%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
Mohanraj Ramalingam
150
Reputation points
commented 2024-06-11T17:50:32.37+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
11,370
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Managing Salesforce Metadata Changes in ADF Pipelines
Hi there! I'm new to ADF and really curious about handling changes in Salesforce data coming into SQL Server. We mostly want to move it over as-is, but things change in Salesforce sometimes, like adding new fields or changing existing ones. Is there a…
asked 2024-01-25T01:37:45.6666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 75%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EC%3C/text%3E%3C/svg%3E)
ClarissaJacquline
200
Reputation points
commented 2024-06-11T15:45:47.5733333+00:00
Boilard, Andre
46
Reputation points
1 answer
Connecting Azure Database for PostgreSQL and Data Factory
Hello, I'd want to connect my Azure Database for PostgreSQL (from now on referred as DB) to use it as a dataset in Data Factory (from now on referred as ADF). My DB is in a private network in a resource group (rg_1) my ADF is public and is in a…
asked 2024-06-10T11:31:09.75+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(182.40000000000003, 65%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELB%3C/text%3E%3C/svg%3E)
Laura Balbi
0
Reputation points
commented 2024-06-11T09:18:58.1366667+00:00
Amira Bedhiafi
17,786
Reputation points
1 answer
unable to connect adls gen storage from Purview
unable to connect adls gen storage from Purview gettting following error ADLS Gen2 operation failed for: Storage operation '' on container 'bloblblob' get failed with 'Operation returned an invalid status code 'Forbidden''. Possible root causes: (1).…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
1 answer
How to add a GUEST user to my AZURE Portal with my Microsoft Azure Sponsorship Member account
Hello, I have a Microsoft Azure Sponsorship membership that expires in October 2024, I have just begun working on my concept with developers and an AI team. I need help getting GUEST users inside of my AZURE portal instead of inviting them to be a full…
asked 2024-06-03T14:34:02.28+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(259.20000000000005, 31%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBS%3C/text%3E%3C/svg%3E)
BOBBY SHEPPARD
0
Reputation points
edited the question 2024-06-11T07:05:45.3666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 17%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGM%3C/text%3E%3C/svg%3E)
Givary-MSFT
29,341
Reputation points Microsoft Employee
10 answers
ADF - Getting the error message of a previous activity
I have a pipeline with activities like this. When the copy activity fails, I want to use the error message of the copy activity, and set it in the error variable. How do I get the error message of the copy activity - in the set variable activity…
asked 2020-12-21T10:36:31.453+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 6%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKR%3C/text%3E%3C/svg%3E)
Kothai Ramanathan
941
Reputation points Microsoft Employee
answered 2024-06-11T06:55:28.8533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 55.00000000000001%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPR%3C/text%3E%3C/svg%3E)
Prachi Rout
0
Reputation points
0 answers
Facing pipeline failure issue while running Data flow activity
Operation on target DataFlowFactSales failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: com.microsoft.dataflow.broker.InvalidOperationException: Only one valid authentication should be…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(124.80000000000001, 28.999999999999996%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
1 answer
Automating API Connection Key Update in Azure Logic App
I'm currently working on a Logic App project where I'm utilizing an API connection for Azure Blob Storage. The authentication method for this API connection is via account key. I'm aiming to automate the process of updating the account key for this API…
asked 2024-06-10T13:48:11.14+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 88%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBA%3C/text%3E%3C/svg%3E)
Bansal, Ankit Kumar
25
Reputation points
answered 2024-06-10T19:09:03.65+00:00
Amira Bedhiafi
17,786
Reputation points
2 answers
Having error in all my ADF pipeline with: InternalServerError executing request
Hi, Our pipeline in ADF are now having issue " Hit unexpected exception, please retry. If the problem persists, please contact Azure Support". I search this issue related to internal of Microsoft. Pls help to fix it asap. Thank you!
asked 2024-06-05T11:29:35.0866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 34%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPT%3C/text%3E%3C/svg%3E)
Phan Thi Minh Tho
0
Reputation points
edited a comment 2024-06-10T17:14:54.34+00:00
Smaran Thoomu
11,370
Reputation points Microsoft Vendor
1 answer
Cannot connect to SQL Database.
Cannot connect to SQL Database. Please contact SQL server team for further support. Server: 'xxxxxxxxxxx.windows.net', Database: 'xxxxxxxxxxxxxxxx', User: 'ccccccc'. Check the linked service configuration is correct, and make sure the SQL Database…
asked 2023-06-17T05:45:50.09+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 38%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
Varun Reddy
0
Reputation points
edited the question 2024-06-10T17:06:13.7466667+00:00
BhargavaGunnam-MSFT
28,111
Reputation points Microsoft Employee