Content
Azure Data Factory Rest Linked service returns Array og Json objects instead of Set of Json Objects
I have set up a Linked Service of type "REST" to copy data from an API to Blob storage. I have a copy activity, which uses a dataset from the Linked Service and a Json dataset on top of a Blob storage Linked service. The API that I am using,…


How to read .xls file with content type application/octet-stream using copy activity
Hi Team, I am not able to copy .xls file which having content type as application/octet-stream. Can you please help me to resolve this issue without changing file extension.


Dataflow Error: The spark job of Dataflow completed, but the runtime state is either null or still InProgress.
Hi Microsoft Team, I have this error whenever I run with "Debug" a pipeline that contains a Data Flow. I tried to create new pipeline, new Data Flow but still the same error. It can be reproduced all the time. My last Pipeline Run ID is:…


DAG runs stuck in running state using Azure Data Factory Managed Airflow (version 2.4.3)
I'm following Microsoft's tutorial on how does managed airflow work using the tutorial.py script referenced in the documentation (see code block below). I've set up my airflow environment in azure data factory using the same configuration in the…


How to handle pagination with $skipToken in Azure Data Factory
I'm curious how to handle pagination with $skipToken in Azure Data Factory According to the documentation (Pagination support) it seems quite simple in case when we have @odata.nextLink to use as next URL (Microsoft Graph API), but it's more complicated…


ADF Rest API - How To Handle "too many requests" error (429)
Hello, I am struggling with integrating API source into Azure SQL DB using Azure Data Factory. Problem is that I am getting "too many requests" error (429) randomly when trying to get data from API source. It looks like error is caused by some…


SFTP Error of Host Not Found in Azure Data Factory
Hello Team, Greetings of the Day!!! Getting error while connecting to SFTP in Azure Data Factory. Doing Azure Data Factory HOL Lab from GitHub :-…


how to dynamically transform one activity output and feed in another activity in ADF
If I directly give (@{item().id} @{item().col1} @{item().col2} @{item().name} @{item().col5}) in set variable is gives back result "C0004\t1000\t\tAT020\t1102" but I give the activity output (@activity('Lookup_DB_column…


String was not recognized as a valid DateTime
Hi team, The data in my excel is in format dd-MM-yyyy but while copying to azure postgressql database, I am getting error saying that "String was not recognized as a valid DateTime". Could you please help me on this asap


Pulling service now data into azure blob using Azure Service Now connector through ADF pipeline
I am trying to pull entire incident table history from service now. Using following items in my ADF Pipeline. Service Now ADF Connector. Using basic Service Now authentication and tested the connection, it works Able to preview the records…


Azure Data Factory - Smartsheet Linked service errors
The new Smartsheet linked service in Azure data Factory has some errors and I couldn't find any reliable solutions. If there is one please let me know. below are the errors I found: This linked service doesn't work when using it with Azure Key…


ADF's File Exists Trigger does reads the files but doesn't fire off Switch activity in pipeline, any idea?
So our ADF V2 File exists trigger we setup was able read the input files but doesn't fire off the linked Switch activity. Specifically, in our case, it correctly read the file "Mashgin_Order_Items_20221001.csv" from the Azure Blob storage…
Azure Data Factory (ADF) Escape Character Issue When Quoting
I've got a pipeline built in Azure Data Factory that's pulling data from SQL and storing it as csv. My Dataset for the csv has Escape Character set to a blackslash (\) and Quote Character set to Double Quote ("). The problem this causes is when I…


Azure Data Factory Copy Data From ADLS Gen 2 to Synapse failed
Hello, I am trying to copy data from ADLS Gen 2 to Synapse, but encounter the error "An error occurred while sending the request". Message is "The remote server returned an error: (503) Server Unavailable". The synapse is using…


Can't import Live resources to GitHub Repo
I have an existing GitHub repository and I want to import resources from my ADF Live environment to the repo. I can connect to it fine and browse the repos in my account, but when I click Import I get this error. You don't have permission to import…


Issue with ADF Pipeline loosing Connectivity Refreshing Analaysis services Model
Erroor Recieved - Operation on target ThrowErrorOnFailure failed: Error calling the endpoint ''. Response status code: 'NA - Unknown'. More details: Exception message: 'NA - Unknown [ClientSideException] Invalid URI: The URI scheme is not valid.'. …


ADF Dataflows flatten a JSON using unroll by not working.
Hi I have a JSON like this in my file and i am trying t flatten it using the flatten transformation in dataflow. when i click on unroll by all the elements are displayed but i am unable to select any.. how can i do that .? or is there any other way i…


Error code 3608: 'Invoking Azure function failed with HttpStatusCode - InternalServerError.'
Dear Team, Pipelines are getting failure because of this Error associated with Azure Function. Please assist me with your solution.


ADF - Run an activity if any one of a set of activities fail and capture failed activity details in that activity
Currently I am running one generic activity when any of the activities in the pipeline fail based on the answer provided in https://learn.microsoft.com/en-us/answers/questions/208812/adf-run-an-activity-if-any-one-of-a-set-of-activit But I want to…


Nested Execute Pipeline issue in ADF master pipeline
Hi ,In my case i have a master pipeline which has nested Execute pipelines like execute1 pipeline contains lookup and foreach inside foreach execute2 .inside execute 2 get metadata and and foreach again inside foreach execute3 .execute3 contains lookup…

