SQL Pools missing in the data flows source settings
Hi, I have created a couple dedicated sql pools within the synapse workspace with a view to load data from one poll to the other. However, the sql pools doesn't show in the source type (inline). Can you help please? I wonder if I need to add an extension…
scheduled trigger should be disabled automatically once activity is done
Hi , I have ADF pipeline inside that I used stored procedure activity and call stored proc. inside SP I checked DTU percentage , if DTU percentage is less than 5% then create index. That pipeline is scheduled every 15 min but once my DTU condition is…
ADF: Metadata Activity Output - Order/Sort By option
For the ADF: Metadata Activity Output is there a set order, or an option to Order/Sort By ? I need to FTP XML files source to sink in the order of file name i.e. diff_inventory_2020-12-09_10-15-01.xml diff_inventory_2020-12-09_10-18-01.xml …
ADF Error code 133
In ADF, I am trying to debug a pipeline but all the debug runs are throwing the following error: Error code: 133 Failure type: System error Details: InternalServerError:An error occurred while executing the activity:…
Azure Datafactory: Getting "Value cannot be null. Parameter name: dictionary" error
I've set up a pipeline in Azure Datafactory that reads a configuration file from a storage account, then runs a different pipeline for each configuration item found. This pipeline ran perfectly last week. Now, however, the pipeline errors out on the…
![](https://techprofile.blob.core.windows.net/images/VE145SRL20aYBPaIw9DU5w.png?8D81D8)
![](https://techprofile.blob.core.windows.net/images/VE145SRL20aYBPaIw9DU5w.png?8D81D8)
ADF performance troubleshooting
For very basic activities (eg lookup) I'm finding large differences in performance between one execution of a pipeline and the next. For example the following sometimes runs in 3 seconds and other times runs in 20 seconds. What can I do to…
Bug in ADF CSV Escaping
I have exported some data from SQL Server into a CSV file (copyData) The actual text is: leading text \"Jane Doe\" trailing text It comes into the CSV file as: "leading text \"Jane Doe\" trailing text" Which fails on input…
Amazon MWS - Throttling Limits
Hi Team, I can see direct connector available for "Amazon Marketplace Web Service" with option to select tables. But I couldn't see anywhere about throttling limit to get the data from those tables. But those limits are part of API calls.…
![](https://techprofile.blob.core.windows.net/images/FBXTxf5_AwAAAAAAAAAAAA.png?8DCAA2)
CopyActivity not utilising high performance
Hi, we are loading data from SQL SERVER to Azure SQLDB I have triggered 4 tables parallel, after some time 3 tables loaded and 4th table is still running which have more records and at the same time, some tables loaded quickly with high…
How to configuring timings in trigger to run pipeline twice in a day
Hi All, I have a requirement to run a pipeline twice in a day at 8:15 AM UTC and 9:45 AM UTC. How to proceed with this scenario?
copy high volume on prem data to blob
Hi , I am copying data from on prem db Netezza to azure blob storage using ADF copy. But few of the on prem tables are having huge data like 1 trillion rows. Can you please advice the best way to approach for this high volume of data to copy into…
![](https://techprofile.blob.core.windows.net/images/VfQFAmOikEWfBHko2XlWTA.png?8D7F33)
Connect from local SQL database inot Azure Data Factory
Hi. How can I connect a local SQL database into a Azure Data Factory? In my project, It is necessary to have the updated data in Azure storage, once a day. Does Copy Moduel in Data Factory work in this case? Or is it just an offline action (one…
Using a Variable in a Web Activity Body? URGENT PLS
Hello! I have the following as part of my ADF Web Activity Body. My question is, can I do this? "XRefCode": "@variables('Payclass')" "XRefCode": "@{item().employeeType}", "ShortName":…
SHIR Registration in Gov Cloud
I'm attempting to deploy a SHIR via on-prem VMWare VM to connect to an ADF instance in USGovTX region (Gov cloud) During the registration process I'm receiveing the following error in Event Log after clicking "Finish" …
ADF reading json array of documents in Data flow
Hi, In Data flow object I'd like to read data from json files from blob storage (created in a separate pipeline), but when I try to click "Import projection" in "Projection" the result is only one column _corrupt_record: …
How to query 3rd party Azure DataLake Gen2 and only store the results
First, what I am trying to do is I want to query and aggregate raw JSON files stored in a 3rd party's Azure Data Lake (Gen2) and store those aggregates in my own data lake or relation db. I do not want to physically copy all of those raw JSON files…
Linkedin connectivity with Azure
Hi, Is there a way to extract existing content from a Linkedin company page using Azure? Thanks
Resume Azure Data Base from Azure Factory pipeline
Hi, I have an Azure data base which gets paused automatically after 1 hour. I am using this database as a dataset in my Azure data factory pipeline. I have a source inside a data flow which gets failed because my database is in paused state. I do not…
Error when Object Type parameter is passed through another pipeline
Hi All, I have a pipeline with object type parameter that is used to parameterize Additional columns property in Copy activity. The value of the object parameter is:…
ADF @concat giving extra brackets and \....why? Please Help
HI! All I am trying to do is in a VARIABLE concat the fixed url string value with the employee number and I am getting these brackets and backslashes as shown. the employee number is an array that I'm using an ForEach Activity. Any idea why I am getting…