Azure Data Factory Old Security->Credential not clear after deploy
Hi, I was testing with the UserAssignedManagedIdentity credential and I created 2 credentials with the same UserAssignedManagedIdentity but different names(UserAssignedManagedIdentity & credentialUAMI). I ran my pipeline to deploy using the dev…
I can't create a Function with the free subscription
It's been 5 days now trying to create a function app on my free Microsoft Azure account but nothing is working. I have also tried to change regions but nothing is working. Please help me fix the issue.
Data Factory Pipeline, I want to Publish my trigger that I have created through a Http request, is this at all possible.
Hello, I have created a Data Factory Pipeline with help from the below links using a HTTP Rest API call. https://learn.microsoft.com/en-us/rest/api/datafactory/pipelines/create-or-update?view=rest-datafactory-2018-06-01&tabs=HTTP#code-try-0 …
How to parameterize pipeline concurrency (number of parallel runs) in Azure Data Factory
I have two pipelines: parent pipeline to read configuration file (lookup activity), filter it with filter activity and pass the resulting array (e.g. 50 items) to ForEach activity to run child pipeline to perform per-item work in a predefined number of…
AZ104 - 2024 Renewal timer issue and access to a test page
Hello there, I am trying to renew the AZ 104 certification but I am encountering few issues that let me think that something is wrong with the the page. When I start the renewal process I can access the test page where one after another I can answer to…
How to pass error details from Snowflake stored procedure to ADF script activity using the new Snowflake connector?
Since switching to the new Snowflake connector in ADF, error messages from Snowflake stored procedures don't seem to be being passed back to the calling Script activity in ADF as before. Is this intended behaviour, e.g. is there a recommended process for…
Copy Activity SFTP sometimes copies files and sometimes does not
I run once a day a pipeline that has a 'Get Metadata' returning Child items and I filter the result to CSV type. The result is then past to a For Each which has a Copy data task and for each file @item().name it picks up the file from an SFTP and puts…
Create a data factory and pipeline using .NET SDK
Hello, this is my first post here and I hope to find a solution. I am working on a web portal (.NET) hosted in Azure Web Apps and need to create a Service Link for Azure SQL Database in Azure Data Factory when I perform specific actions. The problem is…
How to use AWS Databricks note book in ADF..?
Hi Team, I want to create Linked service in ADF with AWS Databricks notebook. Can you guys help me with list of steps that i need follow. Regards, Naveen.
How to add Auth Headers in REST Linked Service in Azure Data Factory (ADF) when using oAuth2 Client Credentials?
My API provider requires an API-Key additional header when using oAuth2 Client Credential to generate access token. In Azure DataFactory, when I create a REST linked service using OAuth2 Client Credential as authentication type, I also include Auth…
I am getting a BadRequest error while running adf pipeline which contains mapping data flow activity
Hi, I am getting BadRequest error while running adf pipeline which contains data flow activity. It was running fine previously.
In ADF, I am not able to merge data from multiple streams with different columns with different data types
Hello, I am not able to merge data from multiple streams in ADF DATAFLOWS. Stream 1) flow let - only 1 column. Stream 2) flow let - only 1 column. Stream 3) source columns (54 columns) Stream 4) regular…
How do I copy data from a public sharepoint URL to Microsoft Blob Storage?
Hi everyone! I am new to using Azure services. My co-author at Stanford has uploaded several files (in csv format) to a public link. This link starts as: office365stanford-my.sharepoint.com/personal It has 40 files, ranging from a few 100 MB to 100 GB. I…
Copy Data activity fails when copying from ADLS to ADLS
I'm currently attempting to use a Copy Data activity to copy a .json file over from what we treat as a "staging area" of our data over to our Datalake all within Azure Datalake Storage. The files start out in .json format but are copied over…
Using ADF, how to dynamicaly replace excel headers blank spaces with underscores?
Hello, I'm working on an ADF pipeline whose goal is to copy an Excel file to a blob storage as a parquet file. To do so, I'm using a copy data activity. This activity needs to be as parametrized as possible, meaning that the Excel file schema is not to…
Cannot edit an existing data factory in synapse
I have a synapse workspace with several pipelines with data factories. Today I have noticed that existing data factories that are running correctly I cannot see the details when I edit them I attach two examples, one that is working and the other that I…
Power Query activity is executing but not returning and keeps loading forever
Hi, Power query activity was working fine until last Thursday (April 18th), now whenever I run it whether by debug or manual trigger, it keeps loading forever. The job itself is executed and the sink files are generated in Blob storage but it keeps…
Sharepoint
how do you read from sharepoint
Retrieve SQL Broker Messages - On Prem
Hi, I want to read sql broker messages from on-prem sql server and process and send back to another API. What is the best solutions to achieve this? Can Azure data factory help on this?
Insufficient memory when running script/stored procedure activity using synapse
So serverless sql have a limit memory of data processed? I got the error message like this: Operation on target Stored procedure1 failed: Execution fail against sql server. Please contact SQL Server team if you need further support. Sql error number:…