I can't create a Function with the free subscription
It's been 5 days now trying to create a function app on my free Microsoft Azure account but nothing is working. I have also tried to change regions but nothing is working. Please help me fix the issue.
How to refresh table with deadlock?
We Azure postgresql database build. on that we have a ADF process that is doing truncate and load from source schema to prod schema every morning. Teams are querying to prod schema continuously to pull data which causes deadlock on table. then adf…
How to perform SWOT analisys
Help me to explain and perform SWOT analisys
Create a data factory and pipeline using .NET SDK
Hello, this is my first post here and I hope to find a solution. I am working on a web portal (.NET) hosted in Azure Web Apps and need to create a Service Link for Azure SQL Database in Azure Data Factory when I perform specific actions. The problem is…
Issue with output file format in Azure Data Flow (Delimited text dataset returning parquet files)
I have a Data flow, where I take two sources from a sql database and sink them to a delimited text after some transformation. I am using inline sink types. This data flow works well. After I have another data flow which takes the resulting file and…
In ADF, I am not able to merge data from multiple streams with different columns with different data types
Hello, I am not able to merge data from multiple streams in ADF DATAFLOWS. Stream 1) flow let - only 1 column. Stream 2) flow let - only 1 column. Stream 3) source columns (54 columns) Stream 4) regular…
ADF Lookup activity not returning correct datetime value
A pipeline uses a lookup activity and runs a script against Snowflake, returns first row of data and we are using it in a update statement. Running the lookup to get MAX(load_date) from a snowflake table. In Snowflake it is "2024-04-25 17:26:01.548…
Converting UTC to EST with daylight saving
I am trying to convert UTC to EST with following formula but its not taking daylight saving time to consideration. toString(fromUTC(currentUTC(),'EST'),'yyyyMMddHHmmss') it shows time one hour behind EST. Is there a workaround?
Importing csv file with Json and data with text qualifier and column delimiter
Hi Team, I have a CSV with these details. Text qualifier: " Column delimiter: , Row delimiter: CRLF But as attached, I have 2 columns in csv file, which has json and data with text qualifier and column delimiter. I want to store this file as it…
Cannot edit an existing data factory in synapse
I have a synapse workspace with several pipelines with data factories. Today I have noticed that existing data factories that are running correctly I cannot see the details when I edit them I attach two examples, one that is working and the other that I…
Using ADF, how to dynamicaly replace excel headers blank spaces with underscores?
Hello, I'm working on an ADF pipeline whose goal is to copy an Excel file to a blob storage as a parquet file. To do so, I'm using a copy data activity. This activity needs to be as parametrized as possible, meaning that the Excel file schema is not to…
Linked services issue which connecting to OnPrim DB2
I am trying to connect to Linked services for OnPrim DB2 using a VM as middle layer, I have checked the telnet connectivity everything is connected but when I used the selfhosted and tried to connect then I get the attached error, Can any one help to…
Azure Data Factory Old Security->Credential not clear after deploy
Hi, I was testing with the UserAssignedManagedIdentity credential and I created 2 credentials with the same UserAssignedManagedIdentity but different names(UserAssignedManagedIdentity & credentialUAMI). I ran my pipeline to deploy using the dev…
How to translate database content with Azure Translator by ADF or Synapse notebook?
There is an Azure Database table. Some of columns need to be translated from one language to another into additional columns. Such as from English to Spanish, or Portuguese to English, etc. I am exploring how I can use ADF or Synapse notebook to…
ADF dataflow creating temporary folder
Hello everyone, I'm currently generating a file using a data flow, and I've noticed that a temporary file is being created in a directory called _temporary/0/. I have another pipeline that triggers upon the creation of the file mentioned above, and it…
How to parameterize pipeline concurrency (number of parallel runs) in Azure Data Factory
I have two pipelines: parent pipeline to read configuration file (lookup activity), filter it with filter activity and pass the resulting array (e.g. 50 items) to ForEach activity to run child pipeline to perform per-item work in a predefined number of…
Trying to write an expression for a filter activity in Azure Data Factory
I currently am using 3 filter activities where I am using the following condition @contains(item().name,'example') to filter out a certain group of files from the array obtained by get metadata This works fine but is still bringing back 5 files with the…
How to pass error details from Snowflake stored procedure to ADF script activity using the new Snowflake connector?
Since switching to the new Snowflake connector in ADF, error messages from Snowflake stored procedures don't seem to be being passed back to the calling Script activity in ADF as before. Is this intended behaviour, e.g. is there a recommended process for…
Copy Activity SFTP sometimes copies files and sometimes does not
I run once a day a pipeline that has a 'Get Metadata' returning Child items and I filter the result to CSV type. The result is then past to a For Each which has a Copy data task and for each file @item().name it picks up the file from an SFTP and puts…
ADF Power Queries Suddenly not completing - Region Australia East
Hi Team We suddenly have ADF power Query steps not completing and in a hung state. This started happening On Sunday the 21st of April 2024. I have deployed all these pipelines in the Australia East region. It is across all Pipelines where a Power Query…