9,535 questions with Azure Data Factory tags

Sort by: Updated
1 answer

I can't create a Function with the free subscription

It's been 5 days now trying to create a function app on my free Microsoft Azure account but nothing is working. I have also tried to change regions but nothing is working. Please help me fix the issue.

Azure Virtual Machines
Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
7,126 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-25T06:42:30.44+00:00
brian obizi 0 Reputation points
edited a comment 2024-04-26T07:08:58.1566667+00:00
v-vvellanki-MSFT 3,580 Reputation points Microsoft Vendor
2 answers

How to refresh table with deadlock?

We Azure postgresql database build. on that we have a ADF process that is doing truncate and load from source schema to prod schema every morning. Teams are querying to prod schema continuously to pull data which causes deadlock on table. then adf…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-03T18:34:12.2933333+00:00
Grover, Jyoti 40 Reputation points
commented 2024-04-26T07:08:52.7333333+00:00
ShaikMaheer-MSFT 37,896 Reputation points Microsoft Employee
0 answers

How to perform SWOT analisys

Help me to explain and perform SWOT analisys

Microsoft Graph
Microsoft Graph
A Microsoft programmability model that exposes REST APIs and client libraries to access data on Microsoft 365 services.
10,585 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
Outlook Management
Outlook Management
Outlook: A family of Microsoft email and calendar products.Management: The act or process of organizing, handling, directing or controlling something.
4,887 questions
Visual Studio Debugging
Visual Studio Debugging
Visual Studio: A family of Microsoft suites of integrated development tools for building applications for Windows, the web and mobile devices.Debugging: The act or process of detecting, locating, and correcting logical or syntactical errors in a program or malfunctions in hardware. In hardware contexts, the term troubleshoot is the term more frequently used, especially if the problem is major.
940 questions
Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
19,466 questions
asked 2024-04-26T07:06:24.7466667+00:00
Pablo Ojeda 0 Reputation points
1 answer

Create a data factory and pipeline using .NET SDK

Hello, this is my first post here and I hope to find a solution. I am working on a web portal (.NET) hosted in Azure Web Apps and need to create a Service Link for Azure SQL Database in Azure Data Factory when I perform specific actions. The problem is…

Azure SQL Database
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-25T03:13:10.4833333+00:00
Eddy Tegera 0 Reputation points
commented 2024-04-26T06:17:30.4233333+00:00
phemanth 5,735 Reputation points Microsoft Vendor
2 answers

Issue with output file format in Azure Data Flow (Delimited text dataset returning parquet files)

I have a Data flow, where I take two sources from a sql database and sink them to a delimited text after some transformation. I am using inline sink types. This data flow works well. After I have another data flow which takes the resulting file and…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-04T19:17:24.58+00:00
Adrian Searles 0 Reputation points
commented 2024-04-26T06:10:57.8566667+00:00
PRADEEPCHEEKATLA-MSFT 77,081 Reputation points Microsoft Employee
0 answers

In ADF, I am not able to merge data from multiple streams with different columns with different data types

Hello, I am not able to merge data from multiple streams in ADF DATAFLOWS. Stream 1) flow let - only 1 column. Stream 2) flow let - only 1 column. Stream 3) source columns (54 columns) Stream 4) regular…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,365 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-24T16:32:40.4033333+00:00
neeraj aitha 0 Reputation points
commented 2024-04-26T06:02:23.4933333+00:00
phemanth 5,735 Reputation points Microsoft Vendor
1 answer

ADF Lookup activity not returning correct datetime value

A pipeline uses a lookup activity and runs a script against Snowflake, returns first row of data and we are using it in a update statement. Running the lookup to get MAX(load_date) from a snowflake table. In Snowflake it is "2024-04-25 17:26:01.548…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-26T04:24:18.2966667+00:00
Hari 21 Reputation points
commented 2024-04-26T06:02:17.4166667+00:00
AnnuKumari-MSFT 30,676 Reputation points Microsoft Employee
3 answers

Converting UTC to EST with daylight saving

I am trying to convert UTC to EST with following formula but its not taking daylight saving time to consideration. toString(fromUTC(currentUTC(),'EST'),'yyyyMMddHHmmss') it shows time one hour behind EST. Is there a workaround?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-02T14:40:10.5866667+00:00
Jiby Jacob 20 Reputation points
answered 2024-04-26T05:40:44.9833333+00:00
PRADEEPCHEEKATLA-MSFT 77,081 Reputation points Microsoft Employee
1 answer

Importing csv file with Json and data with text qualifier and column delimiter

Hi Team, I have a CSV with these details. Text qualifier: " Column delimiter: , Row delimiter: CRLF But as attached, I have 2 columns in csv file, which has json and data with text qualifier and column delimiter. I want to store this file as it…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-02T03:26:36.93+00:00
VickyD 81 Reputation points
commented 2024-04-26T05:30:04.4566667+00:00
PRADEEPCHEEKATLA-MSFT 77,081 Reputation points Microsoft Employee
1 answer

Cannot edit an existing data factory in synapse

I have a synapse workspace with several pipelines with data factories. Today I have noticed that existing data factories that are running correctly I cannot see the details when I edit them I attach two examples, one that is working and the other that I…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,365 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-22T13:25:21.7566667+00:00
Emilio Salas Roura 25 Reputation points
edited an answer 2024-04-26T05:13:35.4733333+00:00
Smaran Thoomu 9,525 Reputation points Microsoft Vendor
0 answers

Using ADF, how to dynamicaly replace excel headers blank spaces with underscores?

Hello, I'm working on an ADF pipeline whose goal is to copy an Excel file to a blob storage as a parquet file. To do so, I'm using a copy data activity. This activity needs to be as parametrized as possible, meaning that the Excel file schema is not to…

Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,427 questions
Excel
Excel
A family of Microsoft spreadsheet software with tools for analyzing, charting, and communicating data.
1,459 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-24T13:33:57.16+00:00
DIAMANT Marius 0 Reputation points
commented 2024-04-26T05:05:45.9633333+00:00
Harishga 3,260 Reputation points Microsoft Vendor
1 answer

Linked services issue which connecting to OnPrim DB2

I am trying to connect to Linked services for OnPrim DB2 using a VM as middle layer, I have checked the telnet connectivity everything is connected but when I used the selfhosted and tried to connect then I get the attached error, Can any one help to…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-25T18:49:31.19+00:00
Mallik 0 Reputation points
edited the question 2024-04-26T04:03:03.0866667+00:00
OMMI NAVEEN KUMAR 195 Reputation points Microsoft Vendor
0 answers

Azure Data Factory Old Security->Credential not clear after deploy

Hi, I was testing with the UserAssignedManagedIdentity credential and I created 2 credentials with the same UserAssignedManagedIdentity but different names(UserAssignedManagedIdentity & credentialUAMI). I ran my pipeline to deploy using the dev…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-25T12:54:35.7966667+00:00
Lee, Steven Hong Sing 1 Reputation point
commented 2024-04-26T03:44:31.8433333+00:00
Lee, Steven Hong Sing 1 Reputation point
1 answer

How to translate database content with Azure Translator by ADF or Synapse notebook?

There is an Azure Database table. Some of columns need to be translated from one language to another into additional columns. Such as from English to Spanish, or Portuguese to English, etc. I am exploring how I can use ADF or Synapse notebook to…

Azure SQL Database
Azure Translator
Azure Translator
An Azure service to easily conduct machine translation with a simple REST API call.
340 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,365 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-22T15:17:46.4133333+00:00
Dataholic 80 Reputation points
commented 2024-04-26T00:22:11.5966667+00:00
BhargavaGunnam-MSFT 26,136 Reputation points Microsoft Employee
0 answers

ADF dataflow creating temporary folder

Hello everyone, I'm currently generating a file using a data flow, and I've noticed that a temporary file is being created in a directory called _temporary/0/. I have another pipeline that triggers upon the creation of the file mentioned above, and it…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-24T06:41:20.44+00:00
Jaganath Kumar 105 Reputation points
edited a comment 2024-04-25T23:58:02.9833333+00:00
Jaganath Kumar 105 Reputation points
1 answer

How to parameterize pipeline concurrency (number of parallel runs) in Azure Data Factory

I have two pipelines: parent pipeline to read configuration file (lookup activity), filter it with filter activity and pass the resulting array (e.g. 50 items) to ForEach activity to run child pipeline to perform per-item work in a predefined number of…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-25T11:49:35.2833333+00:00
Azurin4848484 0 Reputation points
answered 2024-04-25T20:32:26.5033333+00:00
Amira Bedhiafi 15,216 Reputation points
1 answer

Trying to write an expression for a filter activity in Azure Data Factory

I currently am using 3 filter activities where I am using the following condition @contains(item().name,'example') to filter out a certain group of files from the array obtained by get metadata This works fine but is still bringing back 5 files with the…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-25T15:31:35.3566667+00:00
azure-000 1 Reputation point
answered 2024-04-25T19:53:54.15+00:00
Amira Bedhiafi 15,216 Reputation points
0 answers

How to pass error details from Snowflake stored procedure to ADF script activity using the new Snowflake connector?

Since switching to the new Snowflake connector in ADF, error messages from Snowflake stored procedures don't seem to be being passed back to the calling Script activity in ADF as before. Is this intended behaviour, e.g. is there a recommended process for…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-25T10:54:06.9366667+00:00
Tom Dann 0 Reputation points
commented 2024-04-25T19:47:06.6266667+00:00
AnnuKumari-MSFT 30,676 Reputation points Microsoft Employee
1 answer

Copy Activity SFTP sometimes copies files and sometimes does not

I run once a day a pipeline that has a 'Get Metadata' returning Child items and I filter the result to CSV type. The result is then past to a For Each which has a Copy data task and for each file @item().name it picks up the file from an SFTP and puts…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-25T07:48:47.2866667+00:00
Jay 536 Reputation points
answered 2024-04-25T19:37:07.2966667+00:00
Amira Bedhiafi 15,216 Reputation points
1 answer

ADF Power Queries Suddenly not completing - Region Australia East

Hi Team We suddenly have ADF power Query steps not completing and in a hung state. This started happening On Sunday the 21st of April 2024. I have deployed all these pipelines in the Australia East region. It is across all Pipelines where a Power Query…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,535 questions
asked 2024-04-21T22:55:02.6933333+00:00
Manura De Silva 10 Reputation points
commented 2024-04-25T18:54:39.0233333+00:00
Smaran Thoomu 9,525 Reputation points Microsoft Vendor