cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Azure Data Factory Blog
Options
3,446

Data Factory Increases Maximum Activities Per Pipeline to 80

Noelle_Li on Mar 29 2024 10:00 AM
This week we have doubled the limit on number of activities you may define in a pipeline, from 40 to 80. With more freed...
1,544

Action Required: Switch from Memory Optimized Data Flows in Azure Data Factory to General Purpose

Noelle_Li on Mar 25 2024 04:57 PM
Azure Data Factory Memory Optimized Data Flows will be fully retired on April 1, 2027. Going forward, all ADF Data Flows...
1,645

Continued region expansion: Azure Data Factory is generally available in two more regions

Chunhua on Jan 17 2024 10:00 AM
Azure Data Factory is now available in two new regions. 
2,491

Continued region expansion: Azure Data Factory is generally available in Poland Central

Chunhua on Oct 27 2023 12:00 PM
Azure Data Factory is now available in Poland Central
3,771

General Availability of Time to Live (TTL) for Managed Virtual Network in Azure Data Factory

lrtoyou1223 on Oct 10 2023 09:00 PM
We are excited to announce the general availability of Azure Data Factory Managed Virtual Network Time to Live (TTL).
14.1K

Metadata Driven Pipelines for Dynamic Full and Incremental Processing in Azure SQL

Marc_Bushong on Sep 28 2023 07:35 AM
With Azure SQL or Synapse, you can facilitate a dynamic ETL for incremental processing (dynamic SQL delete and insert) a...
5,078

Integer Type Available for Pipeline Variables

ChenyeCharlieZhu on Aug 17 2023 02:10 PM
We are excited to announce that integer type has been added to pipeline variables. This will significantly simplify your...
4,377

Documentation search now embedded in Azure Data Factory

Mark Kromer on Jul 13 2023 02:40 PM
Azure Data Factory has added documentation search to the embedded search bar
9,275

Comment Out Part of Pipeline

ChenyeCharlieZhu on Jul 07 2023 01:46 PM
To kick start the second half of 2023, ADF team has brought you major improvements in pipeline development and authoring...
3,881

Continued region expansion: Azure Data Factory just became generally available in Sweden Central

Chunhua on Jun 29 2023 10:37 PM
Azure Data Factory is now available in Sweden Central.
7,414

Securing outbound traffic with Azure Data Factory's outbound network rules

Abhishek Narain on Jun 12 2023 12:00 PM
The Outbound Rules feature in Azure Data Factory allows organizations to exercise granular control over outbound traffic...
6,253

Introducing optional Source settings for DelimitedText and JSON sources for top-level CDC resource

Krishnakumar_Rukmangathan on Jun 02 2023 06:00 AM
We are excited to introduce optional source configurations for Delimited source and JSON sources in top-level CDC resour...
8,600

Unroll multiple arrays in a single Flatten step in ADF

Mark Kromer on Apr 21 2023 12:48 PM
You can now easily unroll multiple arrays inside a single Flatten transformation in Azure Data Factory and Azure Synapse...
4,541

ADF private DNS zone overrides ARM DNS resolution causing ‘Not found’ error.

Sachin215 on Apr 18 2023 04:47 PM
#Issue: ADF private DNS zone overrides ARM DNS resolution causing ‘Not found’ error #Solution Steps: Note: If the custom...
5,116

Trigger ADF pipeline using Storage event trigger over private network.

Sachin215 on Apr 18 2023 12:31 PM
Project Technology: Azure function, ADF, Azure Synapse, ADLS Issue description: Customer has a strict regulatory complia...
7,348

Pipeline Logic 3: Error Handling and Try Catch

ChenyeCharlieZhu on Apr 05 2023 10:20 AM
In this series on Orchestration, we will dive deep to understand conditional executions in ADF, and build complex logic ...
6,469

Capture Changed Data From your Cosmos DB analytical store (Preview)

Mark Kromer on Mar 30 2023 09:33 AM
Making it super-easy to create efficient and fast ETL processing the cloud, Azure Data Factory has invested heavily in c...
7,484

Azure Data Factory February 2023 Monthly Update

Noelle_Li on Mar 09 2023 04:20 PM
Welcome to Azure Data Factory’s February monthly update! Here we’ll share the latest updates on what’s new in Azure Data...
20K

Process your data in seconds with new ADF real-time CDC

Mark Kromer on Mar 03 2023 01:50 PM
In January, we announced that we've elevated our Change Data Capture features front-and-center in ADF. Up until just tod...
18.7K

Introducing Dark Mode for ADF Studio

Noelle_Li on Mar 02 2023 01:57 PM
Today, we are excited to share that Dark Mode is now available in the Azure Data Factory Studio!
8,862

Best practices for Azure Data Factory Integration Runtime

lrtoyou1223 on Feb 17 2023 01:07 PM
Integration runtime is a core component of Azure Data Factory. Users can use the integration runtime created by default ...
21.2K

You can now customize the return value from your pipeline!

Mark Kromer on Feb 16 2023 10:25 AM
When building complex workflows in the cloud with Azure Data Factory and Azure Synapse Pipelines, a very common pattern ...
6,429

Azure Data Factory January 2023 Monthly Update

Noelle_Li on Feb 07 2023 02:00 PM
Welcome to Azure Data Factory’s January monthly update! Here we’ll share the latest updates on what’s new in Azure Data ...
4,779

Directly Launch Pipeline Template Gallery through the Azure Portal

Penny_Zhou on Feb 07 2023 11:22 AM
We are excited to announce an easier way to access pipeline template gallery - launch the Pipeline Template Gallery dire...
55.9K

Introducing 'Workflow Orchestration Manager' powered by Apache Airflow in Azure Data Factory

Abhishek Narain on Feb 01 2023 12:03 PM
Today, we are excited to announce the capability to run Apache Airflow DAGs (Directed Acyclic Graph) within Azure Data F...
10.4K

Orchestrate and operationalize Synapse Notebooks and Spark Job Definitions from Azure Data Factory

Abhishek Narain on Jan 25 2023 11:18 AM
Today, we are introducing support for orchestrating Synapse notebooks and Synapse spark job definitions (SJD) natively f...
6,447

Pipeline Logic 2: OR (at least 1 activity succeeded or failed)

ChenyeCharlieZhu on Jan 24 2023 01:58 PM
In this series on Orchestration, we will dive deep to understand conditional executions in ADF, and build complex logic ...
27.6K

Announcing the Public Preview of a new top-level CDC resource in ADF

Noelle_Li on Jan 23 2023 12:37 PM
Azure Data Factory (ADF) has recently added many new CDC-enabled connectors to process change data from SQL, Storage, Co...
10.4K

Pipeline Logic 1: Error Handling and Best Effort Step

ChenyeCharlieZhu on Jan 13 2023 03:45 PM
In this series on Orchestration, we will dive deep to better understand conditional executions in ADF, and build complex...
9,267

Connecting to Snowflake using Key-Pair Authentication in Azure Data Factory

sgsshankar on Jan 05 2023 11:08 AM
In this article, Shankar Narayanan describes a workaround method for connecting to Snowflake from Azure Data Factory usi...

Latest Comments

sbalakrishnanbe
in Data Factory Increases Maximum Activities Per Pipeline to 80 on May 07 2024 06:17 AM
@haochen - Thank you for your response.Please let us know when the issue has been resolved.
0 Likes
haochen
in Data Factory Increases Maximum Activities Per Pipeline to 80 on May 06 2024 08:46 PM
@sbalakrishnanbe apologize for the delay. The additional fix is rolling out, and should be available by end of this week You do not need to upgrade your NPM package. It'll work with your existing package once deployment is done.
2 Likes
PardhaV
in Configuring Private endpoint to Snowflake Private link using managed Virtual Network on May 05 2024 01:06 PM
@sgsshankar How will it function for disaster recovery? We've established a Multi-Region Environment (MPE) for Snowflake Private Link, which is operating smoothly in the eastus2 location. Now, we're considering adding a Private Link in the westus region. If we incorporate Private Links in both locat...
0 Likes
sbalakrishnanbe
in Data Factory Increases Maximum Activities Per Pipeline to 80 on May 02 2024 09:49 AM
Hi @haochen - I wanted to check in with you regarding the NPM validation issue. We are still experiencing NPM validation errors, and I was wondering if it has been resolved. Currently, we are using the following NPM package:@microsoft/azure-data-factory-utilities": "^1.0.0";Our node version is 18.0....
0 Likes
matt_gibson
in New data flow functions for dynamic, reusable patterns on Apr 30 2024 08:23 AM
I notice that the sha2() function seems to simply concatenate its values before hashing the whole result. This means that, for example, the following calls generate the same hash:sha2(256, 'ABC', 'DEF') /* e9c0f8b575cbfcb42ab3b78ecc87efa3b011d9a5d10b09fa4e96f240bf6a82f5 */ sha2(256, 'AB', 'CDEF') /*...
0 Likes