Edit

What's new in Azure Data Factory

  • Article
  • 4 minutes to read

Azure Data Factory is improved on an ongoing basis. To stay up to date with the most recent developments, this article provides you with information about:

  • The latest releases.
  • Known issues.
  • Bug fixes.
  • Deprecated functionality.
  • Plans for changes.

This page is updated monthly, so revisit it regularly. For older months' updates, refer to the What's new archive.

Check out our What's New video archive for all of our monthly update videos.

December 2022

Data flow

SQL change data capture (CDC) incremental extract - supports numeric columns in mapping dataflow Learn more

Data movement

Express virtual network injection for SSIS in Azure Data Factory is generally available Learn more

Region expansion

Continued region expansion - Azure Data Factory is now available in China North 3 Learn more

November 2022

Data flow

  • Incremental only is available in SAP CDC - get changes only from SAP system without initial full load Learn more
  • Source partitions in initial full data load of SAP CDC to improve performance Learn more
  • A new pipeline template - Load multiple objects with big amounts from SAP via SAP CDC Learn more

Data Movement

  • Support to Azure Databricks through private link from a Data Factory managed virtual network Learn more

User Interface

3 Pipeline designer enhancements added to ADF Studio preview experience

  • Dynamic content flyout - make it easier to set dynamic content in your pipeline activities without using the expression builder Learn more
  • Error message relocation to status column - make it easier for you to view errors when you see a Failed pipeline run Learn more
  • Container view - in Author Tab, Pipeline can change output view from list to container Learn more

Continuous integration and continuous deployment

In auto publish config, disable publish button is available to void overwriting the last automated publish deployment Learn more

October 2022

Video summary

Data flow

  • Export up to 1000 rows from data flow preview Learn more
  • SQL CDC in Mapping Data Flows now available (Public Preview) Learn more
  • Unlock advanced analytics with Microsoft 365 Mapping Data Flow Connector Learn more
  • SAP Change Data Capture (CDC) in now generally available Learn more

Developer productivity

  • Now accepting community contributions to Template Gallery Learn more
  • New design in Azure portal – easily discover how to launch ADF Studio Learn more
  • Learning Center now available in the Azure Data Factory studio Learn more
  • One-click to try Azure Data Factory Learn more

Orchestration

  • Granular billing view available for ADF – see detailed billing information by pipeline (Public Preview) Learn more
  • Script activity execution timeout now configurable Learn more

Region expansion

Continued region expansion – Qatar Central now supported Learn more

Continuous integration and continuous deployment

Exclude pipeline triggers that did not change in deployment now generally available Learn more

September 2022

Video summary

Data flow

  • Amazon S3 source connector added Learn more
  • Google Sheets REST-based connector added as Source (Preview) Learn more
  • Maximum column optimization in dataflow Learn more
  • SAP Change Data Capture capabilities in Mapping Data Flow (Preview) - Extract and transform data changes from SAP systems for a more efficient data refresh Learn more
  • Writing data to a lookup field via alternative keys supported in Dynamics 365/CRM connectors for mapping data flows Learn more

Data movement

Support to convert Oracle NUMBER type to corresponding integer in source Learn more

Monitoring

  • Additional monitoring improvements in Azure Data Factory Learn more
    • Monitoring loading improvements - pipeline re-run groupings data fetched only when expanded
    • Pagination added to pipeline activity runs view to show all activity records in pipeline run
    • Monitoring consumption improvement – loading icon added to know when consumption report is fully calculated
    • Additional sorting columns in monitoring – sorting added for Pipeline name, Run End, and Status
    • Time-zone settings now saved in monitoring
  • Gantt chart view now supported in IR monitoring Learn more

Orchestration

DELETE method in the Web activity now supports sending a body with HTTP request Learn more

User interface

  • Native UI support of parameterization added for 6 additional linked services – SAP ODP, ODBC, Microsoft Access, Informix, Snowflake, and DB2 Learn more
  • Pipeline designer enhancements added in Studio Preview experience – users can view workflow inside pipeline objects like For Each, If Then, etc. Learn more

August 2022

Video summary

Data flow

  • Appfigures connector added as Source (Preview) Learn more
  • Cast transformation added – visually convert data types Learn more
  • New UI for inline datasets - categories added to easily find data sources Learn more

Data movement

Service principal authentication type added for Azure Blob storage Learn more

Developer productivity

  • Default activity time-out changed from 7 days to 12 hours Learn more
  • New data factory creation experience - one click to have your factory ready within seconds Learn more
  • Expression builder UI update – categorical tabs added for easier use Learn more

Continuous integration and continuous delivery (CI/CD)

When CI/CD integrating ARM template, instead of turning off all triggers, it can exclude triggers that didn't change in deployment Learn more

July 2022

Video summary

Data flow

  • Asana connector added as source Learn more
  • Three new data transformation functions now supported Learn more
    • collectUnique() - Create a new collection of unique values in an array.
    • substringIndex() - Extract the substring before n occurrences of a delimiter.
    • topN() - Return the top n results after sorting your data.
  • Refetch from source available in Refresh for data source change scenarios Learn more
  • User defined functions (GA) - Create reusable and customized expressions to avoid building complex logic over and over Learn more Video
  • Easier configuration on data flow runtime - choose compute size among Small, Medium and Large to pre-configure all integration runtime settings Learn more

Continuous integration and continuous delivery (CI/CD)

Include Global parameters supported in ARM template. Learn more

Developer productivity

Be a part of Azure Data Factory studio preview features - Experience the latest Azure Data Factory capabilities and be the first to share your feedback Learn more

More information