What's new in Azure Data Factory
Azure Data Factory is improved on an ongoing basis. To stay up to date with the most recent developments, this article provides you with information about:
- The latest releases.
- Known issues.
- Bug fixes.
- Deprecated functionality.
- Plans for changes.
This page is updated monthly, so revisit it regularly. For older months' updates, refer to the What's new archive.
Check out our What's New video archive for all of our monthly update videos.
December 2022
Data flow
SQL change data capture (CDC) incremental extract - supports numeric columns in mapping dataflow Learn more
Data movement
Express virtual network injection for SSIS in Azure Data Factory is generally available Learn more
Region expansion
Continued region expansion - Azure Data Factory is now available in China North 3 Learn more
November 2022
Data flow
- Incremental only is available in SAP CDC - get changes only from SAP system without initial full load Learn more
- Source partitions in initial full data load of SAP CDC to improve performance Learn more
- A new pipeline template - Load multiple objects with big amounts from SAP via SAP CDC Learn more
Data Movement
- Support to Azure Databricks through private link from a Data Factory managed virtual network Learn more
User Interface
3 Pipeline designer enhancements added to ADF Studio preview experience
- Dynamic content flyout - make it easier to set dynamic content in your pipeline activities without using the expression builder Learn more
- Error message relocation to status column - make it easier for you to view errors when you see a Failed pipeline run Learn more
- Container view - in Author Tab, Pipeline can change output view from list to container Learn more
Continuous integration and continuous deployment
In auto publish config, disable publish button is available to void overwriting the last automated publish deployment Learn more
October 2022
Video summary
Data flow
- Export up to 1000 rows from data flow preview Learn more
- SQL CDC in Mapping Data Flows now available (Public Preview) Learn more
- Unlock advanced analytics with Microsoft 365 Mapping Data Flow Connector Learn more
- SAP Change Data Capture (CDC) in now generally available Learn more
Developer productivity
- Now accepting community contributions to Template Gallery Learn more
- New design in Azure portal – easily discover how to launch ADF Studio Learn more
- Learning Center now available in the Azure Data Factory studio Learn more
- One-click to try Azure Data Factory Learn more
Orchestration
- Granular billing view available for ADF – see detailed billing information by pipeline (Public Preview) Learn more
- Script activity execution timeout now configurable Learn more
Region expansion
Continued region expansion – Qatar Central now supported Learn more
Continuous integration and continuous deployment
Exclude pipeline triggers that did not change in deployment now generally available Learn more
September 2022
Video summary
Data flow
- Amazon S3 source connector added Learn more
- Google Sheets REST-based connector added as Source (Preview) Learn more
- Maximum column optimization in dataflow Learn more
- SAP Change Data Capture capabilities in Mapping Data Flow (Preview) - Extract and transform data changes from SAP systems for a more efficient data refresh Learn more
- Writing data to a lookup field via alternative keys supported in Dynamics 365/CRM connectors for mapping data flows Learn more
Data movement
Support to convert Oracle NUMBER type to corresponding integer in source Learn more
Monitoring
- Additional monitoring improvements in Azure Data Factory Learn more
- Monitoring loading improvements - pipeline re-run groupings data fetched only when expanded
- Pagination added to pipeline activity runs view to show all activity records in pipeline run
- Monitoring consumption improvement – loading icon added to know when consumption report is fully calculated
- Additional sorting columns in monitoring – sorting added for Pipeline name, Run End, and Status
- Time-zone settings now saved in monitoring
- Gantt chart view now supported in IR monitoring Learn more
Orchestration
DELETE method in the Web activity now supports sending a body with HTTP request Learn more
User interface
- Native UI support of parameterization added for 6 additional linked services – SAP ODP, ODBC, Microsoft Access, Informix, Snowflake, and DB2 Learn more
- Pipeline designer enhancements added in Studio Preview experience – users can view workflow inside pipeline objects like For Each, If Then, etc. Learn more
August 2022
Video summary
Data flow
- Appfigures connector added as Source (Preview) Learn more
- Cast transformation added – visually convert data types Learn more
- New UI for inline datasets - categories added to easily find data sources Learn more
Data movement
Service principal authentication type added for Azure Blob storage Learn more
Developer productivity
- Default activity time-out changed from 7 days to 12 hours Learn more
- New data factory creation experience - one click to have your factory ready within seconds Learn more
- Expression builder UI update – categorical tabs added for easier use Learn more
Continuous integration and continuous delivery (CI/CD)
When CI/CD integrating ARM template, instead of turning off all triggers, it can exclude triggers that didn't change in deployment Learn more
July 2022
Video summary
Data flow
- Asana connector added as source Learn more
- Three new data transformation functions now supported Learn more
- collectUnique() - Create a new collection of unique values in an array.
- substringIndex() - Extract the substring before n occurrences of a delimiter.
- topN() - Return the top n results after sorting your data.
- Refetch from source available in Refresh for data source change scenarios Learn more
- User defined functions (GA) - Create reusable and customized expressions to avoid building complex logic over and over Learn more Video
- Easier configuration on data flow runtime - choose compute size among Small, Medium and Large to pre-configure all integration runtime settings Learn more
Continuous integration and continuous delivery (CI/CD)
Include Global parameters supported in ARM template. Learn more
Developer productivity
Be a part of Azure Data Factory studio preview features - Experience the latest Azure Data Factory capabilities and be the first to share your feedback Learn more
More information
Feedback
Submit and view feedback for