Accelerate time to insights with data in Delta Lake format

Important

Some of the functionality described in this release plan has not been released. Delivery timelines may change and projected functionality may not be released (see Microsoft policy). Learn more: What's new and planned

Enabled for	Public preview	General availability
Admins, makers, marketers, or analysts, automatically	Aug 2024	To be announced

Business value

Your customer data updates constantly, with rapidly changing signals like online activities and mobile interactions. When a traditional data warehouse is used to generate customer insights, this leads to increased time to insight with each update while the volume of unprocessed signals keeps growing. Longer queues result in slow or stale insights. With native support for Delta Lake storage format in Customer Insights, you can now accelerate processing times even with a higher volume of fast-changing data updates. Get customer profiles and associated insights updated more frequently and react to customer engagements based on the most current information available. This empowers you to create the most relevant and personalized experiences for your customers and the most up-to-date and timely insights for your business users.

Feature details

One of the key values of the Delta Lake format is the ability to track changes over time to the data. With native support for the Delta Lake table format in Customer Insights, you can now take advantage of Delta Lake incremental data updates to accelerate processing in Customer Insights - Data.

It is typical for customer profile data to only change by a few percent each day. Instead of reprocessing the entire data set every day, Customer Insights - Data can process just the changed source data for unification in the Delta format, saving time and getting you results faster. This improvement happens automatically when all the tables are in the Delta format. No configuration or change is required.

Incremental processing is also implemented for unification, segments, measures, activities, and predictions.

Note

In this initial release, incremental unification requires that all source tables involved in the unification process use the Delta format. Any tables ingested to Customer Insights - Data using the Delta Lake connector will be in the Delta format. Starting in February, tables imported using the Dataverse connector will also be in the Delta format.