Managing Salesforce Metadata Changes in ADF Pipelines

ClarissaJacquline 200 Reputation points
2024-01-25T01:37:45.6666667+00:00

Hi there! I'm new to ADF and really curious about handling changes in Salesforce data coming into SQL Server. We mostly want to move it over as-is, but things change in Salesforce sometimes, like adding new fields or changing existing ones. Is there a magic button to handle all that automatically, especially the never-ending stream of new fields? It would save us tons of time compared to doing it all by hand. Any tips on how to deal with that in ADF would be awesome!

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,136 questions
{count} votes

Accepted answer
  1. Smaran Thoomu 6,740 Reputation points Microsoft Vendor
    2024-01-25T08:04:10.16+00:00

    Hi @ClarissaJacquline ,

    Welcome to Microsoft Q&A platform and thanks for posting your question.

    As I understand, you want to know how to handle metadata changes in an ADF pipeline, where the source is Salesforce and the target is SQL Server, without manual intervention.

    You can achieve this using the MetaData-driven copy task, as explained here. When there's a need to add new fields or make any other metadata changes, simply edit the control table. Updating the control table will generate a new SQL script for you. Just rerun the SQL script to update your control table.

    I found this video to be very helpful in explaining the process.

    Hope this helps. Do let us know if you any further queries.


    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful