Azure Data Factory Salesforce connector - export column names only

Question

Im running an export from adf to a blob storage, using the salesforce connector, where we each day send all updated rows of the databases into a blob storage. If there are no changes, no file is created. Is it possible to still create the file and send the column names if there is no data extracted?

Answer 1

You can follow these steps if you want to export only column names if no data is extracted :

1. Create a Dummy Data Source:
   • Create a dummy data source in Salesforce that contains a single record with the desired schema (column names) but no actual data. This dummy record can be excluded from the daily extracts using filters or deleted after the initial export.
2. Conditional Processing:
   • Use an If Condition activity within your pipeline to check if there are any records returned by the Salesforce query.
   • If there are records, proceed with the normal export process.
   • If no records are returned, trigger a separate branch of the pipeline that handles the creation of the file with only column headers.
3. Use a Stored Procedure or Logic App:
   • You can utilize a stored procedure (if working with SQL) or a Logic App triggered by the ADF pipeline to manually create a CSV file with only the column headers and store it in Blob Storage.

If the above methods are not feasible, you can use a Custom Activity in ADF that is written in Python or .NET to connect to Salesforce, retrieve the schema (column names), and create a CSV file with just the headers.