question

LiuVeronicaSiqi-2329 avatar image
0 Votes"
LiuVeronicaSiqi-2329 asked ShaikMaheer-MSFT answered

Error Handling when copy data to Salesforce in Azure Data Factory

Dear Sir,

I have a question about error handling during Copy data to Salesforce.

The scenario is that I use Copy data Activity to load data into Salesforce, and I get some error message from Salesforce responded in the debug output. But it seems the output content is not a full response from Salesforce.

As we know Salesforce Copy data Activity is based on the REST/Bulk API, it can choose automatically.
I want to catch all the error message and error lines which responded from Salesforce Bulk API, and transform it to a csv, then save to Blob or write into data base. How can I implement this via Azure data factory?

azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

ShaikMaheer-MSFT avatar image
0 Votes"
ShaikMaheer-MSFT answered

Hi @LiuVeronicaSiqi-2329 ,

Thank you for posting query in Microsoft Q&A Platform.

As you rightly pointed Salesforce connector is built on top of the Salesforce REST/Bulk API. Based on data size service automatically pics either REST or Bulk API. If data size is huge then it will choose Bulk API for better performance. The API version to choose can be controlled from apiVersion option in linked service. Click here to know more about salesforce connector.

Usually any error you get in ADF which using copy activity or any other activity can be seen in output JSON of activity execution. You can consider taking required details from output json of copy activity and then passing them to stored procedure to store in a table or storing them in to csv file according to your requirement.

Below few resources or videos which helps.

If you want to log error details to csv file then use another copy activity on failure of main copy activity. and In source use some dummy file and add additional column under source tab. For this addition column pass your error details from previous copy activity. Under mappings tab load only required columns as csv file.

Check this video which explains Add additional columns during copy in Azure Data Factory

Hope this helps. Please let us know if any further queries.


Please consider hitting Accept Answer button. Accepted answers help community as well.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.