Is there a way to force the connector to use the old direct login and BULKAPI method and not OAUTH / BulkAPI 2.0 when creating a new linked service on Azure Data Factory?
- From the new documentation it appears the new Salesforce connector for Azure Data factory relies on OAuth/Connected Apps and uses BulkAPI 2.0.
- This is a an issue for a few reasons:
- It is PITA to control access/test security/rights/auditing as opposed to a direct access to a single user.
- This is especially true in environments where the admin is not well versed in all the byzantine world of SFA record/object access and rights.
- Navigating this issue is a no go on many projects where companies have tight timelines to standup their SFA environments and integrations.
- Bulk API 2.0 is very inefficient for frequent small updates across many objects.
- Not being able to control batch size makes it unusable for many custom objects configured by teams that write haphazard/poor triggers / workflows which forces the need for specific batch sizes so you don't encounter timeouts and unexpected errors in downstream processes.
- For most environments, write a more efficient trigger isn't actually an option or one companies will spend money on.
- If you've worked in many different SFA Orgs you're going to encounter this and BULKAPI 2.0 after years still does not automatically select a workable batch size correctly (i.e. marketing materials aren't reality).
- In actually environments with lots of sales interactions infrequent updates/data maintenance are not an option.