Feature request - Datafactory copy activity default copy option to be polybase at the backend.

Question

Scenario:

1. In azure synapse analytics, built in copy activity uses polybase as default.
2. Pipeline in synapse with source as postgre and sink as synapse dedicated pool works fine with polybase as copy method.
3. However datafactory throws following exception, with same set up of polybase.
4. error:
   Operation on target Copy data1 failed: ErrorCode=PolybaseOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error happened when loading data into SQL Data Warehouse. Operation: 'Polybase operation'.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Data.SqlClient.SqlException,Message=HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: HadoopExecutionException: Not enough columns in this line.,Source=.Net SqlClient Data Provider,SqlErrorNumber=107090,Class=16,ErrorCode=-2146232060,State=1,Errors=[{Class=16,Number=107090,State=1,Message=HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: HadoopExecutionException: Not enough columns in this line.,},],'

This denotes that, there is something in ADF polybase setting, which is uncommon to Synapse polybase setting.

Feature Request:

1. Can ADF engine use same polybase as in synapse.

Answer 1

Hello @Shaik, Khaja ,
Thanks for the feedback and using Microsoft Q&A platform .We will pass on this feedback to the ADF team and share if we get any updates from them .
You can also pass the feedback from the ADF UI

Thanks
Himanshu

-------------------------------------------------------------------------------------------------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators