ADF pipeline failing while pulling the data from lake(parquet file) to synapse.

Question

We have pipeline that pulls the data from dynamics AX to azure data lake and then push the data into synapse analytics database. This is scheduled to run every day at night. Pipeline is failing sometimes with below error, and I am not able to find out the exact reason for error. This piepeline getting succeeded after the reran without any issue. The copy activity that pulls data (parquet) from lake to synapse temp table is failing. Can someone help me to confirm my thought.

here is the error message for your reference

Operation on target for each file failed: Activity failed because an inner activity failed; Inner activity name: copy data from data lake to dw temp, Error: ErrorCode=UserErrorSqlDWCopyCommandError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=SQL DW Copy Command operation failed with error 'One or more errors occurred.',Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Data.SqlClient.SqlException,Message=One or more errors occurred.,Source=.Net SqlClient Data Provider,SqlErrorNumber=110813,Class=16,ErrorCode=-2146232060,State=1,Errors=[{Class=16,Number=110813,State=1,Message=One or more errors occurred.,},],'

Answer 1

Hi @Sheena Jithesh ,

Welcome to Microsoft Q&A platform and thanks for posting your question here.

As I understand your issue, you are trying to load the data from ADLS file to Synapse DW, and you are getting error during the process, please let me know if that is not the case.

This issue might have been caused due to Connectivity issue between the self-hosted IR and your Azure Synapse instance or the self-hosted IR and the data source or sink

To know how to troubleshoot the error , kindly visit : Troubleshoot guide for connectivity issue between the self-hosted IR and your Azure Synapse instance

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you.
  Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators