ADF Error copying pipeline when no rows

Dean Holmes 21 Reputation points
2021-06-22T10:53:17.533+00:00

Hi

I have a SQL View that returns the days changes to be applied to a table in Dynamics. This is working fine when there are rows of data to copy, but I am getting this error when the SQL View returns zero rows.

This seems like a bug to me. I don't seem to be able to find any settings to stop this happening

Failure happened on 'Sink' side. ErrorCode=DynamicsMissingKeyColumns,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=,Source=,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Input DataSet must contain keycolumn(s) in Upsert/Update scenario. Missing key column(s): accountnumber,Source=Microsoft.DataTransfer.ClientLibrary.DynamicsPlugin,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Input DataSet must contain keycolumn(s) in Upsert/Update scenario. Missing key column(s): accountnumber,Source=Microsoft.DataTransfer.ClientLibrary.DynamicsPlugin,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Input DataSet must contain keycolumn(s) in Upsert/Update scenario. Missing key column(s): accountnumber,Source=Microsoft.DataTransfer.ClientLibrary.DynamicsPlugin,'

The Pipeline is going from Snowflake DB -> Azure Blob Storage for staging -> Dynamics 365

@KranthiPakala-MSFT Are you able to help?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,134 questions
{count} votes

Accepted answer
  1. HimanshuSinha-msft 19,381 Reputation points Microsoft Employee
    2021-06-22T23:10:33.917+00:00

    Hello @Dean Holmes ,
    Thanks for the ask and using the Microsoft Q&A platform .
    I think you can add a lookup activity before the copy activity . The lookup should call the view and it should execute the copy activity only when the value returned is more tthe 0 .

    Lookup -> IF ( lookup value is > 0 ) -> Copy activity .

    Please do let me know how it goes .
    Thanks
    Himanshu
    Please do consider clicking on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful