on premise file system Microsoft.DataTransfer.Common.Shared.HybridDeliveryException

Question

Hi Guys,

We have copy activity, executing stored procedure and storing the file to on premise file system some time copy activity give below error
{
"errorCode": "2200",
"message": "ErrorCode=UserErrorInvalidEncryptedCredentialProvided Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException Message=The data source credential had been deleted from current Integration Runtime node with payload"had been deleted from current Integration Runtime node with payload '' when you delete the link service on Azure Portal, or the task has the wrong payload, please create new link service with your credential again.,Source=diawp,'",
"failureType": "UserError",
"target": "",
"details": []
}

Tried differnt options, included wait times, changed to ip, saved user/pwd to key vault etc nothing works.

When we test connection from Linked service connection successfully; but from datasets we get above error, but on multiple tries it works fine , gateway server dont have any ping loss; connectivity is perfect, even IR logs show this error

Regards,
navin

Answer 1

Reason for failure:

Server moved from one dc to other DC

Steps to Resolve:
Export Backup
Uninstall IR
Install IR
Import backup

MS have to provide proper Error messages

Answer 2

For anyone in the future that lands in this page. I didn't have High-Availability enabled, tried re-creating my source Linked Service and still no luck. The way I solved it was by re-entering the credentials for the Linked Service on the Sink side.

Answer 3

Hi @Anonymous ,

Thank you for using this forum and sorry you are experiencing this.

Is your SHIR a High Availability mode with two or more nodes? If so, could you please verify if the credentials are in sync state, which means the credentials stored on dispatcher node are synced to other worker nodes.

You could probably verify this from ADF UI (Author & Monitor) -> Monitor - > Integration Runtimes -> Click on your Self Hosted Integration Runtime and see if the dispatcher node and worker nodes are in sync.

To avoid this issue, please make sure that dispatcher and worker nodes are in credentials sync state. Otherwise try to re-input credentials for new dispatcher.

Below are few workarounds you could also try to mitigate/resolve this issue:

1. Try Integration Runtimes - Sync Credentials
2. Try to restart your Integration Runtime from Integration Runtime Configuration Manager.
3. Try to re-input the username and password in your linked service and republish the linked service again.
   Nothing but resetting the data source credentials on gateway machine again with ADF portal or azure cmdlet.
   a. Login the ADF portal and go to relative data factory panel
   b. Open the “Author and deploy”, then find your linked service.
   c. In the LS, you just need to re-input the relative credentials on the UI and click “OK” button, the credentials of your data source will be reset.
   d. Redeploy your LS when you finish reset operation. Later trigger your pipeline to run again, it can be successful.
4. If any of the above doesn't work, please try to create a new linked service as mentioned in the error message and try to run the pipeline. Hope the above info helps. Let us know how it goes.

---

Thank you
Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.

Answer 4

its not HA, and tried multiple restarts, even server hosting IR too