Denied access error on SFTP to delete file

Question

Hello,

I'm using a Copy Data activity to download a file from a SFTP server to ADLS and then delete it (Deletes files after completion, on the Source tab). I have read and write rights on the SFTP server. The dataset format is binary.

The activity complete successfully when I turn off the delete after completion feature, but when I turn it on, I get the following error message and my pipeline fail:

Failure happened on 'Sink' side. ErrorCode=SftpPermissionDenied,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Permission denied to access '/out/myfile.csv.gz'.,Source=Microsoft.DataTransfer.ClientLibrary.SftpConnector,''Type=Renci.SshNet.Common.SftpPermissionDeniedException,Message=Permission denied,Source=Renci.SshNet,'

I do not understand this permission denied access error message, as I'm able to delete files from the SFTP server when I use Filezilla or other tool, using the same account.

I'm also recently getting this same error message on another SFTP server which was working fine for months, with the pipeline in production.

I tried to put max concurrent connections to 1, disable "recursivity", nothing fix it.

The source dataset :

Thanks for the help

Answer 1

Hi @Alexandre G , I heard back from the product group. The permissions issue is due to a product defect in Data Factory, no error on your part. The team is aware of the issue, and knows the cause, but I do not have a date for a fix.

Please chain a Delete activity after the Copy activity. The reason the Delete activity works, is because the completion of the Copy activity closes the connection, making the Delete activity connection a separate transaction.

Answer 2

Hi MartinJaffer-MSFT, Do you have any update is this issue fixed on your end in ADFV2 or still you have to fix.

Answer 3

Having the same issue also. I can connect to the SFTP server properly with basic authentication, but i am not able to browse to the correct folder. Specifying the root or subfolder makes no difference. With WinSCP i can read, write, delete, so it is not the user not having enough authorization on the SFTP server. I would expect this basic functionality to properly work.