Trigger does not work when using SFTP container

I have a set up a trigger on Azure Data Factory. If I upload a file manually to the container it gets triggered, but if I upload it by SFTP it does not.

I know that the functionality was not present on the past, but according to the documentation: https://learn.microsoft.com/en-us/azure/storage/blobs/storage-feature-support-in-storage-accounts now it is enabled.

How I can make it so it can also work when uploading a file via SFTP with a client such as FileZilla?

1 answer

KranthiPakala-MSFT 46,422 Reputation points Microsoft Employee

2022-07-07T20:14:47.423+00:00

Hello @Guillermo González-Santander ,

Thanks for the question and using MS Q&A platform.

Could you please confirm if you are referring to the SFTP associated with Azure Storage? Or any third party SFTP?

If it is SFTP related Azure Storage account, then it is a known bug identified by product team and they have confirmed that the fix will be deployed when the Storage SFTP feature goes to GA. Please note Azure Storage SFTP feature is currently in preview.

Hope this info helps. Let us know if you have further query.

Thank you
Please sign in to rate this answer.
KranthiPakala-MSFT 46,422 Reputation points Microsoft Employee

2022-07-13T20:55:44.933+00:00

Hello @Guillermo González-Santander ,

Just checking in to see if the above info helped. And, if you have any further query do let us know.

Ivan Claudio-Galarza 6 Reputation points

2022-10-24T18:26:00.877+00:00

Hi @KranthiPakala-MSFT ,
I believe that Azure Storage SFTP feature is no longer in preview, but Im still having the same issue.
Can you confirm that the bug was fixed?

Jason Hyland 1 Reputation point

2022-11-09T22:09:31.693+00:00

Has this been resolved now we are GA with SFTP ?

KranthiPakala-MSFT 46,422 Reputation points Microsoft Employee

2022-11-10T04:59:55.707+00:00

Hi @Jason Hyland , @Ivan Claudio-Galarza ,

Could you please try adding SftpCommit, SftpCreate and SftpMakeDir to the advanced filters of unmatched events for the container and let us know if that helps to resolve the issue? Thanks

Let us know how it goes.

Thanks

Nandan Hegde 29,891 Reputation points MVP

2022-11-10T07:14:18.717+00:00

can you please check the file property :

If content type is blank the trigger is not initiated.

Ivan Claudio-Galarza 6 Reputation points

2022-11-10T12:08:12.747+00:00

Hi,
This solved the issue, I added SFTPCommit to the AdvanceFilters on the EventGrid SystemTopic thats auto generagted when the trigger is created from datafactory for the storage account.
I added only SFTPCommit since I need the trigger to start when the file is completety loaded on the sftp container.
Thanks

KranthiPakala-MSFT 46,422 Reputation points Microsoft Employee

2022-11-14T23:16:42.667+00:00

Hello @Guillermo González-Santander ,

Glad to know that the above information was helpful to resolve your issue. Appreciate much for confirming and sharing your inputs as it will help others reading this thread.

Påskesen, Simon Frank 1 Reputation point

2022-12-05T16:12:35.493+00:00

I have the same issue, but adding SftpCommit, SftpCreate and SftpMakeDir did not resolve it in my case.

Alexander Marek 0 Reputation points

2023-03-30T09:25:00.1333333+00:00

Hi!

Events are fired, but they are not forwarded to Azure Data Factory.

You can verify this by connecting with the SFTP client of your choice. (e.g. FileZilla)

When you upload a file by SFTP, the event is visible in azure blob storage.

However, it does not reach Azure Data Factory.

The problem is with the way the trigger is configured by Azure Data Factory:

In azure portal,

go to your blob storage resource

open "Events"

click the subscription (WebHook) that should fire on your SFTP uploads

and then open the "Filters" tab

You will see this:

Going through the SFTP docs, it shows that an SFTP upload actually provides two additional values:

So add the following two strings

And your ADF trigger will be fired! 🚀

There is another answer out there that explains this in a bit more detail:
https://techcommunity.microsoft.com/t5/azure-paas-blog/working-with-adf-storage-event-trigger-over-sftp/ba-p/3659652

nishant yadav 5 Reputation points

2023-05-08T13:35:36.5966667+00:00

Hi KranthiPakala-MSFT, It is mentioned here that the fix for the known issue will be deployed when the Storage SFTP feature goes to GA. With the Storage SFTP feature now in GA, is there a timeline on getting fix for the issue releases?
Sign in to comment