BlobEventsTrigger could not be fired if the Blob file is updated via sftp

Shao Peng Sun 81

I created a BlobEventsTrigger in Synapse to trigger one of my pipeline.
If the file is uploaded via Azure Storage Explorer or curl, then the trigger could be fired and works fine.
But if the file is loaded via sftp command, then the trigger could not be fired.

Could you please help to understand why could this happen and how could make it work with SFTP approach?

KranthiPakala-MSFT 46,597 Reputation points Microsoft Employee

2022-11-23T03:26:50.173+00:00

Hello @Shao Peng Sun ,

Just checking in to see if the below answers were helpful. If it answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

2 answers

Nandan Hegde 32,906 Reputation points MVP

2022-11-21T12:34:24.1+00:00

A similar thread :
https://learn.microsoft.com/en-us/answers/questions/917727/trigger-does-not-work-when-using-sftp-container.html
Please sign in to rate this answer.
Shao Peng Sun 81 Reputation points

2022-11-23T06:58:51.737+00:00

Thank you for your help!
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
KranthiPakala-MSFT 46,597 Reputation points Microsoft Employee

2022-11-21T19:23:34.28+00:00
Hello @Shao Peng Sun ,

Thanks for the question and using MS Q&A platform.

As @Nandan Hegde , pointed, please refer to above thread link where a similar issue has been discussed.

In order to make it work, please added SFTPCommit to the AdvanceFilters on the EventGrid SystemTopic thats auto generagted when the trigger is created from datafactory for the storage account.

SFTPCommit should be added as you need the trigger to start when the file is completely loaded on the sftp container.

I'm working with internal team, to see if there is scope to improve the product or to clarify this in public documentation. Will keep you posted as soon as we have an update.

But in the meantime, as a workaround, you can follow the above process to mitigate the issue.

Hope this will help. Please let us know if any further queries.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Please sign in to rate this answer.
Shao Peng Sun 81 Reputation points

2022-11-23T06:58:19.893+00:00

Thank you so much for your help @KranthiPakala-MSFT ! I think this is helpful. I go to "Event Grid" -> "System Topics" in my Azure Portal and only find one topic which is related to my Storage account, but I am not sure if this is the topic auto generated when trigger is created for the storage account. Then I get into the topic and then click "JSON view" and I see a very simple JSON as below. Should I edit manually to add AdvancedFilters or this is not the right place to add?

KranthiPakala-MSFT 46,597 Reputation points Microsoft Employee

2022-11-24T03:07:41.237+00:00

Hello @Shao Peng Sun ,

You will have to go to Event Grid and to System Topics and then select the topic and under the Event subscritpion go to Filters and then in Advanced Filters section add above mentioned and it will resolve your issue. Please see below screenshot for more clarification.

Hope this helps. Let us know if you have further query.

Thank you

----------

Please do consider clicking on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.

Shao Peng Sun 81 Reputation points

2022-11-24T14:28:58.673+00:00

Thank you! @KranthiPakala-MSFT , This works for me when I try this in our dev environment. But when I go to another environment which is owned by us but we need running jobs there, I go to Event Grid -> System Topics, then there is not any system topics even we have created 2 Storage Event triggers. I assume there would be system topics generated automatically once we created and published Storage Event triggers, am I correct. Since I could not find any system topic created in that environment, is that any system configuration causing this issue? Or it's just because I don't have right permission to view system topics?
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

BlobEventsTrigger could not be fired if the Blob file is updated via sftp

2 answers

Your answer