Storage event not matched within bulk insert.

Jose Gonzalez Gongora 5 Microsoft Employee

Hello everyone,

I am trying to set a storage event trigger that runs when a blob named _SUCESS is created in a container. I tested the trigger by uploading the blob manually and worked as expected. But when testing the trigger in the targeted scenario (A synapse pipeline creates 200+ blobs and one of those blobs is named _SUCESS) the triggered fails to fire.

Looking at the Event Grid Metrics I can see when the manual upload was made, the event was properly processed.

User's image

But not in the case of when multiples blobs were created.

Trigger configuration:

User's image

Thanks in advance.

Ramya Harinarthini_MSFT 5,306 Reputation points Microsoft Employee

2023-09-25T13:24:44.4033333+00:00

@Jose Gonzalez Gongora

We have tried to reproduce your scenario by creating a pipeline with trigger as below.

I have provided the blob path ends with as _SUCESS.docx

Uploaded multiple blobs from my local machine to Azure Storage account through AZ copy to same blob path provided in the pipeline.

there are 3 matching blob names ending with _SUCESS.docx

azcopy copy "C:\Users\raharina\Downloads\testdir" "https://storageaccountname.blob.core.windows.net/pranathi<sas>" --recursive

Source location

successfully uploaded multiple blobs to the Azure Storage account

From the trigger runs could see 3 storage events and blobs matching with 3 _SUCESS.docx

Could you please try to edit the blob path ends with field with extension and then check if the trigger runs successfully.

Blob path must end with filename or extension i.e _SUCESS.docx or .docx

Please let us know if you have any further queries. I’m happy to assist you further.
Jose Gonzalez Gongora 5 Reputation points Microsoft Employee

2023-09-25T17:16:10.85+00:00

Thanks Ramya Harinarthini_MSFT, just to clarify; I tested the trigger by uploading a single SUCCESS file using the Azure portal and it works fine. The issue is when there are 200+ blobs created (with the SUCCESS file included); the trigger never fires.

Are you suggesting that the file extension is the problem? Because if that's the case, I would rather to find a different approach because I can't add a extension to the SUCCESS file, because those are created by Spark and can't modify those.
Ramya Harinarthini_MSFT 5,306 Reputation points Microsoft Employee

2023-09-26T04:24:18.3233333+00:00

Jose Gonzalez Gongora

As the manual upload is working fine for the blob storage we would check with Azure Synapse team to get insights on this.
Ramya Harinarthini_MSFT 5,306 Reputation points Microsoft Employee

2023-10-04T08:19:21.75+00:00

Jose Gonzalez Gongora

For quicker resolution I would recommend you contact support, so If you have a support plan, I request you file a support ticket, else please do let us know, we will try and help you get a one-time free technical support. In this case, could you send an email to AzCommunity[at]Microsoft[dot]com referencing this thread as well as your subscription ID. Please mention "ATTN subm" in the subject field. Thank you for your cooperation on this matter and look forward to your reply.

Share via

Storage event not matched within bulk insert.