Azure Data Factory is not firing the trigger when I create a blob as output from ETL tool

Azevedo Emanoel 1

Hi,

I'm using a ETL tool to save files into a blob container and it is not firing the trigger that I have created to the process.

When I manually upload the file to the storage, it works fine.

Can someone help?

BhargavaGunnam-MSFT 26,226 Reputation points Microsoft Employee

2024-04-19T17:07:24.86+00:00

Hello Azevedo Emanoel,

Are you using the storage event triggers in ADF? if yes, can you please check the trigger configuration?
Azevedo Emanoel 1 Reputation point

2024-04-19T17:52:19.7766667+00:00

Yes, I'm using storage events.

As I told, when I upload or use AzCopy to upload the files, the trigger works fine. When I create the file as output of my ETL tool, it doesn't fire the trigger.

Find attached the trigger configuration.

Blob Container File:
BhargavaGunnam-MSFT 26,226 Reputation points Microsoft Employee

2024-04-19T18:33:40.7833333+00:00

from the below document:
The Storage Event Trigger currently supports only Azure Data Lake Storage Gen2 and General-purpose version 2 storage accounts.

Can you please confirm if you are using ADLS gen2 or general purpose v2 ?

https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-event-trigger?tabs=data-factory

also can you please check if Soft delete enabled. When the JSON file is created on Storage side, check "show deleted objects"

It's possible that a file with same name was previously deleted and now is waiting for 7days to be flushed.

Check the file path in the container. If the file is located in subfolders, try to use the full path up to that file in the trigger configuration.

Ex. File the file is located in container/folder/subfolder/file.json, use :

Container name : container/folder/subfolder
Azevedo Emanoel 1 Reputation point

2024-04-19T19:03:26.53+00:00

Yes, it is ADLS gen2 and the soft delete is not enabled.

Once again, as I've told, it is not working just when I create the file as output of my ETL tool.

When I copy the file with AzCopy, or upload the file manually, it works with no problem.

The difference I can see now is that, when I upload it manually, the content of the blob is text/plain:

When it is created from the ETL, the content is application/octet-stream:

Could it be the problem?
BhargavaGunnam-MSFT 26,226 Reputation points Microsoft Employee

2024-04-19T19:32:09.37+00:00

Thanks for checking these.

I am not sure "content type" would cause an issue.

Can you please change the ignore Empty Blobs to No and see if it makes any difference.
Azevedo Emanoel 1 Reputation point

2024-04-19T20:09:53.01+00:00

I changed it, but no success. The trigger is not starting.
BhargavaGunnam-MSFT 26,226 Reputation points Microsoft Employee

2024-04-19T22:12:55.6366667+00:00

Could you try using a different storage account? It appears that the current storage account is not generating the event for ADF to process.

If that doesn't help, further investigation from the storage end will be needed. I suggest opening a support case for additional troubleshooting. If you don't have a support plan, please let me know so I can enable a one-time free support request for this case.

I look forward to hearing from you.