how long to generate azure blob inventory

Question

We set blob inventory on a storage account with about 100M objects, and it's still running 48 hours later. We can see the JSON that says "inventoryStartTime", but it is now more than 48 hours, and no actual files have appeared.

How long should it take?

Answer 1

@Mathieu Longtin Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

How was the inventory frequency was set (weekly or Daily)? A blob inventory run is automatically scheduled every day when 'Daily' is chosen. Selecting 'Weekly' schedule will only trigger the inventory run on Sundays.

Prefix match: Filter blobs by name or first letters. To find items in a specific container, enter the name of the container followed by a forward slash, then the blob name or first letters. For example, to show all blobs starting with “a”, type: “myContainer/a”.

Please cross verify the Rule filters set: https://learn.microsoft.com/en-us/azure/storage/blobs/blob-inventory#rule-filters

If you configure a rule to run daily, then it will be scheduled to run every day. If you configure a rule to run weekly, then it will be scheduled to run each week on Sunday UTC time.

Most inventory runs complete within 24 hours. For hierarchical namespace enabled accounts, a run can take as long as two days, and depending on the number of files being processed, the run might not complete by end of that two days. The maximum amount of time that a run can complete **before it fails is six days.
**
Runs don't overlap so a run must complete before another run of the same rule can begin. For example, if a rule is scheduled to run daily, but the previous day's run of that same rule is still in progress, then a new run won't be initiated that day. Rules that are scheduled to run weekly will run each Sunday regardless of whether a previous run succeeds or fails. If a run doesn't complete successfully, check subsequent runs to see if they complete before contacting support. The performance of a run can vary, so if a run doesn't complete, it's possible that subsequent runs will.

Inventory policies are read or written in full. Partial updates aren't supported. Inventory rules are evaluated daily. Therefore, if you change the definition of a rule, but the rules of a policy have already been evaluated for that day, then your updates won't be evaluated until the following day. Learn more here

Enable blob inventory reports for your storage account. See Enable Azure Storage blob inventory reports.

Inventory jobs take a longer time to complete in certain cases

An inventory job can take a longer amount of time in these cases:

• A large amount new data is added
• A rule or set of rules is being run for the first time The inventory run might take longer time to run as compared to the subsequent inventory runs.
• In inventory run is processing a large amount of data in hierarchical namespace enabled accounts An inventory job might take more than one day to complete for hierarchical namespace enabled accounts that have hundreds of millions of blobs. Sometimes the inventory job fails and doesn't create an inventory file. If a job doesn't complete successfully, check subsequent jobs to see if they're complete before contacting support. Please let us know if you have any further queries. I’m happy to assist you further.  

---

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

What is the cutoff datetime for a daily blob inventory?