The best way in azure to store hundred of thousands of files ( docs, pdfs, mp4s) with queryable custom metadata

Question

What is the best way to store large numbers of files in Azure ( of any type ) with custom metadata ( Topic, category , location, sensitivity etc.. ) , which can be queried? So I can store 500,000 to 1 million records and query the 'table' to return all the files and metadata where, for example, the type is 'mp4', the topic is either 'AArdvark' or 'Anteater' and the location is 'London' ?

Answer 1

@Richard Scannell Thank you for reaching out to Microsoft Q&A. I understand that you want to know the best way to store a large number of files in Azure with query able custom metadata.

You can use Azure Blob storage for storing large amounts of data. Since you mentioned docs/pdfs/mp4s specifically, you can go with Block Blobs. Here is the different type of storage options available- https://learn.microsoft.com/en-us/azure/storage/common/storage-account-overview?toc=%2Fazure%2Fstorage%2Fblobs%2Ftoc.json#types-of-storage-accounts.

You can check this thread for different storage options and their specific use cases- https://stackoverflow.com/questions/29079268/differences-between-azure-block-blob-and-page-blob

In regard to the metadata, please check out how to setup and query the Blob metadata using REST API- https://learn.microsoft.com/en-us/rest/api/storageservices/setting-and-retrieving-properties-and-metadata-for-blob-resources

Hope this helps. If you still need any further assistance, please let me know and I will assist further. Thank you!
Remember:

Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

Want a reminder to come back and check responses? Here is how to subscribe to a notification.

Answer 2

Still No Accept Answer button

Answer 3

Thanks @SaiKishor-MSFT , I noticed that too, but I dont know where those additional numbers have come from - as far as I'm concerned I have one account, and neither has a number tacked on the end.

Thanks for your help. I'll try to explicitly authenticate with a numeric credential

Answer 4

Curious ! sign out & sign in with the same email address, reverted my credential to the 9815 version. I have accepted your answer, Thank you for your help.