Running Azure Cognitive search over archive tier blob storage

Ivan Teage 21 Reputation points
2021-05-10T08:18:08.077+00:00

Hi I have some containers in blob storage, a mixture of cool and archive tiers managed with Azure lifecycle management. I really want to build a search interface with Azure search across the blob storage, but the indexer won't index any blobs in the archive tier.
The documentation suggests the metadata is left online for archive blob storage - so I was hoping the search indexer would be able to use that metadata to make it's index?
What are my options / what is best practice?
thanks
Ivan

Azure AI Search
Azure AI Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
694 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,415 questions
Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
924 questions

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. deherman-MSFT 33,141 Reputation points Microsoft Employee
    2021-05-13T16:55:00.047+00:00

    @Ivan Teage
    I was able to confirm with the product team that there is currently no support for archive tier blobs with Azure Cognitive Search. I have asked that the documentation team add a note of this to avoid future confusion.

    For product feedback and feature requests I will refer you to our feedback forum. This allows the community to add their voice and upvote popular ideas. The forums are monitored and responded to by our product teams.
    https://feedback.azure.com/forums/932041-azure-cognitive-services

    Thanks for your patience while we looked into this. Let us know if you have further questions or issues and we will do our best to assist.

    -------------------------------

    Please don’t forget to "Accept the answer" and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

    0 comments
  2. Ivan Teage 21 Reputation points
    2021-05-14T07:14:29.71+00:00

    Thanks @deherman-MSFT .
    The confusion is because the documentation states that the metadata will be left on line for discovery, when in fact the search is not able to use that metadata even if 'use storage metadata only' option is ticked.

    From what I can tell I can index the blob files whilst the files are in cool or hot tiers, then keep the index active to allow search to identify them in the index even when they have moved to archive tier. We will be using Azure lifecycle management to move from cool to archive, so all blobs will be indexable initially. If you can comment on if this approach sounds sensible that would be appreciated, thanks

    Ivan

    0 comments