Share via

What's the best way to run an indexer (Az Search) when a new file is uploaded to the source (Az Blob storage)?

Ajmal Khan 20 Reputation points
2023-01-16T16:31:46.44+00:00

We would like Azure Cognitive Search to recognise when there has been a change to the source data (in our case Azure blob storage) and subsequently refresh the index to show the new data in the search. We are aware of the Azure libraries and methods that are available and can trigger the indexer but is there a better solution? We assumed there would be considering the data stream is within the Azure architecture/cloud. We have considered running a schedule to refresh and also per upload, of course, the latter being extremely inefficient and requiring additional programming logic for multiple/many file uploads.

Your thoughts, direction and help are greatly appreciated.

TIA,
Ajmal

Azure AI Search
Azure AI Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
1,354 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
3,201 questions
0 comments
Accepted answer
  1. VenkateshDodda-MSFT 25,111 Reputation points Microsoft Employee Moderator
    2023-01-19T03:37:24.47+00:00

    @Ajmal Khan Thank you for reaching out to Microsoft Q&A, Sorry for the delay in my response.
    Azure Blob storage indexer supports change detection mechanism. Change detection with built-in indexer schedule running every 5 minutes is the recommended setup for your scenario from the search end.

    You can also check for other setups such as having event grid with push API (https://learn.microsoft.com/en-us/azure/search/search-what-is-data-import), in case the suggested solution of pull indexer with a 5 minute schedule is not something that would help them. Built-in indexers recommended schedule is every 5 minutes as a minimum.

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.