Azure Cognitive Search Indexer Not Reflecting Decreased Document Count After Deleting Files from Storage Account

Question

I have set up Azure Cognitive Search with a data source, index, and indexer. Additionally, I have created a skillset with incremental enrichment enabled and an indexer cache location connection in a storage account with a container named "enrichments". The cache and enrichments are being recorded successfully.

Initially, I had 5 text files in my Azure storage account, and the indexer correctly reflected a document count of 5/5 in both the indexers and index. However, when I delete or add 2 files from the Azure storage account and run or reset and run the indexer, the document count in the index increases when adding files, but it does not decrease when removing files from the storage account.

Steps taken:

Created data source, index, and indexer in Azure Cognitive Search.

Created a skillset with incremental enrichment enabled.

Created an indexer cache location connection in the storage account with the "enrichments" container.

Initially had 5 text files in the Azure storage account.

Indexer correctly reflected a document count of 5/5 in both the indexers and index.

Deleted or added 2 files from the Azure storage account.

Ran or reset and ran the indexer.

Document count in the index increased when adding files, but did not decrease when removing files.

Expected outcome: When deleting files from the Azure storage account and running the indexer, the document count in the index should decrease accordingly, considering the incremental enrichment and indexer cache settings.

Additional information:

The indexer is running without any errors or warnings.

The field mappings and data source connection are correctly configured.

The search index schema accurately reflects the structure of the data source.

The skillset has incremental enrichment enabled.

The indexer cache location connection is established successfully, and cache and enrichments are being recorded.

I would appreciate any insights or suggestions on how to resolve this issue and ensure that the document count in the index accurately reflects the files present in the Azure storage account, considering the incremental enrichment and indexer cache settings. Thank you.

Answer 1

Hello @Dinnemidi Ananda Kumar thanks for your patience. Based on the issue described. It is most likely you may not have implemented a deletion policy for your indexer. Please refer to Changed and deleted blobs - Azure AI Search | Microsoft Learn and let us know.

Best,

Grace