PDF Documents Not Fully Indexed by Azure Search

Question

Why are my PDF documents not fully indexed by Azure Search indexer, even though no errors are raised?

PS - Based on common issues that we have seen from customers and other sources, we are posting these questions to help the Azure community.

Answer 1

The problem arises due to the text extraction limits of the Azure Search indexer, which vary by pricing tier. Specifically, the Free tier allows for 32,000 characters, Basic tier for 64,000 characters, and Standard tiers from 4 million to 16 million characters. If your PDF documents contain more text than the allowed limit, the excess text is truncated and not indexed. This can lead to situations where the indexer processes the document without raising errors but fails to index all the content.

To address this, you should split documents with large amounts of text into smaller documents. This ensures that each document remains within the character limit of your pricing tier, allowing all text to be properly indexed.

For further details, you can refer to the Service limits in Azure AI Search documentation.

Please do not forget to "up-vote" wherever the information provided helps you, as this can be beneficial to other community members.