Azure Cognitive Search - Blob Indexer with Skillset fails

Question

Azure Cognitive Search - Blob Indexer with Skillset fails

Anonymous

When using Blob Indexer with Built-In skillset, it gives the following error message: "Document could not be processed -it may be encrypted or password protected, or its contents may be exceeding size limits."

The data source is a storage account with the default/automatic service-side encryption (SSE). The default indexer without any skillset works well.

SnehaAgrawal-MSFT 22,346 Reputation points

2023-09-08T13:58:45.07+00:00
@Anonymous Thanks for reaching here!

The error message:

"Document could not be processed - it may be encrypted or password protected, or its contents may be exceeding size limits"

indicates that the Blob Indexer with Built-In skillset was unable to extract the content or metadata from the document. This can happen due to several reasons:

The blob is encrypted or password protected.

The blob's contents may be exceeding size limits.

The blob has an unsupported content type.

To resolve this issue, you can try the following steps:

If the blob is encrypted or password protected, you can skip the blob with blob settings.

If the blob's contents are exceeding size limits, you can try to reduce the size of the blob or increase the maximum size limit for document extraction for your current service tier.

If the blob has an unsupported content type, you can try to convert the blob to a supported format or skip the blob with blob settings.

You may refer to detailed document here- Error: Could not extract content or metadata from your document.

If none of these steps work, you can try running the document through your indexer again later.

If the issue persists, please provide more information about the blob and the indexer configuration so that I can assist you better.
SnehaAgrawal-MSFT 22,346 Reputation points

2023-09-11T04:29:17.1433333+00:00

@Anonymous Just checking if you have chance to see recent response.

Please let us know if further query or issue remains.

Please accept as "Yes" if the answer provided is useful, so that you can help others in the community looking for remediation for similar issues.
Anonymous

2023-09-11T14:30:23.6333333+00:00

The Blob Indexer without any Built-In skillset works without any problem; but, the moment I added a built-in skillet, for example "#Microsoft.Skills.Text.KeyPhraseExtractionSkill", the blob indexer stops working. Note that the Blob Indexer with/without skillsets are tested on the same set of documents/blobs. All the above solution steps are about the blob data. If that is indeed the cause, the Blob Indexer without skillsets should also fail.
SnehaAgrawal-MSFT 22,346 Reputation points

2023-09-12T06:49:38.52+00:00

@Anonymous Thanks for clarification. It seems you are not splitting the text so it can be fed to the Key Phrase Extraction skill.

As per Key Phrase Extraction cognitive skill - Azure Cognitive Search | Microsoft Learn

Data limits The maximum size of a record should be 50,000 characters as measured by String.Length.

If you need to break up your data before sending it to the key phrase extractor, consider using the Text Split skill.

If you do use a text split skill, set the page length to 5000 for the best performance.

Please let us know if further query or issue remains.
Anonymous

2023-09-12T13:04:54.2966667+00:00

Hi @SnehaAgrawal-MSFT , I tested with a short document and it has the same error: "Document could not be processed - it may be encrypted or password protected." I found this article seems to be the most relevant: https://learn.microsoft.com/en-us/azure/search/search-howto-index-encrypted-blobs. However, the blob only has the default settings and does not have any customer managed encryption.
SnehaAgrawal-MSFT 22,346 Reputation points

2023-09-13T05:35:57.5666667+00:00

@Anonymous Thanks for reply! Could you please reach out to us by sending an email with subject line “Attn:Sneha |thread title” to AzCommunity[at]microsoft[dot]com referencing this thread, would require deeper investigation.
SnehaAgrawal-MSFT 22,346 Reputation points

2023-09-15T05:03:43.0666667+00:00

@Anonymous A gentle follow-up please reach out to us by sending an email with subject line “Attn:Sneha |thread title” to AzCommunity[at]microsoft[dot]com referencing this thread.

Your answer

SnehaAgrawal-MSFT 22,346 Reputation points

2023-09-08T13:58:45.07+00:00

@Anonymous Thanks for reaching here!

The error message:

"Document could not be processed - it may be encrypted or password protected, or its contents may be exceeding size limits"

indicates that the Blob Indexer with Built-In skillset was unable to extract the content or metadata from the document. This can happen due to several reasons:

The blob is encrypted or password protected.

The blob's contents may be exceeding size limits.

The blob has an unsupported content type.

To resolve this issue, you can try the following steps:

If the blob is encrypted or password protected, you can skip the blob with blob settings.

If the blob's contents are exceeding size limits, you can try to reduce the size of the blob or increase the maximum size limit for document extraction for your current service tier.

If the blob has an unsupported content type, you can try to convert the blob to a supported format or skip the blob with blob settings.

You may refer to detailed document here- Error: Could not extract content or metadata from your document.

If none of these steps work, you can try running the document through your indexer again later.

If the issue persists, please provide more information about the blob and the indexer configuration so that I can assist you better.
SnehaAgrawal-MSFT 22,346 Reputation points

2023-09-11T04:29:17.1433333+00:00

@Anonymous Just checking if you have chance to see recent response.

Please let us know if further query or issue remains.

Please accept as "Yes" if the answer provided is useful, so that you can help others in the community looking for remediation for similar issues.
Anonymous

2023-09-11T14:30:23.6333333+00:00

The Blob Indexer without any Built-In skillset works without any problem; but, the moment I added a built-in skillet, for example "#Microsoft.Skills.Text.KeyPhraseExtractionSkill", the blob indexer stops working. Note that the Blob Indexer with/without skillsets are tested on the same set of documents/blobs. All the above solution steps are about the blob data. If that is indeed the cause, the Blob Indexer without skillsets should also fail.
SnehaAgrawal-MSFT 22,346 Reputation points

2023-09-12T06:49:38.52+00:00

@Anonymous Thanks for clarification. It seems you are not splitting the text so it can be fed to the Key Phrase Extraction skill.

As per Key Phrase Extraction cognitive skill - Azure Cognitive Search | Microsoft Learn

Data limits The maximum size of a record should be 50,000 characters as measured by String.Length.

If you need to break up your data before sending it to the key phrase extractor, consider using the Text Split skill.

If you do use a text split skill, set the page length to 5000 for the best performance.

Please let us know if further query or issue remains.
Anonymous

2023-09-12T13:04:54.2966667+00:00

Hi @SnehaAgrawal-MSFT , I tested with a short document and it has the same error: "Document could not be processed - it may be encrypted or password protected." I found this article seems to be the most relevant: https://learn.microsoft.com/en-us/azure/search/search-howto-index-encrypted-blobs. However, the blob only has the default settings and does not have any customer managed encryption.
SnehaAgrawal-MSFT 22,346 Reputation points

2023-09-13T05:35:57.5666667+00:00

@Anonymous Thanks for reply! Could you please reach out to us by sending an email with subject line “Attn:Sneha |thread title” to AzCommunity[at]microsoft[dot]com referencing this thread, would require deeper investigation.
SnehaAgrawal-MSFT 22,346 Reputation points

2023-09-15T05:03:43.0666667+00:00

@Anonymous A gentle follow-up please reach out to us by sending an email with subject line “Attn:Sneha |thread title” to AzCommunity[at]microsoft[dot]com referencing this thread.

Share via

Azure Cognitive Search - Blob Indexer with Skillset fails

Your answer