Other than using Cognitive Service for Language is there a way to get coherent document summaries from unstructured PDFs using Cognitive Search?

@Education is the Key , Based on my understanding of your scenario description, highlighting the following approaches: You may leverage Azure OpenAI - this guide shows how to perform document summarization by using the Azure OpenAI GPT-3 model. Kindly take a look at the document architecture and workflow. Query-based document summarization You may use Azure Cognitive Search to extract information from unstructured PDFs and create document summaries. With Azure Cognitive Search feature "Skillsets" that allows you to define a set of skills to extract information from unstructured data. To create document summaries, you may use the "Text Analytics" skill to extract key phrases and sentiment from the PDFs, and then use the "Custom Entity Recognition" skill to identify important entities in the text. You can then use this information to generate a summary of the document. You can find more information on how to create a skillset for document summarization in the Azure Cognitive Search documentation. You can also use the Cognitive Search Enrichment Pipeline to perform additional processing on the extracted content, such as entity recognition or sentiment analysis, to further enhance the quality of the document summaries. Kindly let us know if you have further questions.

Creating Quality Document Summaries With Azure

Accepted answer

ajkuma 22,416 Reputation points Microsoft Employee

2023-05-02T19:54:34.15+00:00

@Education is the Key ,

Based on my understanding of your scenario description, highlighting the following approaches:

You may leverage Azure OpenAI - this guide shows how to perform document summarization by using the Azure OpenAI GPT-3 model. Kindly take a look at the document architecture and workflow.
Query-based document summarization

You may use Azure Cognitive Search to extract information from unstructured PDFs and create document summaries. With Azure Cognitive Search feature "Skillsets" that allows you to define a set of skills to extract information from unstructured data.

To create document summaries, you may use the "Text Analytics" skill to extract key phrases and sentiment from the PDFs, and then use the "Custom Entity Recognition" skill to identify important entities in the text. You can then use this information to generate a summary of the document.

You can find more information on how to create a skillset for document summarization in the Azure Cognitive Search documentation.

You can also use the Cognitive Search Enrichment Pipeline to perform additional processing on the extracted content, such as entity recognition or sentiment analysis, to further enhance the quality of the document summaries.

Kindly let us know if you have further questions.
Please sign in to rate this answer.
Education is the Key 65 Reputation points

2023-05-19T13:33:11.6633333+00:00

@ajkuma - One additional question as we attempt a workaround. Asking this question as a non-technical co-founder: are these solutions "plug-and-play" and is there an SDK available? My developers are concerned that extensive architecture changes may be required. Thanks!

ajkuma 22,416 Reputation points Microsoft Employee

2023-05-22T19:32:28.0833333+00:00

Apologies for the delay from over the weekend. Thanks for the follow-up.

Azure Cognitive Search provides the set of pre-built cognitive skills provided in Search can be used out-of-the-box, including the Document Extraction skill.
Having mentioned that, using the SDKs, you can create and manage search services, indexes, skillsets, and indexers programmatically, without having to manually construct HTTP requests. This can save you time and effort and make it easier to integrate Azure Cognitive Search into your existing application architecture.

Overall, based on the approach you choose, while there may be some initial setup and configuration required, Azure Cognitive Search is designed to be easy to use and integrate with your existing application architecture.

Education is the Key 65 Reputation points

2023-05-22T20:49:43.83+00:00

Really helpful insights, thank you!
Sign in to comment

Creating Quality Document Summaries With Azure

0 additional answers