Newbie need some help on Cognitive Services

Question

Hi there,

I am a newbie to Cognitive service and wondering if someone
can guide me in the right direction please. I want to create a document
AI solution. I have the following requirements but not sure
where to start? :) Do I need to start with LUIS? What is the benefit
of using MS cognitive services vs. writing your own models in pythons
and run on AWS?

• Ability to review the documents in Word/PDf and to have the
  system automatically tag certain part based on predefined rules.
• Add certain information (from the source docs) to my library
  while reviewing.
• Question based search and to get the results. Example
  What is the company license renewal date? How may employees
  contracts expiring this month?

Thanks in advance.

Answer 1

@TechTalk LUIS offers pre-built domains for your apps that are authored with the resource. These domains are not specific to certain area or an industry but these are a set of common entities and intents that get added to your app so you need not create them manually. For example, If you add the calendar pre built domain it adds examples related to some common phrases like Please create a meeting from 2 to 3 and tags 2 as the calendar.starttime and 3 as calendar.endtime. This is really helpful if your end user types in something like this as it automatically tags these entities and provides you an output in response which can be used for further processing. This allows you to build phrases specific to your business logic and train the models further.

You can import data from a CSV file to LUIS but the format of the file or columns need to setup based on the programmatic API request parameters. An example of this scenario is listed in our documentation with a sample app.

Yes, if you want to build a bot with a FAQ or Question & Answer scenario then LUIS & QnA can be integrated with your bot framework. For all these scenarios you can use QnAMaker along with sharepoint. But the document format is important to check if QnAMaker can parse these documents to create a knowledge base. Ideally, QnAMaker is useful if you have a long FAQ type document with question and answer format or a sheet with data mapping between columns.

If you would like to lookup questions like Show me all contracts with notice period 'less that 60 days' with 'expiry in 2020'? Then, QnAMaker would not be able to pull this data as this time/date dependent lookup. This can be achieved if you integrate your bot with a DB source to run a query where you can use LUIS to extract information like Year:2020, Validity: <60, Type:expiry and run this query against your backend DB source. In this case, LUIS app needs to be trained with the appropriate entities and intents by adding such phrases in the training phase. The biggest advantage of these scenarios is you can train your model incrementally instead of covering all scenarios from day 1. This helps you review the questions customers are asking and modify your app to include all phrases that might have been missed.

Yes, data can be fetched from a file which is in the required format from SharePoint if you are using QnAMaker and it can be added as KB.

Form recognizer can be used for any valid file formats. The standard models available will help you recognize standard formats like receipts but if you have clauses in paragraphs you can still use the Form recognizer labeling tool to train and label certain sections. Using a labeling tool is very simple to use and you can try this out by downloading the container to check if this works for your doc format.

Cognitive services can be integrated with PowerBI and one of the common scenario is sentiment analysis and key phrase extraction. PowerBI documentation provides an example or tutorial that can help you get started.

Answer 2

@romungi-MSFT . Thanks for the well explained reply. I am surely feeling more knowledgeable now. I prefer MS solution so I have some more questions for you, if I may? :)

• My understanding is that my starting point would be LUIS and I would develop a model based on that? Does LUIS offers models for Legal and Real Estate industry?
• Do I need to write a custom pipeline to extract data from the WORD/PDF documents for storage and reference purposes?
• With regard to bots, would I be using bots/QnA together? Scneario example, lets say, we have about 10,000 documents in our Sharepoint. Some of the documents talk about a 'lease notice period' and 'expiry'. Ideally I would like to type a question: Show me all contracts with notice period 'less that 60 days' with 'expiry in 2020'? OR may be something like, Show me 'all leases' with 'auto renewals in 2020'? The person querying that data would ask similar questions? Is this something bot service, alone, would do? Does it require extensive training on the type of questions a user may ask? For example , set aside 200 types of question and then train for each question? ** This requirement is critical**
• Can data be fetched and read directly from Sharepoint online (CDS)?
• You also mentioned form recognizer. It seems like, it is more suitable for a document with standard tables. Whereas, a lease contract, contains clauses in paragraphs. Will form recognizer still be used in this case?
• I have read about AI+Power BI. I use it extensively in our business. Can you please cite some examples of where power bi customers can leverage it using cognitive services.

Again, I appreciate your help and I look forward to hear from you.

Answer 3

@TechTalk Thanks for your interest in Azure Cognitive Services. Cognitive services like LUIS help you to train your models based on language understanding without any setup to create a model from scratch, It provides a dedicated portal where you can create a LUIS resource and author your applications which get trained based on the entities & intents created. This helps to customize your app to predict based on the information you have trained it for and provide accurate information to predict users intention. This helps you to provide appropriate choices to customer based on which further processing is determined. Running a custom model with python on AWS might require code to create your model in python and spinning up additional resources of compute to setup and run your model. LUIS helps you avoid this by just calling the API and you get billed based on the number of requests.

The other scenarios you have mentioned indicate that you are looking at a informative bot like solution where you would like to add content to your library and then train models to identify documents or part of it and provide customers important details about their documents.

I think the following resources from Azure will help you get this setup done by integrating them to augment their responses.

1. Azure Form recognizer is a service that will enable you to provide important information from documents when you upload them in a certain format. This service/models can be customized to pull information from document structures that you train from with the ability to tag this information against a custom label. For example: A contract date can be pulled from an image of a receipt and returned in the JSON response as a label. This can be utilized to inform your customer of upcoming dates if they are processing thousands of such documents.
2. If more such document formats are being introduced you can train new models and compose various models to create a unified model that can provide a response for multiple document formats. For example: You can use the unified model to analyze a document with tables and it can also analyze a document with receipts to check if the total of a bill is correct. This can also be trained for hand written documents so you need not worry about missing information on such documents.
3. The last scenario indicates that you want to present a bot which can pull the information from the above documents and inform customers about the details. This can be achieved by using a bot service which can retrieve information from the processed documents above. The processed document response from above should be stored in a SQL or cosmosdb like database and queried from the bot service to provide an appropriate response. Bot service allows integration with LUIS and QnAMaker so you can effectively use LUIS to understand user's intent and then lookup related information from QnAMaker for FAQ scenario questions or lookup a DB to provide information from documents that are analyzed by form recognizer.

I hope this helps you understand the structure of cognitive services offering and allows you to experiment with these services which are available under free tier and proceed to higher tiers based on your requirements.