Is LUIS the right tool for this scenario?
I think this can be accomplished with LUIS but I don't know if this is the best way or there are other tools for it.
I have a list of 1,000 terms like:
- Cash Paid For Income Tax
- Cash Paid For Interest And Income Taxes
I have another list with millions of terms entered by users, and I need to find the best match to that master list of 1,000 records. So for example these two terms:
- Cash Paid For Tax and Interest
- Dollars Paid For Interest Or Income Taxes
would resolve to "Cash Paid For Interest And Income Taxes".
If I was doing this in LUIS I think I would make each term an Intent, and the millions of items to match would be Utterances. Since a single LUIS instance can only have 500 intents I'd have to set up a Dispatch approach with 500 intents each. Then to test it I'd use the batch testing to process 1,000 at a time.
Would that work well? Is there a better tool I can use for this type of analysis?
@Aiden Kaskela Yes the utterances do look to similar to group them. You can create more utterances and entities to distinguish the second part of the utterance like "Before", "After", "Income Tax" etc. These can be classified as entities like "Timeline" for before, after and "Events" for Income tax related terms.
The final JSON should list these under the classified intent along with the entities. The client application should be able to use this result and process the same in your application. I would recommend to try with sample data and check if the response from LUIS works for your application.
Sign in to comment