Azure Language Studio Clu

Question

i'm using language studio conversational language understanding in azure, added some intents with some phrases for each intent, submitted them for training 80% train and 20% test, deployed the model and tested it with a phrase that doesn't suppose to match anything in the intents phrases but however it matches above 85% !
can you tell me what im doing wrong, or what should i do to get above 70% match for the phrases that are actually close to the intent phrases ?

Ex:

intent: transportationRequest

intentPhrases: i want a transport request - i would like to request a transport..etc
testPhrase: don't

when i use this test phrase i get 97% match for the transportationRequest intent !

Answer 1

@Ahmed Magdy The model should have ideally returned None intent for the test phrase you have used. However, there are couple of things that might effect the model scores.

1. The training data, If you do not have enough training data i.e phrases and intents to train the model, the model can report lower accuracy. You should consider adding more labeled data along with entities and ensure the split between your training and test data is proportional. See more about the guidance on training here.
2. For the case where the above response is not reported as None intent, Check the project settings on the threshold that is set for None intent. If it set as a higher value between 0 and 1 consider to lower it and add examples to None intent to train the model. In the above example, the test phrase is actually not a sentence and you would ideally like to have them reported under None intent.

Try to check the above scenarios and retrain the model to check if you see better results.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.