Azure CLU is not producing expected result.

Question

During negative scenario testing (e.g., 'xyx'), the kb_search intent incorrectly fetches an entity with 100% confidence, expected entity should be empty.Testing Negative Scenarios.png

Answer 1

It seems that the issue you're encountering is related to how Azure CLU (Conversational Language Understanding) is handling entity extraction during negative scenarios. Here are a few steps you can try to address the problem:

1. Entity Definition Review: Double-check how entities are defined within your CLU project. Ensure that entities are specific and that irrelevant or nonsensical inputs like 'xyz' are excluded from entity extraction rules.
2. Training Data Adjustment: Include more examples of negative scenarios (inputs where no entity should be identified) in your training data. This can help the model understand patterns where an entity should not be detected.
3. Confidence Score Tuning: You can adjust the confidence threshold for entity detection. If the confidence score is too high for incorrect matches, consider lowering the threshold to reduce false positives.
4. Negative Samples in KB: If you're using a knowledge base (KB) search, ensure that the KB contains negative samples or that there’s a mechanism to reject queries that don’t match valid entries.
5. Custom Logic: Implement custom validation logic post-entity extraction to ensure that low-confidence entities or unexpected matches (like in negative scenarios) are handled accordingly, for example by filtering out results with certain characteristics.

If this persists, you might need to revise the model or escalate to Azure support for further diagnostics based on the specifics of your configuration.