Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
2,901 questions
Inconsistency in QA Responses Across Different Environments
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 74%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMP%3C/text%3E%3C/svg%3E)
Marlus Pasinato
0
Reputation points
We have encountered an issue with our client's setup, where we have designed and implemented intelligent bots for webchat and WhatsApp, structured in an orchestrator/CLU/QA framework.
In our configuration, we have three distinct environments: Development (DEV), Homologation (HML), and Production (PRD). We have observed the issue where, despite utilizing identical knowledge bases (KBs), undergoing the same training, and employing consistent release versions across all environments, we encounter disparities in the responses provided by the QA component. Specifically, this discrepancy in responses occurs in approximately 5-10% of the inquiries.
We are seeking assistance with the following:
- An explanation as to why these differences in responses are occurring.
- Guidance on how we can prevent this issue from arising in the future.
We look forward to any ideas.
Regards
{count} votes
-
romungi-MSFT 46,831 Reputation points Microsoft Employee2024-03-08T08:56:35.9733333+00:00 @Marlus Pasinato With respect to your first question, I would suggest to raise a support case with more details like your subscription, language resource, projects used, and deployments used to trace the responses from your queries.
With respect to best practice, based on my earlier experience, If it occurs 5-10% of the time, it could be a case where the user queries could be slightly different, and the model might provide a different response. With custom Q&A as a good practice you have the option to use active learning to review similar questions and accept or reject them or add alternate questions. I hope this helps!!
-
romungi-MSFT 46,831 Reputation points Microsoft Employee
2024-03-13T07:54:28.8033333+00:00 @Marlus Pasinato Did you get a chance to check any of the best practices helped in preventing inconsistencies? Thanks!!
-
Marlus Pasinato 0 Reputation points
2024-03-13T11:32:40.2366667+00:00 @romungi-MSFT we are getting our client to open a call with MS support but no, not sure id there is any doccumenttion on preventing inconsistencies.
Sign in to comment
Sign in to answer