In our chatbot, we experience the answer's confidence score of the same question changes - and it seems to depend on the timing it was asked.
If it is asked first time from our home menu, the score is consistent.
If the same question is asked after a certain reply, the score we get is lower.
- the observation I got is that, this happens if the reply of the previous question has related questions
Can I ask if this is the expected behavior - theoretically I think it is, but I hope it can be confirmed as it helps on our design of the threshold score and multi-turn conversations.
May I ask if there are other factors that can affect the score too?