Share via

A/B test LLM model serving

Michael Dong 40 Reputation points Microsoft Employee
2025-01-02T16:49:46.2866667+00:00

I have two version of fine-tuned Qwen2.5 model, how to do A/B test for these models ?

Azure AI services
Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
3,065 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. santoshkc 11,785 Reputation points Microsoft Vendor
    2025-01-03T12:23:25.1+00:00

    Hi @Michael Dong,

    Thank you for reaching out to Microsoft Q&A forum!

    You can A/B test your two fine-tuned Qwen2.5 models in Azure AI Services by deploying both versions as separate endpoints. Then, split the traffic between them, using Azure tools like Load Balancer.

    Track performance metrics, like response time and user feedback. After collecting enough data, compare the results to see which model performs better and use that version going forward.

    I hope this helps. Thank you.

    If this answers your query, do click Accept Answer and Yes for was this answer helpful.

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.