Custom Translator : Baseline score vs BLEU score

Brian Lam 0

How to train a new model on top of the baseline model? I found that the baseline score is always higher than my new model, even if I use some simple sentences on my train model.

Thanks.

romungi-MSFT 41,961 Reputation points Microsoft Employee

2023-02-02T07:53:31.37+00:00

Brian Lam you can view the model's BLEU score and baseline model BLEU score on the model details page. The same set of test data is used to generate both the model's BLEU score and the baseline BLEU score. Since you are seeing higher score for baseline model, I think the baseline model is better suited in your case instead of the custom model. If you would like to continue using the custom model, new data might need to be added for training a new custom model. I hope this helps!!
Brian Lam 0 Reputation points

2023-02-03T01:54:38.3533333+00:00

Thanks for your prompt reply.

On the other hand, is there a separate baseline model for each domain (generic, custom, business...etc)? How can I get the BLEU score even if I only apply the vocabulary for training? Is this correct if my training sentences are vocabulary lists and not full sentences?
Brian Lam 0 Reputation points

2023-02-03T01:57:03.8933333+00:00

Thank you for your prompt reply.

On the other hand, does each domain (general, custom, business ... etc.) have an independent baseline model? How can I have a BLEU score even I only applied glossary for training? Is this correct if my training sentences are vocabulary lists and not full sentences? Thanks.