how do I change the prompt loss weight when fine tuning through azure open ai?

Question

Specifically through python? Also what is the default value for prompt loss weight. I believe the other hyperparamaters you can change are learning_rate_multiplier, batch_size, and n_epochs, are there any others I can use to better adjust my model?

Answer 1

Hello Kayla,

Thanks for reaching out to us, the hyperparameters current supported for Python is as you mentioned, there is no other hyperparameters we can leverage in the fine-tuning step.

But you can analyze the result to see how to change the hyperparameters better

Look for your loss to decrease over time, and your accuracy to increase. If you see a divergence between your training and validation data, that may indicate that you are overfitting. Try training with fewer epochs, or a smaller learning rate multiplier.

I hope this helps. Let us know if you need more information.

Regards,

Yutong

-Please kindly accept the answer if you feel helpful to support the community, thanks a lot.