@Tushar Gupta You can go to Models section of Azure Open AI Studio and select column options to add additional columns like Fine Tune, Completion etc. and then you can check the models with a green checkmark under Completion column to identify models supporting the completions API(See screenshot below)
Please let me know if you have any other questions.
Thanks
Saurabh
Please 'Accept as answer' and Upvote if it helped so that it can help others in the community looking for help on similar topics.