Generating documentation of software project from Models in Azure OpenAI Service.

Question

Hello Microsoft team,

I have a goal to generate documentation of an entire software project written in c#. The text content generated from the model should be stored inside an .md file format. I am using the API of GPT-35-turbo-16k model from Azure OpenAI Service. I wrote the code that reads all the file content from various code files inside directories and then pass the code into prompt. But everytime I am getting an error: 'Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2023-07-01-preview have exceeded token rate limit of your current OpenAI S0 pricing tier. Please retry after 19 seconds' or an error like token limit is exceeded. How can I generate a documentation of the software application project that is within the token limit. Any help is much appreciated.

Answer 1

@Abhishek Narvekar There are various quotas and limits applicable to the Azure OpenAI Service and the error that you are getting is related to the token limits which are defined at a regional level by model for all services in a subscription in the official docs.

Firstly, you could check the quota for your deployment in the Quotas Page to ensure your model is assigned the maximum available quota.

Next, you should implement retry policies in your code so that it can work around these rate limits.

And finally, if you are already exhausting all available quota, you could try requesting for more. Do note that due to overwhelming demand, requests are being accepted and will be filled in the order they are received.