Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
2,645 questions
Chat in Azure AI Studio fails when max tokens set at 5000
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 69%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
matsuo_basho
10
Reputation points
I'm using the Chat playground in Azure AI Studio, running Meta Llama 3.1 70B instruct with a Serverless compute.
When I set the max tokens to 500, it works fine. However, when I set it to 5000, I get an error:
Request failed with status code 500. Clear the output to start a new dialog.
According to Meta's announcement, the model's context length is 128K, so why is there an issue?
Please let me know if this is some sort of MS quota issue and if so, whether it's something I have to request and wait for. Perhaps it's the fact that I'm running this on serverless compute.