Phi-3 Serverless Deployment Just Stopped working

Amiya Patanaik 25 Reputation points
2024-06-26T05:54:17.2466667+00:00

I have been using multiple serverless deployments of foundation models on Azure AI Studio (East-2) specifically. All of a sudden the Phi-3 128K deployment stopped working, both through Langchain as well as in the playground within Azure AI Studio. When queried through langchain or CURL directly, the request times out after long while. When using the playground I get: Request failed with status code 424. Clear the output to start a new dialog.

The endpoint is working because if I supply a wrong key, it immediately responds with an invalid key response. I tried deleting the endpoint and redeploying it. Still does not work. It only happens with Phi-3 128K instruct model.

This is the endpoint: https://Phi-3-mini-128k-instruct-hxaff-serverless.eastus2.inference.ai.azure.com/v1/chat/completions

Screenshot 2024-06-26 at 1.50.35 PM

Azure AI services
Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
2,632 questions
{count} vote

Accepted answer
  1. navba-MSFT 20,560 Reputation points Microsoft Employee
    2024-06-28T04:34:13.2333333+00:00

    @Amiya Patanaik I am glad that it is working fine now. I had involved the Product Owners to look into this issue in the background and address this.

    .

    If you encounter the same issue in future, please feel free to reach out to me. I would be happy to assist.

    **

    Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.


0 additional answers

Sort by: Most helpful