Azure AI Studio model deployment issue - HTTP 424 Error

Ulan Yisaev 25

Hi everyone,

I have been successfully using the Cohere-command-r-plus model deployed in Azure AI Studio for over a month. However, today I encountered an issue where my app returns the following error:

[_send_single_request] INFO HTTP Request: POST https://cohere-command-r-plus-XXXX-serverless.swedencentral.inference.ai.azure.com/v1/chat "HTTP/1.1 424 Failed Dependency"

I also tested the model in the Chat playground and received this error:

Request failed with status code 424. Clear the output to start a new dialog.

Has anyone else experienced this issue? Any insights or solutions would be greatly appreciated.

Thank you!

Luca Stosch 10 Reputation points

2024-05-27T13:59:42.4966667+00:00

Same here, using the API yields "no healthy upstream" response.
romungi-MSFT 43,696 Reputation points Microsoft Employee

2024-05-27T14:15:16.8666667+00:00

@Ulan Yisaev 424 error suggests something is wrong with the model. Although you are using Azure AI studio, this deployment is essentially an online endpoint in your project's ML workspace. The ML workspace under ml.azure.com should show this deployment and its related details. See this page on what different error codes mean for online endpoints.

I would suggest to retry the scenario after some time to check if the error goes away or try a new deployment and check if the same error is seen in new deployments. Thanks!!
Ulan Yisaev 25 Reputation points

2024-05-27T15:39:52.9966667+00:00

@romungi-MSFT , thanks for the hint! I’ll check the ML workspace for more details. I tried finding the model logs but was unsuccessful, so this is helpful. I also already tried deleting and redeploying the model, but the issue persists.
Ulan Yisaev 25 Reputation points

2024-05-27T16:18:38.72+00:00
@romungi-MSFT , I followed the page you mentioned and found this about the 424 error:

If your model container returns a non-200 response, Azure returns a 424.

I used the inference sample from the Cohere model page. The headers showed:

The request failed with status code: 424 Content-Length: 19 Content-Type: text/plain ms-azureml-model-error-reason: model_not_ready x-ms-client-request-id: a3c73060-8fb2-4f19-8045-9f896b7fb7b7 Request-Context: appId=cid-v1:bf86afeb-6735-4630-a419-996fd98375c2 azureml-model-session: dep-sdc-coh-cmdrpl-240404061618 azureml-model-group: offer-cohere-cmd-r-plus-paygo x-request-id: 8b69761b-89ea-42de-bd50-5989af710de8 Date: Mon, 27 May 2024 16:11:39 GMT Connection: close no healthy upstream

I also checked ml.azure.com but didn't find specific logs for the deployed model, only the deployment under "Endpoints | Serverless endpoints" that I already have in ai.azure.com. So, the headers showed ms-azureml-model-error-reason: model_not_ready and no healthy upstream. It seems the problem might be on Azure's side because other deployed models work well, except for models from Cohere.
Ulan Yisaev 25 Reputation points

2024-05-27T19:42:56.4133333+00:00

Update: I deployed the same model in a different AI Studio project in the East US 2 region (original deployment was in Sweden Central), and it works there. This indicates that the issue is specific to the Sweden Central region with Cohere models. Can MS diagnose this issue, please?
romungi-MSFT 43,696 Reputation points Microsoft Employee

2024-05-28T03:55:06.68+00:00

@Ulan Yisaev Thanks for the debugging and updates to check the issue. For the issue with your region, I would advise to report the same through Azure ML portal using feedback smiley option on the top right corner or by raising an azure support case from azure portal. This will help the service team lookup your request ids and check for any issues. I am not part of the ML team nor have access to lookup the issues, I hope this helps!!

Share via

Azure AI Studio model deployment issue - HTTP 424 Error