Share via

Getting net/http: request canceled while waiting for connection error when deploying a model trained with Azure Auto ML via Azure Studio

Jurijs Tolokoncevs 16 Reputation points
2023-01-03T20:34:14.923+00:00

0

I am training a simple text classification model in Azure Auto ML. Got a decent model and now trying to deploy it. Deployment goes ok initially and I get:

Provisioning state: Succeeded  

But then after a while, Provisioning state is returning an error: and no traffic as allocated to the model. ResourceNotReady: User container has crashed or terminated: Liveness probe failed: HTTP probe failed with statuscode: 502.

When I logs at deployment logs, I see following on top of bunch successful env creation logs:

Container events:  
Kind: Pod, Name: LivenessProbeFailed, Type: Warning, Time: 2023-01-03T13:22:14.905684Z, Message: Liveness probe failed: HTTP probe failed with statuscode: 502  
Kind: Pod, Name: Killing, Type: Normal, Time: 2023-01-03T13:22:14.912478Z, Message: Stopping container inference-server  
Kind: Pod, Name: ReadinessProbeFailed, Type: Warning, Time: 2023-01-03T13:22:18.653423Z, Message: Readiness probe failed: Get http://10.66.0.2:5001/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)  

Looks to me like contained was deployed but is not reachable?

I tried various combinations of models and virtual machines, but it ends up in this error every time.

Azure Machine Learning

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.