An Azure machine learning service for building and deploying models.
Getting net/http: request canceled while waiting for connection error when deploying a model trained with Azure Auto ML via Azure Studio
0
I am training a simple text classification model in Azure Auto ML. Got a decent model and now trying to deploy it. Deployment goes ok initially and I get:
Provisioning state: Succeeded
But then after a while, Provisioning state is returning an error: and no traffic as allocated to the model. ResourceNotReady: User container has crashed or terminated: Liveness probe failed: HTTP probe failed with statuscode: 502.
When I logs at deployment logs, I see following on top of bunch successful env creation logs:
Container events:
Kind: Pod, Name: LivenessProbeFailed, Type: Warning, Time: 2023-01-03T13:22:14.905684Z, Message: Liveness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: Killing, Type: Normal, Time: 2023-01-03T13:22:14.912478Z, Message: Stopping container inference-server
Kind: Pod, Name: ReadinessProbeFailed, Type: Warning, Time: 2023-01-03T13:22:18.653423Z, Message: Readiness probe failed: Get http://10.66.0.2:5001/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Looks to me like contained was deployed but is not reachable?
I tried various combinations of models and virtual machines, but it ends up in this error every time.