Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,917 questions
ML Studio endpoint problem after Aks/VM Restart
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 89%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
Luciano
6
Reputation points
After restarting VM's associated to AKS created with to ML Studio deployed the ws endpoint gets unreachable.
Try several time to restart but it doesn't work.
Any idea? Any workaround besides re-create the cluster?
{count} votes
-
romungi-MSFT 46,141 Reputation points Microsoft Employee2020-11-17T08:24:36.107+00:00 @Luciano Could you post any logs of your endpoint deployment? Are you using an existing AKS cluster and restarting it from Azure portal or is this a inference cluster created from Azure ML portal ml.azure.com?
-
Luciano 6 Reputation points
2020-11-17T11:56:09.967+00:00 Hi!.
I started the vm this morning again and it start it to work. I don't have logs beacause the last one is the one of this "normal" startup. Is there any other place to see log details?, besides that part, if it's help to diagnose it, we created the AKS Cluster with Azure ML Portal automatically and we shutdown VM's from the azure portal (as any vm). Is this a correct approach? -
romungi-MSFT 46,141 Reputation points Microsoft Employee
2020-11-17T13:56:34.303+00:00 It is preferable to detach the cluster and then shut it down instead of shutting down the VMs directly.
-
Luciano 6 Reputation points
2020-11-17T14:22:25.067+00:00 Is preferable or the correct way to do it?
And the other part, after dettach i have to shutdown manually the VM's and then start it and re-attach it to the AKS, Right?
Thanks.
-
romungi-MSFT 46,141 Reputation points Microsoft Employee
2020-11-18T06:45:12.573+00:00 @Luciano It is preferable and correct way to do it from the ML portal.
You can use scaling option to set the minimum number of VMs so you don't get billed when there are no requests for compute and your cluster will scale up/down automatically without the need to manually start/shut your VMs. -
Luciano 6 Reputation points
2020-11-26T02:08:56.14+00:00 We are having again the same issue. And this time we tried to detacha before shutdown VM and it doesn't work either. We startup the VM, attach again and we see the message "Provisionan State = Failed".
-
romungi-MSFT 46,141 Reputation points Microsoft Employee
2020-11-30T10:08:06.457+00:00 @Luciano Is it possible to provide details of your subscription and compute which are failing to start to our team from the feedback option on the top right corner of ML portal ml.azure.com?
Please select the last option to email you on the feedback so they can advise you on how to avoid such failures for future scenarios.
-
Luciano 6 Reputation points
2020-11-30T12:02:20.74+00:00 Yes. I've done that
Sign in to comment
Sign in to answer