Hello I have a Azure ML workspace in "East US" using only private IP addresses. Whenever I request a new compute instance I receive the below error. I tried with the studio web page and powershell. Both have the same outcome. "Desired number of dedicated nodes could not be allocated" Here the powershell command: az ml compute create --name myinstanceaz4 --size Standard_D2 --identity-type SystemAssigned --type ComputeInstance --resource-group rg --workspace-name labdatascripting3 --vnet-name vnet --subnet newsubnet I took a look on Standard_D2 compute quota and I don't see any issue on it. Could you help ? Thanks!

Christian HUARD Greetings! I have a Azure ML workspace in "East US" using only private IP addresses. Whenever I request a new compute instance I receive the below error. I tried with the studio web page and powershell. Both have the same outcome. "Desired number of dedicated nodes could not be allocated" This error message usually occurs when there are not enough resources available in the region to fulfill your request. Since you are using only private IP addresses, it is possible that the issue is related to the virtual network configuration. Please make sure that your virtual network is properly configured to allow the creation of compute instances. You can check the virtual network configuration by going to the Azure portal and navigating to your virtual network. Also, please make sure that you have enough quota for the instance size in the region where you are trying to create the compute instance. You can check your quota by going to the Azure portal and navigating to the "Usage + quotas" section of your subscription. It's also worth noting that when you create a compute instance in a workspace with a private endpoint, the compute instance and compute cluster must be in the same Azure region as the workspace. please check the Limitations when Configuring a private endpoint for an Azure Machine Learning workspace I hope this helps. Do let know if you have any further questions or concerns.

While creating compute instance on Azure ML workspace I receive this error "Desired number of dedicated nodes could not be allocated"

Accepted answer

AshokPeddakotla-MSFT 34,016 Reputation points

2024-03-12T12:16:58.5366667+00:00

Christian HUARD Greetings!

I have a Azure ML workspace in "East US" using only private IP addresses. Whenever I request a new compute instance I receive the below error. I tried with the studio web page and powershell. Both have the same outcome. "Desired number of dedicated nodes could not be allocated"

This error message usually occurs when there are not enough resources available in the region to fulfill your request.

Since you are using only private IP addresses, it is possible that the issue is related to the virtual network configuration. Please make sure that your virtual network is properly configured to allow the creation of compute instances. You can check the virtual network configuration by going to the Azure portal and navigating to your virtual network.

Also, please make sure that you have enough quota for the instance size in the region where you are trying to create the compute instance. You can check your quota by going to the Azure portal and navigating to the "Usage + quotas" section of your subscription.

It's also worth noting that when you create a compute instance in a workspace with a private endpoint, the compute instance and compute cluster must be in the same Azure region as the workspace.

please check the Limitations when Configuring a private endpoint for an Azure Machine Learning workspace

I hope this helps. Do let know if you have any further questions or concerns.
Please sign in to rate this answer.
Christian HUARD 40 Reputation points

2024-03-12T13:44:23.51+00:00

Thank you to come back to me.

Subnet has 236 Available IP addresses.

Here are the current compute quotas. As you can see there is no standard d2 used so far.

The Azure Machine Learning workspace is there "East US" as the resource group. I didn't put the region in the powershell command. I'll fix that.

Christian HUARD 40 Reputation points

2024-03-12T14:09:05.2266667+00:00

Actually location field is forbidden when using powshell az to create a compute instance. So I cannot do any mistake here.

Christian HUARD 40 Reputation points

2024-03-12T14:51:26.33+00:00

Here another try without success:

az ml compute create --name myinstanceaz5 --size STANDARD_DS3_v2 --resource-group rg --workspace-name labdatascripting3 --identity user_assigned --user-assigned-identities "/subscriptions/5fade077-c954-4778-b08b-8262c14254f8/resourcegroups/rg/providers/Microsoft.ManagedIdentity/userAssignedIdentities/datalabmanagedid" --location "East US" --vnet-name vnet --subnet newsubnet --type ComputeInstance Field 'identity': This is an experimental field, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information. Warning: 'Location' is not supported for compute type computeinstance and will not be used. (BadRequest) {"id":"https://resourceprovider.batchai-eastus.svc/subscriptions/5fade077-c954-4778-b08b-8262c14254f8/providers/Microsoft.BatchAI/locations/eastus/operationresults/9b4b659f-b34c-4e36-b4d1-b540c54b8a3f","name":"9b4b659f-b34c-4e36-b4d1-b540c54b8a3f","status":"Failed","startTime":"2024-03-12T14:08:00.434Z","endTime":"2024-03-12T14:33:15.786Z","error":{"code":"AllocationFailed","message":"Desired number of dedicated nodes could not be allocated"}} Code: BadRequest Message: {"id":"https://resourceprovider.batchai-eastus.svc/subscriptions/5fade077-c954-4778-b08b-8262c14254f8/providers/Microsoft.BatchAI/locations/eastus/operationresults/9b4b659f-b34c-4e36-b4d1-b540c54b8a3f","name":"9b4b659f-b34c-4e36-b4d1-b540c54b8a3f","status":"Failed","startTime":"2024-03-12T14:08:00.434Z","endTime":"2024-03-12T14:33:15.786Z","error":{"code":"AllocationFailed","message":"Desired number of dedicated nodes could not be allocated"}}

AshokPeddakotla-MSFT 34,016 Reputation points

2024-03-14T05:25:29.2866667+00:00

Christian HUARD It seems that the error message "Desired number of dedicated nodes could not be allocated" is still appearing even though the subnet has enough available IP addresses and there are no quota limitations for the virtual machine size you are trying to use.

It is possible that the issue is related to resource availability in the region.

You can try creating the compute instance again after waiting for some time. If the issue persists, you can try creating the compute instance in a different region to see if the issue is specific to the East US region.

Do let me know if that helps.

AshokPeddakotla-MSFT 34,016 Reputation points

2024-03-27T03:22:00.5266667+00:00

Are you still blocked? Do let me know if you need any further help.

Christian HUARD 40 Reputation points

2024-03-27T08:10:11.6233333+00:00

I still have the issue. Nevertheless, I discovered the "Desired number of dedicated nodes could not be allocated" is used in multiple cases. One of them is when a FW is preventing outgoing flows to go through. I'm currently opening these flows one by one. Crossing fingers I'll make it work.

AshokPeddakotla-MSFT 34,016 Reputation points

2024-03-27T08:51:05.0866667+00:00

Sure. Please share the outcome.

Christian HUARD 40 Reputation points

2024-03-27T10:01:32.5533333+00:00

I do confirm. Preventing network flows to move to internet produces this error. From the below microsoft url, I have added flows towards AzureMachineLearning, AzureResourceManager, AzureActiveDirectory and BatchNodeManagement. Following to this it does work.

https://learn.microsoft.com/en-us/azure/machine-learning/how-to-access-azureml-behind-firewall?view=azureml-api-2&tabs=ipaddress%2Cpublic

AshokPeddakotla-MSFT 34,016 Reputation points

2024-03-29T06:43:40.3966667+00:00

Christian HUARD Thanks for the confirmation. Glad to hear your issue is resolved and appreciate for sharing the resolution.

Do let me know if you have any further queries.

If your issue is resolved, do click Accept Answer and Yes for was this answer helpful. So that we can close this thread.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

While creating compute instance on Azure ML workspace I receive this error "Desired number of dedicated nodes could not be allocated"

0 additional answers

Your answer