"ResourceNotFoundError: The request did not have a subscription or a valid tenant level resource provider" when creating Serverless AKS

Siegfried Heintze 1,861 Reputation points

I'm trying to follow theses instructions to reduce the costs of an AKS cluster and I'm getting the error in the title.

Since this command (step 1) did not work I did the register command below.

az provider list --query "[?contains(namespace,'Microsoft.ContainerInstance')]" -o table  
] was unexpected at this time.  
C:\Users\shein>  "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\\..\python.exe" -IBm azure.cli   
provider list --query [?contains(namespace,'Microsoft.ContainerInstance')] -o table  

I then switched from powershell to cygwin bash.

I just did a
az provide list
and did a manual search and concluded the next step was necessary:

az.cmd provider register --namespace Microsoft.ContainerInstance  

Since I'm using my existing resource group I did not create a new one.

I created a new service principle. The directions did not specify a name and I specified a name:

az.cmd ad sp create-for-rbac --skip-assignment --name ClusterServicePrincipal-todo-0003-skip-assignment  

I created some environment variables:


I created a virtual network:

az.cmd network vnet create --resource-group $AZ_RESOURCE_GROUP_NAME --name   
$AZ_NETWORK_NAME --address-prefixes $AZ_NETWORK_BASE_ADDRESS/8 --subnet-name   

The results looked fine. I'm not sure about security and posting the JSON results of this (and other) command... What sorts of things do I not want to post? I think I don't want to post AppIDs and security principle secrets... what else?

Ok more env vars:


Create another virtual subnet:

az.cmd network vnet subnet create --resource-group $AZ_RESOURCE_GROUP_NAME --vnet-name $AZ_NETWORK_NAME --name $AZ_NETWORK_VNODE_SUBNET_NAME --address-prefixes $AZ_NETWORK_SUBNET_VNODE_BASE_ADDRESS/16  

The resulting JSON looked fine...

It looks like the purpose of this command it to fetch the network name -- but I already know that!

az.cmd network vnet show --resource-group $AZ_RESOURCE_GROUP_NAME --name $AZ_NETWORK_NAME --query id -o tsv      

I get

/subscriptions/<looks like a UUID>/resourceGroups/rg-todo-temp/providers/Microsoft.Network/virtualNetworks/ToDoVNet  

So that is that number that looks like a UUID? Is this the APPID of the network I just created? I assigned this to a new environment variable called AZ_NETWORK_APPID.

az.cmd role assignment create --assignee $AZ_NETWORK_APPID --scope $AZ_NETWORK_NAME --role Contributor   
ResourceNotFoundError: The request did not have a subscription or a valid tenant level resource provider.  

OK, the directions mentioned that APPID in the last two steps... OK, I'll try again and use the APPID of the newly created service principal:

az.cmd role assignment create --assignee $AZ_SP_APPID --scope $AZ_NETWORK_NAME --role Contributor   
ResourceNotFoundError: The request did not have a subscription or a valid tenant level resource provider  

No luck... What am I doing wrong?

Oh, I forgot to mention: I'm running azure-cli 2.14.2 and the directions say 2.0.49 or later... looks like I'm good.

Tue Dec 14 2020 evening update:

Yahoo! It works.... More questions:

(1) Is it possible to specify "--nodecount 0" and have 100% serverless (virtual) AKS cluster so there are no compute charges when it is idle?
(2) Will new virtual nodes be added as needed as I create more deployments of various applications?
(3) What is special about (docker-bridge-address)? Is this specific to Docker?
(4) Where do the values for the 10.0.0/16 and for the service-cidr & dns-service come from? Could I have just as easily have used and
(5) Where are the parameters that control how long a virtual node is idle before there are no more compute charges?
(6) Can I specify different node sizes (using "--node-vm-size -s" perhaps?) for virtual and non-virtual nodes?
(7) Is there any thing special about aci-hello-world images that allows it to run in a virtual node? I'm worried because I'm not familiar with the ACI feature...
(8) I believe the only thing I have to do to my ASP.NET web apps is to deploy them with the new magic nodeSelector and tolerations yaml clauses? Do I just cut and paste this fragment into my yaml deployment files and then it will run on the virtual nodes?

    kubernetes.io/role: agent  
    beta.kubernetes.io/os: linux  
    type: virtual-kubelet  
  - key: virtual-kubelet.io/provider  
    operator: Exists  
  - key: azure.com/aci  
    effect: NoSchedule  



Azure Container Instances
Azure Container Instances
An Azure service that provides customers with a serverless container experience.
635 questions
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,852 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. shiva patpi 13,131 Reputation points Microsoft Employee

    Hello @Siegfried Heintze ,
    Thanks for your query and the detailed steps. I was able to repro your issue in my local environment.

    az role assignment create --assignee <appid> --scope myvnet --role Contributor

    In the above command
    appid - we need to provide the appid from the output of
    az ad sp create-for-rbac --skip-assignment

    For example: When we run the command

    az ad sp create-for-rbac --skip-assignment

    I will get below output:
    "appId": "c8a3e4a9-26fa-4d71-b1d3-b9219d7a4cd8",
    "displayName": "azure-cli-2020-12-15-05-49-05",
    "name": "http://azure-cli-2020-12-15-05-49-05",
    "password": "pwd",
    "tenant": "tenantid"

    grab the appId and pass it in <appid>
    --scope field should refer to the entire uri path which looks like below (it should have the subid , resourcegroup , vnetname etc

    /subscriptions/subid/resourceGroups/testvirtualnodes/providers/Microsoft.Network/virtualNetworks/myvnet (output of the command az network vnet show --resource-group testvirtualnodes --name myVnet --query id -o tsv)

    If we just do
    az role assignment create --assignee c8a3e4a9-26fa-4d71-b1d3-b9219d7a4cd8 --scope myvnet --role Contributor
    You will get the below error , which clearly says that the subid does not exist. The command is looking for the vnetid under the subid.

    ResourceNotFoundError: The request did not have a subscription or a valid tenant level resource provider.

    To Summarize , you did everything correct except
    $AZ_NETWORK_APPID = "This is not the Network AppID , it is the AppID of the output of the command: az ad sp create-for-rbac --skip-assignment"
    --scope is expecting the complete output of az network vnet show --resource-group testvirtualnodes --name myVnet --query id -o tsv

    az.cmd role assignment create --assignee $AZ_NETWORK_APPID --scope $AZ_NETWORK_NAME --role Contributor

    I would suggest , if you can follow the document https://learn.microsoft.com/en-us/azure/aks/virtual-nodes-cli
    you should be good.

    Hope the above explanation helps . If yes , kindly "Accept the Answer".

    0 comments No comments