"ResourceNotFoundError: The request did not have a subscription or a valid tenant level resource provider" when creating Serverless AKS

Siegfried Heintze 1,861 Reputation points
2020-12-12T20:15:15.157+00:00

I'm trying to follow theses instructions to reduce the costs of an AKS cluster and I'm getting the error in the title.

Since this command (step 1) did not work I did the register command below.

az provider list --query "[?contains(namespace,'Microsoft.ContainerInstance')]" -o table  
] was unexpected at this time.  
C:\Users\shein>  "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\\..\python.exe" -IBm azure.cli   
provider list --query [?contains(namespace,'Microsoft.ContainerInstance')] -o table  

I then switched from powershell to cygwin bash.

I just did a
az provide list
and did a manual search and concluded the next step was necessary:

az.cmd provider register --namespace Microsoft.ContainerInstance  

Since I'm using my existing resource group I did not create a new one.

I created a new service principle. The directions did not specify a name and I specified a name:

az.cmd ad sp create-for-rbac --skip-assignment --name ClusterServicePrincipal-todo-0003-skip-assignment  

I created some environment variables:

AZ_NETWORK_NAME=ToDoVNet  
AZ_NETWORK_BASE_ADDRESS=10.0.0.0   
AZ_NETWORK_SUBNET_NAME=ToDoAKSSubnet  
AZ_NETWORK_SUBNET_BASE_ADDRESS=10.240.0.0  

I created a virtual network:

az.cmd network vnet create --resource-group $AZ_RESOURCE_GROUP_NAME --name   
$AZ_NETWORK_NAME --address-prefixes $AZ_NETWORK_BASE_ADDRESS/8 --subnet-name   
$AZ_NETWORK_SUBNET_NAME --subnet-prefix $AZ_NETWORK_SUBNET_BASE_ADDRESS/16  

The results looked fine. I'm not sure about security and posting the JSON results of this (and other) command... What sorts of things do I not want to post? I think I don't want to post AppIDs and security principle secrets... what else?

Ok more env vars:

AZ_NETWORK_VNODE_SUBNET_NAME=ToDoVirtualNodeSubnet  
AZ_NETWORK_SUBNET_VNODE_BASE_ADDRESS=10.241.0.0  

Create another virtual subnet:

az.cmd network vnet subnet create --resource-group $AZ_RESOURCE_GROUP_NAME --vnet-name $AZ_NETWORK_NAME --name $AZ_NETWORK_VNODE_SUBNET_NAME --address-prefixes $AZ_NETWORK_SUBNET_VNODE_BASE_ADDRESS/16  

The resulting JSON looked fine...

It looks like the purpose of this command it to fetch the network name -- but I already know that!

az.cmd network vnet show --resource-group $AZ_RESOURCE_GROUP_NAME --name $AZ_NETWORK_NAME --query id -o tsv      

I get

/subscriptions/<looks like a UUID>/resourceGroups/rg-todo-temp/providers/Microsoft.Network/virtualNetworks/ToDoVNet  

So that is that number that looks like a UUID? Is this the APPID of the network I just created? I assigned this to a new environment variable called AZ_NETWORK_APPID.

az.cmd role assignment create --assignee $AZ_NETWORK_APPID --scope $AZ_NETWORK_NAME --role Contributor   
ResourceNotFoundError: The request did not have a subscription or a valid tenant level resource provider.  

OK, the directions mentioned that APPID in the last two steps... OK, I'll try again and use the APPID of the newly created service principal:

az.cmd role assignment create --assignee $AZ_SP_APPID --scope $AZ_NETWORK_NAME --role Contributor   
ResourceNotFoundError: The request did not have a subscription or a valid tenant level resource provider  
  
  

No luck... What am I doing wrong?

Oh, I forgot to mention: I'm running azure-cli 2.14.2 and the directions say 2.0.49 or later... looks like I'm good.

Tue Dec 14 2020 evening update:

Yahoo! It works.... More questions:

(1) Is it possible to specify "--nodecount 0" and have 100% serverless (virtual) AKS cluster so there are no compute charges when it is idle?
(2) Will new virtual nodes be added as needed as I create more deployments of various applications?
(3) What is special about 172.17.0.1 (docker-bridge-address)? Is this specific to Docker?
(4) Where do the values for the 10.0.0/16 and 10.0.0.10 for the service-cidr & dns-service come from? Could I have just as easily have used 192.168.0.0/16 and 192.168.0.10?
(5) Where are the parameters that control how long a virtual node is idle before there are no more compute charges?
(6) Can I specify different node sizes (using "--node-vm-size -s" perhaps?) for virtual and non-virtual nodes?
(7) Is there any thing special about aci-hello-world images that allows it to run in a virtual node? I'm worried because I'm not familiar with the ACI feature...
(8) I believe the only thing I have to do to my ASP.NET web apps is to deploy them with the new magic nodeSelector and tolerations yaml clauses? Do I just cut and paste this fragment into my yaml deployment files and then it will run on the virtual nodes?

  nodeSelector:  
    kubernetes.io/role: agent  
    beta.kubernetes.io/os: linux  
    type: virtual-kubelet  
  tolerations:  
  - key: virtual-kubelet.io/provider  
    operator: Exists  
  - key: azure.com/aci  
    effect: NoSchedule  

Thanks

Siegfried

Azure Container Instances
Azure Container Instances
An Azure service that provides customers with a serverless container experience.
635 questions
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,852 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. shiva patpi 13,131 Reputation points Microsoft Employee
    2020-12-15T06:14:33.56+00:00

    Hello @Siegfried Heintze ,
    Thanks for your query and the detailed steps. I was able to repro your issue in my local environment.

    az role assignment create --assignee <appid> --scope myvnet --role Contributor

    In the above command
    appid - we need to provide the appid from the output of
    az ad sp create-for-rbac --skip-assignment

    For example: When we run the command

    az ad sp create-for-rbac --skip-assignment

    I will get below output:
    {
    "appId": "c8a3e4a9-26fa-4d71-b1d3-b9219d7a4cd8",
    "displayName": "azure-cli-2020-12-15-05-49-05",
    "name": "http://azure-cli-2020-12-15-05-49-05",
    "password": "pwd",
    "tenant": "tenantid"
    }

    grab the appId and pass it in <appid>
    --scope field should refer to the entire uri path which looks like below (it should have the subid , resourcegroup , vnetname etc

    /subscriptions/subid/resourceGroups/testvirtualnodes/providers/Microsoft.Network/virtualNetworks/myvnet (output of the command az network vnet show --resource-group testvirtualnodes --name myVnet --query id -o tsv)

    If we just do
    az role assignment create --assignee c8a3e4a9-26fa-4d71-b1d3-b9219d7a4cd8 --scope myvnet --role Contributor
    You will get the below error , which clearly says that the subid does not exist. The command is looking for the vnetid under the subid.

    ResourceNotFoundError: The request did not have a subscription or a valid tenant level resource provider.

    To Summarize , you did everything correct except
    $AZ_NETWORK_APPID = "This is not the Network AppID , it is the AppID of the output of the command: az ad sp create-for-rbac --skip-assignment"
    --scope is expecting the complete output of az network vnet show --resource-group testvirtualnodes --name myVnet --query id -o tsv

    az.cmd role assignment create --assignee $AZ_NETWORK_APPID --scope $AZ_NETWORK_NAME --role Contributor

    I would suggest , if you can follow the document https://learn.microsoft.com/en-us/azure/aks/virtual-nodes-cli
    you should be good.

    Hope the above explanation helps . If yes , kindly "Accept the Answer".

    0 comments No comments