Bare metal server for HCI Stack + AKS Cluster for test environment

Question

We are trying to explore the Azure HCI stack capabilities and having nested virtualization (VM in Azure with 16vCPU + 128 GB RAM). We are have installed Hypervisor on it and able to create a VM and install Azure Stack HCI. When we tried to install the AKS cluster then we are getting error "Insufficient resources on the HCI cluster." Looks like for AKS we need minimum 8 vCPU and 256GB RAM https://learn.microsoft.com/en-us/azure-stack/aks-hci/system-requirements?WT.mc_id=Portal-Microsoft_Azure_Support#compute-requirements and also recommended to use Physical hardware.

When we tried on the Azure portal and Stack HCI Catalog, we are not getting the hardware cost for the physical server.

Can some one help us on the minimal hardware requirement for HCI Stack + AKS Cluster and pricing?

Regards,
Sakthi

Answer 1

Hi - for evaluation purposes, AKS-HCI will install with less than 256GB RAM - inside your Azure VM, can you provide details of your nested HCI nodes? How many vCPUs, how much memory, disk etc?

It's not recommended to run AKS-HCI on a nested Azure Stack HCI cluster in Azure, due to the 3 levels of nesting (Azure hypervisor, Azure VM hypervisor, AzSHCI hypervisor), however it is possible.

In a production environment, the minimal configuration that is recommended is an Azure Stack HCI cluster (minimum 2 nodes) with at least 16 cores and 256GB RAM per node, but AKS-HCI will install and run fine on less - it's largely driven by your workloads, the number of worker nodes etc.

Your OEM of choice should be able to provide pricing for a solution.

Answer 2

Hello Matt, thanks for your quick response.

Here is my configuration:

1. Azure VM with 16 vCPU and 128 GB RAM as Host VM.
2. Installed Hpervisor on this VM.
3. Created two HCI Node with 8vCPU and 24GB RAM with 4HDD of size 100GB
4. I've also AD and Windows Admin running on the host VM.

Please let me know, raising the host VM RAM size to 512 and HCI node to 256 will help us to install AKS on the HCI node.

This is only for evaluation or test environment. Not for production.

Answer 3

Hey - so those HCI sizes should allow you to install AKS-HCI, at least with PowerShell. Windows Admin Center may check for higher levels of memory, but technically, it only needs 8GB RAM (plus small overhead) to deploy the AKS-HCI management cluster onto your HCI cluster, and from there, additional memory for the target cluster that you deploy - this includes a load balancer, control plane and at least 1 worker node. Whilst you can make this K8s VMs small, you're still looking at ~18GB RAM for those 3 VMs. These would be spread over your HCI cluster nodes.

Did you deploy from the latest Azure Stack HCI Azure ARM template? If so, your HCI nested nodes would have already been deployed and running for you. If you didn't deploy with the new template, I would redeploy with that one: https://github.com/Azure/AzureStackHCI-EvalGuide/blob/main/deployment/steps/1_DeployAzureVM.md

Also, did you register your Azure Stack HCI cluster with Azure before trying to install AKS-HCI?

Thanks,
Matt

Answer 4

Hello Matt,

The above link 1_DeployAzureVM.md talks about installing the Host VM using the ARM template. Looks like I need to install the HCI cluster, management node, etc again and then try the AKS cluster setup.

As we created the base VM with the same configuration mentioned in the ARM template, does it make any difference? I had registered the HCI Cluster to Azure portal before installing the AKS.

What is the correct steps to configure AKS cluster on the nested virtualization env?