Default and minimum virtual machine size recommendations and capacity planning for HDInsight on AKS

Important

This feature is currently in preview. The Supplemental Terms of Use for Microsoft Azure Previews include more legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability. For information about this specific preview, see Azure HDInsight on AKS preview information. For questions or feature suggestions, please submit a request on AskHDInsight with the details and follow us for more updates on Azure HDInsight Community.

This article discusses default and recommended node configurations for Azure HDInsight on AKS clusters.

Cluster pools

For creating cluster pools, the currently available VM Options are F4s_v2, D4a_v4, D4as_v4, and E4s_v3.

Clusters

HDInsight on AKS currently supports Virtual Machines (VMs) from the Memory Optimized and General Purpose categories for cluster creation. Memory optimized VM sizes offer a high Memory-to-CPU ratio, which is great for relational database servers, medium to large caches, and in-memory analytics. General purpose VM sizes provide balanced Memory-to-CPU ratio and are ideal for testing and development, small to medium databases, and low to medium traffic web servers.

If your use case requires the usage of Memory Optimized VMs, the default recommendation is to use VMs from the families Eadsv5 or Easv5. For use cases requiring the usage of General Purpose VMs, we recommend VMs from the families Dadsv5 or Ddsv5.

If there is insufficient capacity/quota on the recommended VM Families, look for alternatives in the table that lists all the VM Families currently supported in HDInsight on AKS for cluster creation:

Type	Virtual machine family	Temp storage	Premium storage support
Memory optimized	Eadsv5	Yes	Yes
	Easv5	No	Yes
	Edsv5	Yes	Yes
	Edv5	Yes	No
	Easv4	Yes	Yes
	Eav4	Yes	No
	Edv4	Yes	No
General purpose	Dadsv5	Yes	Yes
	Ddsv5	Yes	Yes
	Ddv5	Yes	No
	Dasv4	Yes	Yes
	Dav4	Yes	No

The minimum VM specifications recommended (regardless of the chosen VM Family for the cluster) is 8vCPUs and 32 GiB RAM. Higher vCPU and GiB variants may be chosen as per the workload being processed.

Capacity planning

The Virtual Machines used in HDInsight on AKS clusters requires the same Quota as Azure VMs. This is unlike the original version of HDInsight, where users had to request a dedicated quota to create HDInsight clusters by selecting 'Azure HDInsight' from the quota selection dropdown as shown in the image. For HDInsight on AKS, customers need to select ‘Compute’ from the Quota selection dropdown in order to request extra capacity for the VMs they intend to use in their clusters. Find detailed instructions for increasing your quota here.