HDInsight cluster creation Fails

Question

{

"code": "InvalidNetworkConfigurationErrorCode",

"message": "Virtual Network configuration is incompatible with HDInsight requirements. Error: 'Cluster Validation timed out.', Please follow https://go.microsoft.com/fwlink/?linkid=853974 to fix it. There is a possibility that this issue occurred due to HDInsight's transition to Standard Load Balancer. Please refer to https://learn.microsoft.com/en-us/azure/hdinsight/load-balancer-migration-guidelines to recreate the cluster with standard load balancers if not done yet, or contact support for assistance if it remains unresolved."

}

Error code

InvalidNetworkConfigurationErrorCode

Error message

Virtual Network configuration is incompatible with HDInsight requirements. Error: 'Cluster Validation timed out.', Please follow https://go.microsoft.com/fwlink/?linkid=853974 to fix it. There is a possibility that this issue occurred due to HDInsight's transition to Standard Load Balancer. Please refer to https://learn.microsoft.com/en-us/azure/hdinsight/load-balancer-migration-guidelines to recreate the cluster with standard load balancers if not done yet, or contact support for assistance if it remains unresolved.

I am getting this error after creation of HDinsight cluster creation, What are the possible areas I need to look on to avoid these issues.

and further I couldn't able to raise support ticket on the console create case is not showing

Answer 1

@Kavin Kingslin
You are encountering the error "Virtual Network configuration is incompatible with HDInsight requirements" because the HDInsight cluster validation is timing out due to issues in the virtual network (VNet) configuration. This often happens when the VNet is either not using a Standard Load Balancer, has insufficient subnet size, or contains overly restrictive Network Security Groups (NSGs) or custom DNS settings that prevent the cluster from reaching required Azure services. Microsoft has recently transitioned HDInsight to exclusively use Standard Load Balancers, and any cluster deployed on a VNet without this configuration will fail to validate. To resolve the issue, we recommend recreating the VNet with a Standard Load Balancer, ensuring the subnet is at least /26, and verifying that there are no NSGs or DNS settings blocking outbound access.
You can refer to the official Microsoft documentation for step-by-step guidance on how to fix this.
https://learn.microsoft.com/en-us/azure/hdinsight/hadoop/troubleshoot-invalidnetworkconfigurationerrorcode-cluster-creation-fails

https://learn.microsoft.com/en-us/azure/hdinsight/load-balancer-migration-guidelines

If you are unable to raise a support request through the portal, please check if your Azure subscription has an active support plan and that your user account has the required Support Request Contributor role, as described.

https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request#prerequisites

I hope this information helps.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues