Use NSG to restrict traffic to HDInsight on AKS

Important

This feature is currently in preview. The Supplemental Terms of Use for Microsoft Azure Previews include more legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability. For information about this specific preview, see Azure HDInsight on AKS preview information. For questions or feature suggestions, please submit a request on AskHDInsight with the details and follow us for more updates on Azure HDInsight Community.

HDInsight on AKS relies on AKS outbound dependencies and they're entirely defined with FQDNs, which don't have static addresses behind them. The lack of static IP addresses means one can't use Network Security Groups (NSGs) to lock down the outbound traffic from the cluster using IPs.

If you still prefer to use NSG to secure your traffic, then you need to configure the following rules in the NSG to do a coarse-grained control.

Learn how to create a security rule in NSG.

Outbound security rules (Egress traffic)

Common traffic

Destination	Destination Endpoint	Protocol	Port
Service Tag	AzureCloud.<Region>	UDP	1194
Service Tag	AzureCloud.<Region>	TCP	9000
Any	*	TCP	443, 80

Cluster specific traffic

This section outlines cluster specific traffic that an enterprise can apply.

Trino

Destination	Destination Endpoint	Protocol	Port
Any	*	TCP	1433
Service Tag	Sql.<Region>	TCP	11000-11999

Spark

Destination	Destination Endpoint	Protocol	Port
Any	*	TCP	1433
Service Tag	Sql.<Region>	TCP	11000-11999
Service Tag	Storage.<Region>	TCP	445

Apache Flink

None

Inbound security rules (Ingress traffic)

When clusters are created, then certain ingress public IPs also get created. To allow requests to be sent to the cluster, you need to allowlist the traffic to these public IPs with port 80 and 443.

The following Azure CLI command can help you get the ingress public IP:

aksManagedResourceGroup="az rest --uri https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.HDInsight/clusterpools/{clusterPoolName}\?api-version\=2023-06-01-preview --query properties.managedResourceGroupName -o tsv --query properties.aksManagedResourceGroupName -o tsv"

az network public-ip list --resource-group $aksManagedResourceGroup --query "[?starts_with(name, 'kubernetes')].{Name:name, IngressPublicIP:ipAddress}" --output table

Source	Source IP addresses/CIDR ranges	Protocol	Port
IP Addresses	<Public IP retrieved from above command>	TCP	80
IP Addresses	<Public IP retrieved from above command>	TCP	443