This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 89%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETB%3C/text%3E%3C/svg%3E)
For an AKS cluster with API Server VNet Integration (Preview) enabled and access to the API server via a VPN Gateway established, using the feature API server using authorized IP address ranges breaks any access coming through the VPN Gateway.
We have an AKS cluster configured with API Server VNet Integration (Preview). Cluster mode is set to "public".
We would like to provide access to the API server to admin users via a P2S VPN Gateway using the API server internal VIP and peering the VPN Gateway VNet and the cluster VNet. This works great.
Now to reduce the attack surface, we would also like to limit access from the internet down to to Azure DevOps agents (Microsoft-hosted) where we are running our pipelines. Once we start entering authorized IP ranges, access to the API server internal VIP via the VPN Gateway breaks. We added
to the authorized IP ranges, but still cannot connect.
What IP range(s) do we need to authorize in this setup? Is this combination of features designed to work together?
Thank you for any insights.
(N.B. We are aware of Azure DevOps self-hosted agents as a possible solution, but one we would like to avoid for now to keep operational complexity low.)
Hello Tobias Babin
From the above diagram, only AzureDevOps agent IP are needed for authorized IP ranges. All others are using the internal IP to access the cluster.
Hope this helps.
You cannot use the authorised IP ranges when the API server is injected into the vNet because the API server is no longer available outside of the network it is injected into, as per the docs:
To access the API server from outside the cluster network, utilize either VNet peering or AKS run command.
Thanks for getting back to us so quickly. Yes, we have seen that section in the docs and it holds true for the VNet integrated cluster in private cluster mode. However, with private cluster mode disabled the API server still does have the public endpoint and the "Authorized IP ranges" option is available (it is disabled in private cluster mode).
The setup would look like this:
According to our tests it seems that the Authorized IP ranges, once active, prevent any traffic coming through the VPN gateway from getting through to the API server even if we allow the VPN gateway VNet CIDR range and the gateway itself.
So maybe I can make the question more specific:
For a cluster with API VNet integration enabled and set to public cluster mode, can I use Authorized IP ranges to restrict API server access to a combination of public IP ranges and traffic coming in via a VPN gateway VNet?
Hello Tobias Babin
Apologies for delayed response on this.
From apiserver perspective, as long as its client ip (not the original client ip) is added to the AuthorizedIP config, the connection would work.
If this is not working, then I would recommend you to open a support case to get this checked. Please let me know if you need help raising a support case.
Hello @vipullag-MSFT , thanks a lot for your reply and no worries about the delay, I am aware there is no guaranteed reply in this forum.
We shall be happy to open a support case. Maybe one more thing before we do that: looking at the diagram of our setup, which IP or IP ranges do you think should be the client IP to add to the authorized IP ranges?
Hello @vipullag-MSFT ,
Following your confirmation, we re-examined our setup and it now works. Turns out we got a setting wrong in the VNet peering between the VPN Gateway VNet and the AKS Cluster VNet. For anyone reading this in the future, make sure to have these options activated in the peerings:
I can now access the AKS API server via kubectl from my workstation using the regular kubeconfig generated via the az aks get-credentials command when having the P2S VPN connection active. 🎉
To make that work, I need to tweak my local DNS and associate the hostname of the API server to the frontend IP of the internal LoadBalancer, e.g. by modifying the hosts file. It's a bit hacky but does the trick.
To make the API server accessible for our Microsoft-hosted Azure DevOps (ADO) agents, we configured the IP ranges published by Microsoft (see here) into the "Authorized IP ranges" for the AKS cluster.
We are aware this will still allow any agents run by other ADO organizations to physically reach the API server, but keep out the rest of the internet - a compromise between security and effort saved, i.e. not having to maintain self-hosted ADO agents.
The last thing we are struggling with is that the total number of IP ranges required to configure for the ADO agents of our geography exceeds the maximum of 200 items which can be entered as authorized IP ranges. We worked around this by manually condensing the list, but that's far from perfect especially since it is dynamic.
Is there any way to have that limit increased?