Edit

Share via


How-to: Increase Private Endpoint virtual network limits

Today, users are limited to deploying only 1,000 private endpoints within their virtual network. It's common for users to navigate around this limitation by implementing a Hub and Spoke model or a Mesh network. Doing so would make it possible to deploy extra private endpoints across peered virtual networks to temporarily surpass the per virtual network limit. However, scaling in this manner places users at risk of a silently enforced limitation. Whenever users surpass 4,000 private endpoints across their peered virtual networks, they put themselves at risk of connectivity issues and packet drops.

For users looking to surpass these current limits, we recommend upgrading to High Scale Private Endpoints. This feature increases standard limits to 5,000 private endpoints in a singular virtual network and 20,000 private endpoints across peered networks. This article details how to opt into this feature and provide extra considerations before enablement.

Note

This feature is currently in public preview and available in select regions. We recommend reviewing all considerations before enabling it for your subscription.

Prerequisites

Confirm if you need to upgrade

If you need more than 1,000 private endpoints in a single virtual network or encounter a max private endpoint limit error, consider upgrading to High Scale Private Endpoints.

For customers using a Hub and Spoke or Mesh topology, determine how many private endpoints are connected to your central virtual network containing client virtual machines. Use the provided ARG query to facilitate this process.

Resources

   | where subscriptionId == "\<yourSubscriptionIDHere>"

   | where type =~ 'Microsoft.Network/virtualnetworks'

   | project id, remoteVNetIds = properties.virtualNetworkPeerings

   | mv-expand remoteVNetIds

   | project id, remoteVNetId = tostring(remoteVNetIds.properties.remoteVirtualNetwork.id)

   | where isnotempty(remoteVNetId)

   | join kind=leftouter (

       Resources

           | where type =~ 'Microsoft.Network/privateEndpoints'

           | project id, subnetId = tostring(properties.subnet.id)

           | extend VNetId = split(subnetId ,'/subnets/')[0]

           | project id, VNetId = tostring(VNetId)

           | summarize Count = count() by VNetId) 
           on $left.remoteVNetId == $right.VNetId
           | extend Count = iff(isempty(Count), 0, Count)
           | summarize TotalRemotePE = sum(Count) by ['id']

   | join kind=leftouter (

       Resources

           | where type =~ 'Microsoft.Network/privateEndpoints'

           | project id, subnetId = tostring(properties.subnet.id)

           | extend VNetId = split(subnetId ,'/subnets/')[0]

           | project id, VNetId = tostring(VNetId)

           | summarize Count = count() by VNetId) 

           on $left.id == $right.VNetId

          | extend TotalPE = iff(isempty(Count), 0, Count) + TotalRemotePE

| project VNetId = id, TotalPE

| order by TotalPE desc

| order by ['VNetId'] asc

Enable High Scale Private Endpoints

To enable this feature, configure Private Endpoint virtual network Policies. We recommend enabling this property for all virtual networks you want to include in this feature and for all connected compute virtual networks in peering scenarios.

Warning

Upgrading or downgrading this feature triggers a platform update and results in a one-time connection reset. We recommend performing this action during a maintenance window.

$vnetName = "myVirtualNetwork"
$resourceGroupName = "myResourceGroup"
$vnet = Get-AzVirtualNetwork -ResourceGroupName $resourceGroupName -Name $vnetName

$vnet.PrivateEndpointVNetPolicies = "Basic"
$vnet | Set-AzVirtualNetwork

Validate configuration

To validate the configuration, verify all necessary properties are set correctly. You can do this by checking the following:

  1. In the search box at the top of the portal, enter Virtual network. Select Virtual networks.

  2. Select myVNet.

  3. In settings of myVNet, select Subnets.

  4. Select your subnet.

  5. In the Edit subnet pane, under Network Policy for Private Endpoints, confirm Route Table is selected.

  6. In the virtual network overview page, select JSON view in the top right corner.

  7. In the Resource JSON pane, select the latest API Version.

  8. Validate that the virtual network property privateEndpointVNetPolicies is set to Basic.

  9. Confirm that you can deploy more than 1,000 private endpoints in the respective virtual network.

Additional Considerations

  • Upgrading or downgrading this feature triggers a platform update and results in a one-time connection reset of all long-running private endpoint connections. We recommend configuring High Scale Private Endpoints during a maintenance window.

  • To downgrade from this feature, reduce the total private endpoint count in your virtual network to the limit before the feature was enabled.

  • Monitoring Bytes In / Out will no longer be available on all high scale private endpoints.

  • On-premises private endpoint traffic is now billed as an aggregate on your gateway virtual network. Previously, it was shown on the private endpoint resource in your billing cost center. This change doesn't affect your total bill.

Limitations

Limit Description
Subscription must be enabled before enabling High Scale Private Endpoints. Enabling Private Endpoint virtual network Policies before allow listing subscription feature flag requires a reconfiguration.
Swift based virtual machines aren't supported. Swift based virtual machines deployed within a High Scale Private Endpoint virtual network aren't supported with this feature.
Feature currently available in select regions. West Central US
UK South
East Asia
US East
US North

Next Steps

In this article, you learned how to enable High Scale Private Endpoints and the considerations that come with it. For more information on Azure Private Link, see the following articles: