Azure Operator Nexus limits and quotas
This document provides an overview of the resource limits that apply to the components used in the Nexus solution, encompassing the resources created within Azure cloud and in on-premises instance. It outlines the specific limitations and restrictions that operators should be aware of when deploying and managing the Nexus instance across these environments.
Understanding these resource limits is crucial for optimizing resource utilization and ensuring the smooth operation of the Nexus solution. It's also essential to be aware of any restrictions or constraints that may apply to the creation of these resources to ensure compliance and avoid any potential issues or disruptions.
Azure-specific requirements
In addition to the hardware and software resources available in the customer's on-premises instance, the Azure Operator Nexus also incorporates essential components that must be created within the Azure cloud environment. These components, such as the Network Fabric Controller and Cluster Manager, are integral to the overall functionality and management of the Azure Nexus Operator on-premises instance. These controllers are built utilizing a diverse range of Azure services. Prior to creating these resources in the Azure cloud environment, Operators should thoroughly review and confirm the specific limits and quotas that are in effect. It's crucial to ensure compliance with these limitations to enable successful deployment and operation of the Azure Operator Nexus solution.
Some of these Azure services have adjustable limits. When a service doesn't have adjustable limits, the default and the maximum limits are the same. The limit can be raised above the default limit but not above the maximum limit. If you want to raise the limit or quota above the default limit, open an online customer support request.
The terms soft limit and hard limit often are used informally to describe the current, adjustable limit (soft limit) and the maximum limit (hard limit).
Some of these limits also apply at Azure region level.
Note
It’s highly recommended that Operators create and use a separate Azure subscription for Azure Nexus Operator and not mix it up with other Azure use cases under the same subscription.
Network Fabric
The creation of the Network Fabric related resources is subject to the following resource limits:
| Resource Type | Notes |
|---|---|
| Network Fabric Controllers | Today, its creation depends on underlying Azure components as mentioned in the supporting table under section "Other Azure Resources" |
| Network Fabrics | Up to 20 Network Fabric resources per Network Fabric Controller [To be updated] |
| Network Devices | Up to BOM-specified Network devices per Network Fabric |
| Network Racks | Up to BOM-specified racks per Network Fabric |
| Layer 2 Isolation domains | 3500 isolation domains per Nexus instance |
| Layer 3 Isolation domains | 200 isolation domains per Nexus instance |
| Route policies | 400 route policies per Nexus instance |
| Isolation domain MTU | 1500 - 9200 |
Note
- The number of Nexus instances a pair of NFC + CM can handle has been set to 20 based on some theoretical study for ExpressRoute. These numbers will be refined after more testing.
- Some of these limits are yet to be introduced and are not applied by default today.
Network Cloud
The creation of the Network Cloud specific resources is subject to the following resource limits:
| Resource Type | Notes |
|---|---|
| Cluster Manager | 1:1 mapping with Network Fabric Controller |
| Cluster | Up to 20 Nexus Cluster instances per Cluster Manager (within same region) |
| Racks | Up to BOM-specified Compute Racks per Nexus Cluster |
| Bare Metal Machines | Up to BOM-specified BareMetal machines per Rack |
| Storage Appliances | Up to BOM-specified Storage appliances per Nexus Cluster instance |
| NKS Cluster | Depends on selection of VM flavor and number of nodes per NKS cluster |
| Layer 2 Networks | 3500 per Nexus instance |
| Layer 3 Networks | 200 per Nexus instance |
| Trunked Networks | 3500 per Nexus instance |
| Cloud Service Networks | 100 per Nexus instance |
Note
- The number of Nexus instances a pair of NFC + CM can handle has been set to 20 based on some theoretical study for ExpressRoute. These numbers will be refined post GA after some further testing.
- Some of these limits are yet to be introduced and are not applied by default today.
Other Azure resources
There are several Azure resources that are required to build up Network Fabric Controllers and Cluster Manager. The table here outlines the Azure services that Operators must ensure that they have adequate capacity available for creation for each Network Fabric Controller and Cluster Manager pair.
| Resource Type | # of vCPUs |
|---|---|
| Virtual Machine | 32 (D4_v2), 120 (DS4_v2), 4 (D2s_v3) |
| Standard DSv2 Family vCPUs | quota limit 200; distributed across zones 1, 2, and 3 |
Note
The number of vCPUs and the family SKUs required are subject to change.
The table here briefly mentions other Azure resources that are necessary. However, by utilizing a dedicated subscription for Azure Operator Nexus, operators can alleviate concerns about their quotas.
| Resource Type | Notes |
|---|---|
| Subscription | Subscription limits |
| Resource Group | Resource Group Limits. There's a max limit for RG per subscription. Operators need to make appropriate consideration for how they want to manage Resource Groups for NKS clusters vs Virtual machines per Nexus instance. |
| VM Flavors | Customer generally has VM flavor quota in each region within subscription. You need to ensure that you can still create VMs per the requirements. |
| AKS Clusters | AKS Limits |
| Virtual Networks | Virtual Network Limits |
| Managed Identity | Managed Identity Limits |
| ExpressRoute | ExpressRoute Limits |
| Virtual Network Gateway | Virtual Network Gateway Limits |
| Azure Key Vault | Key Vault Limits |
| Storage Account | Storage Account Limits |
| Load Balancers (Standard) | Load Balancer Limits |
| Public IP Address (Standard) | Public IP Address Limits |
| Azure Monitor Metrics | Azure Monitor Limits |
| Log Analytics Workspace | Log Analytics Workspace Limits |