Manufacturing HPC network topology and connectivity

Article
02/13/2023

This guidance builds on considerations and recommendations defined in the Azure landing zone article for network topology and connectivity. Following the guidance in this article helps you examine key design considerations and best practices for networking and connectivity to, from, and within Microsoft Azure and HPC deployments.

Plan for IP address, virtual network, and subnets

It's vital to plan for IP address needs in Azure to ensure that:

The IP address space doesn't overlap across on-premises locations and Azure regions.
Future VNet peering to existing or planned VNets is possible.
The virtual network (VNet) contains the right address space.
Proper planning for subnet configuration happens in advance.
Sufficient excess addressing is considered for future expansion or other services

HPC manufacturing design considerations

Consider creating separate subnets to assign IP addresses across functional components of the environment. For example, a dedicated HPC VNet could include the following subnets:

Compute
Storage
Infrastructure
Visualization
Sign-in
ANF
HPC Cache

Several services like Azure NetApp Files, Azure HPC Cache, and future storage offerings require dedicated delegated subnets for proper operation. Ensure that you plan appropriate addressing space if you're considering any of these services.

Configure DNS and name resolution for on-premises and Azure resources

Domain Name System (DNS) is a critical design aspect in the overall Azure landing zone architecture. Some organizations might want to use their existing investments in DNS. Others might see cloud adoption as an opportunity to modernize their internal DNS infrastructure and use native Azure capabilities.

HPC networking design considerations

The following recommendations are for when a virtual machine's DNS or virtual name doesn't change during migration.

Use case:

Background DNS and virtual names connect many system interfaces in the HPC environments. Customers are only sometimes aware of the interfaces that developers define over time. Connection challenges arise between various systems when virtual or DNS names change after migrations. It's recommended to retain DNS aliases to prevent these types of difficulties.
Use different DNS zones to distinguish each environment (sandbox, development, preproduction, and production) from each other. The exception is for HPC deployments with their own VNet. Here, private DNS zones might not be necessary.
DNS support is mandatory while using HPC cache so they can access storage and other resources.

High-performance network services

Accelerated networking

Many HPC workloads (for example, seismic processing) require processing a large amount of data. The data stores in large shared file systems like Azure Blob, Azure NetApp Files, Lustre ClusterStor, and other custom storage solutions that you access through the network. It's paramount to rely on a high-performance network to reduce the time for data transfers.

Enabling Accelerated networking provides the VMs a high-throughput and low-latency connection between them and to and from Azure services together with reduced jitter and minimal CPU utilization.

InfiniBand

Parallel HPC applications that rely on Message Passing Interface (MPI) libraries might require significant amount of information to be transferred between many VMs. The InfiniBand interconnect available on RDMA capable H-series and N-series VMs provides the required low latency and high bandwidth to maximize the performance and scalability of HPC and AI applications.

Some examples of MPI jobs include:

Molecular dynamics
Computational fluid dynamics
Oil and gas reservoir simulation
Emerging distributed machine learning workloads in manufacturing

InfiniBand connection is only possible between VMs allocated within the same placement group.

Azure ExpressRoute

If there's a burst application like a hybrid setup for reservoir simulation and modeling, where the on-premises data sets are shared and the Azure compute becomes an extension, Express Route helps you connect your on-premises environment into the Microsoft Cloud over a private connection with the help of a connectivity provider. It provides enterprise-grade resiliency and availability, and the advantage of a Global ExpressRoute partner ecosystem. For information on how to connect your network to Microsoft using ExpressRoute, see ExpressRoute connectivity models.
ExpressRoute connections don't go over the public internet, and they offer more reliability, faster speeds, and lower latencies than typical internet connections. For point-to-site VPN and site-to-site VPN, you can connect on-premises devices or networks to a virtual network by using any combination of these VPN options and Azure ExpressRoute.

Define an Azure network topology

Enterprise-scale landing zones support two network topologies: one based on Azure Virtual WAN and the other a traditional network topology based on hub-and-spoke architecture. This section recommends HPC configurations and practices for both deployment models.

Use a network topology based on Virtual WAN if your organization plans to:

Deploy resources across several Azure regions and connect your global locations to both Azure and on-premises.
Fully integrate software-defined WAN deployments with Azure.
Deploy up to 50,000 virtual machine workloads across all VNets connected to one Virtual WAN hub.

Organizations use Virtual WAN to meet large-scale interconnectivity requirements. Microsoft manages this service, which helps to reduce overall network complexity and modernize your organization's network.

Use a traditional Azure network topology based on hub-and-spoke architecture if your organization:

Plans to deploy resources in only select Azure regions.
Doesn't need a global, interconnected network.
Has few remote or branch locations per region and needs fewer than 30 IP security (IPsec) tunnels.
Requires full control and granularity to manually configure your Azure network.
Uses local and global VNet peering to provide connectivity. Local and global VNet peering are the preferred approaches to ensure connectivity between landing zones for HPC deployments across multiple Azure regions.

Plan for inbound and outbound internet connectivity

This section recommends connectivity models for inbound and outbound connectivity to and from the public internet. Azure-native network security services like Azure Firewall, Azure Web Application Firewall on Application Gateway, and Azure Front Door are fully managed services. So, you don't incur the operational and management costs associated with infrastructure deployments, which can become complex at scale.

Design recommendations for HPC implementation:

For customers with a global footprint, Azure Front Door helps HPC deployments by using Azure Web Application Firewall policies to deliver and protect global HTTP/S applications across Azure regions.
Take advantage of Web Application Firewall policies in Azure Front Door when you're using this service and Azure Application Gateway to protect HTTP/S applications. Lock down Azure Application Gateway to receive traffic only from Azure Front Door.

Define network encryption requirements

This section explores key recommendations for encrypting networks between on-premises and Azure and across Azure regions.

Design considerations for HPC implementations:

Traffic isn't currently encrypted when you use Azure ExpressRoute to configure private peering.
It isn't necessary to encrypt traffic over ExpressRoute for HPC deployments. IPsec tunnels encrypt internet traffic by default. Encryption or decryption could negatively affect the traffic's performance.

It's up to the customer to determine whether HPC traffic should be encrypted. Explore network topology and connectivity to understand network encryption options in enterprise-scale landing zones.