Edit

Azure production network

The users of the Azure production network include both external customers who access their own Azure applications and internal Azure support personnel who manage the production network. This article discusses the security access methods and protection mechanisms for establishing connections to the Azure production network.

Internet routing and fault tolerance

A globally redundant internal and external Azure Domain Name Service (DNS) infrastructure, combined with multiple primary and secondary DNS server clusters, provides fault tolerance. Azure also applies infrastructure-level DDoS protection and other network security controls to help defend against common network-layer DDoS attacks and protect the integrity of Azure DNS services.

The Azure DNS servers are located at multiple datacenter facilities. The Azure DNS implementation incorporates a hierarchy of secondary and primary DNS servers to publicly resolve Azure customer domain names. The domain names usually resolve to a cloudapp.net address, which wraps the virtual IP (VIP) address for the customer's service. Unique to Azure, Microsoft load balancers that are responsible for that VIP translate the VIP to the internal dedicated IP (DIP) address of the tenant.

Microsoft hosts Azure in geographically distributed Azure datacenters within the US. Azure uses state-of-the-art routing platforms that implement strong, scalable architectural standards. Among the notable features are:

  • Multiprotocol Label Switching (MPLS)-based traffic engineering, which provides efficient link utilization and graceful degradation of service if an outage occurs.
  • Microsoft implements networks with "need plus one" (N+1) redundancy architectures or better.
  • Externally, dedicated, high-bandwidth network circuits serve datacenters that redundantly connect properties with over 1,200 internet service providers globally at multiple peering points. This connection provides more than 2,000 gigabytes per second (GBps) of edge capacity.

Because Microsoft owns its own network circuits between datacenters, these attributes help the Azure offering achieve 99.9+ percent network availability without the need for traditional third-party internet service providers.

Connection to production network and associated firewalls

The Azure network internet traffic flow policy directs traffic to the Azure production network that's located in the nearest regional datacenter within the US. Because the Azure production datacenters maintain consistent network architecture and hardware, the traffic flow description that follows applies consistently to all datacenters.

After internet traffic for Azure is routed to the nearest datacenter, the traffic establishes a connection to the access routers. These access routers isolate traffic between Azure nodes and customer-instantiated VMs. Network infrastructure devices at the access and edge locations are the boundary points where ingress and egress filters apply. These routers use a tiered access-control list (ACL) to filter unwanted network traffic and apply traffic rate limits, if necessary. ACL routes allowed traffic to the load balancers. Distribution routers allow only Microsoft-approved IP addresses, provide anti-spoofing, and establish TCP connections that use ACLs.

Microsoft places external load-balancing devices behind the access routers to perform network address translation (NAT) from internet-routable IPs to Azure internal IPs. The devices also route packets to valid production internal IPs and ports. These devices act as a protection mechanism to limit exposure of the internal production network address space.

By default, Microsoft enforces Hypertext Transfer Protocol Secure (HTTPS) for all traffic that's transmitted to customers' web browsers, including sign-in and all traffic thereafter. TLS v1.2 creates a secure tunnel for traffic. ACLs on access and core routers ensure that the source of the traffic matches the expected source.

An important distinction in this architecture, when it's compared to traditional security architecture, is that Azure has no dedicated hardware firewalls, specialized intrusion detection or prevention devices, or other security appliances that are normally expected before connections are made to the Azure production environment. Customers usually expect these hardware firewall devices in the Azure network. However, Azure doesn't employ these devices. Almost exclusively, those security features are built into the software that runs the Azure environment to provide strong, multilayered security mechanisms, including firewall capabilities. Additionally, the software that runs Azure makes the scope of the boundary and associated sprawl of critical security devices easier to manage and inventory.

Core security and firewall features

Azure implements strong software security and firewall features at various levels to enforce security features that customers usually expect in a traditional environment to protect the core security authorization boundary.

Azure security features

Azure implements host-based software firewalls inside the production network. Several core security and firewall features reside within the core Azure environment. These security features reflect a defense-in-depth strategy within the Azure environment. The following firewalls protect customer data in Azure:

Hypervisor firewall (packet filter): The hypervisor implements this firewall, and the fabric controller (FC) agent configures it. This firewall protects the tenant that runs inside the VM from unauthorized access. By default, when a VM is created, Azure blocks all traffic, and then the FC agent adds rules and exceptions in the filter to allow authorized traffic.

Azure programs two categories of rules:

  • Machine config or infrastructure rules: By default, Azure blocks all communication. Exceptions allow a VM to send and receive Dynamic Host Configuration Protocol (DHCP) communications and DNS information and to send traffic to the "public" internet outbound to other VMs within the FC cluster and OS Activation server. Because the VMs' allowed list of outgoing destinations doesn't include Azure router subnets and other Microsoft properties, the rules act as a layer of defense for them.
  • Role configuration file rules: These rules define the inbound ACLs based on the tenants' service model. For example, if a tenant has a web front end on port 80 on a certain VM, port 80 is opened to all IP addresses. If the VM has a worker role running, the worker role is opened only to the VM within the same tenant.

Native host firewall: Azure Service Fabric and Azure Storage run on a native OS, which has no hypervisor and, therefore, the preceding two sets of rules configure Windows Firewall.

Host firewall: The host firewall protects the host partition, which runs the hypervisor. The rules allow only the FC and jump boxes to talk to the host partition on a specific port. The other exceptions are to allow DHCP response and DNS replies. Azure uses a machine configuration file, which contains a template of firewall rules for the host partition. A host firewall exception also exists that allows VMs to communicate to host components, wire server, and metadata server, through specific protocols and ports.

Guest firewall: The Windows Firewall component of the guest OS, which customers can configure on customer VMs and storage.

More security features built into Azure capabilities include:

  • Azure assigns IP addresses from DIPs to infrastructure components. An attacker on the internet can't address traffic to those addresses because it wouldn't reach Microsoft. Internet gateway routers filter packets that are addressed solely to internal addresses, so they wouldn't enter the production network. The only components that accept traffic that's directed to VIPs are load balancers.

  • Firewalls implemented on all internal nodes have three primary security architecture considerations for any given scenario:

    • Firewalls sit behind the load balancer and accept packets from anywhere. These packets are intended to be externally exposed and would correspond to the open ports in a traditional perimeter firewall.
    • Firewalls accept packets only from a limited set of addresses. This consideration is part of the defensive in-depth strategy against DDoS attacks. Such connections are cryptographically authenticated.
    • Only select internal nodes can access firewalls. They accept packets only from an enumerated list of source IP addresses, all of which are DIPs within the Azure network. For example, an attack on the corporate network could direct requests to these addresses, but Azure blocks the attacks unless the source address of the packet is one in the enumerated list within the Azure network.
      • The access router at the perimeter blocks outbound packets that are addressed to an address that's inside the Azure network because of its configured static routes.

Next steps

To learn more about what Microsoft does to secure the Azure infrastructure, see:

  • Last updated on