Networking
As you create and manage Azure Service Fabric clusters, you're providing network connectivity for your nodes and applications. The networking resources include IP address ranges, virtual networks, load balancers, and network security groups. In this article, you learn best practices for these resources.
Review Azure Service Fabric Networking Patterns to learn how to create clusters that use the following features: Existing virtual network or subnet, Static public IP address, Internal-only load balancer, or Internal and external load balancer.
Infrastructure Networking
Maximize your Virtual Machine's performance with Accelerated Networking, by declaring enableAcceleratedNetworking property in your Resource Manager template, the following snippet is of a Virtual Machine Scale Set NetworkInterfaceConfigurations that enables Accelerated Networking:
"networkInterfaceConfigurations": [
{
"name": "[concat(variables('nicName'), '-0')]",
"properties": {
"enableAcceleratedNetworking": true,
"ipConfigurations": [
{
<snip>
}
],
"primary": true
}
}
]
Service Fabric cluster can be provisioned on Linux with Accelerated Networking, and Windows with Accelerated Networking.
Accelerated Networking is supported for Azure Virtual Machine Series SKUs: D/DSv2, D/DSv3, E/ESv3, F/FS, FSv2, and Ms/Mms. Accelerated Networking was tested successfully using the Standard_DS8_v3 SKU on 01/23/2019 for a Service Fabric Windows Cluster, and using Standard_DS12_v2 on 01/29/2019 for a Service Fabric Linux Cluster. Note that Accelerated Networking requires at least 4 vCPUs.
To enable Accelerated Networking on an existing Service Fabric cluster, you need to first Scale a Service Fabric cluster out by adding a Virtual Machine Scale Set, to perform the following steps:
- Provision a NodeType with Accelerated Networking enabled
- Migrate your services and their state to the provisioned NodeType with Accelerated Networking enabled
Scaling out infrastructure is required to enable Accelerated Networking on an existing cluster, because enabling Accelerated Networking in place would cause downtime, as it requires all virtual machines in an availability set be stop and deallocate before enabling Accelerated networking on any existing NIC.
Cluster Networking
Service Fabric clusters can be deployed into an existing virtual network by following the steps outlined in Service Fabric networking patterns.
Network security groups (NSGs) are recommended for node types to restrict inbound and outbound traffic to their cluster. Ensure that the necessary ports are opened in the NSG.
The primary node type, which contains the Service Fabric system services doesn't need to be exposed via the external load balancer and can be exposed by an internal load balancer
Use a static public IP address for your cluster.
Network Security Rules
The rules described next are the recommended minimum for a typical configuration. We also include what rules are mandatory for an operational cluster if optional rules aren't desired. It allows a complete security lockdown with network peering and jumpbox concepts like Azure Bastion. Failure to open the mandatory ports or approving the IP/URL will prevent proper operation of the cluster and might not be supported.
Inbound
Priority | Name | Port | Protocol | Source | Destination | Action | Mandatory |
---|---|---|---|---|---|---|---|
3900 | Azure portal | 19080 | TCP | ServiceFabric | Any | Allow | Yes |
3910 | Client API | 19000 | TCP | Internet | Any | Allow | No |
3920 | SFX + Client API | 19080 | TCP | Internet | Any | Allow | No |
3930 | Cluster | 1025-1027 | TCP | VirtualNetwork | Any | Allow | Yes |
3940 | Ephemeral | 49152-65534 | TCP | VirtualNetwork | Any | Allow | Yes |
3950 | Application | 20000-30000 | TCP | VirtualNetwork | Any | Allow | Yes |
3960 | RDP | 3389 | TCP | Internet | Any | Deny | No |
3970 | SSH | 22 | TCP | Internet | Any | Deny | No |
3980 | Custom endpoint | 443 | TCP | Internet | Any | Deny | No |
More information about the inbound security rules:
Azure portal. This port is used by the Service Fabric Resource Provider to query information about your cluster in order to display in the Azure Management Portal. If this port isn't accessible from the Service Fabric Resource Provider, you see a message such as 'Nodes Not Found' or 'UpgradeServiceNotReachable' in the Azure portal and your node and application list appears empty. This means that if you wish to have visibility of your cluster in the Azure Management Portal then your load balancer must expose a public IP address and your NSG must allow incoming 19080 traffic. This port is recommended for extended management operations from the Service Fabric Resource Provider to guarantee higher reliability.
Client API. The client connection endpoint for APIs used by PowerShell.
SFX + Client API. This port is used by Service Fabric Explorer to browse and manage your cluster. It's used by most common APIs like REST/PowerShell (Microsoft.ServiceFabric.PowerShell.Http)/CLI/.NET in the same way.
Cluster. Used for inter-node communication.
Ephemeral. Service Fabric uses a part of these ports as application ports, and the remaining are available for the OS. It also maps this range to the existing range present in the OS, so for all purposes, you can use the ranges given in the sample here. Make sure that the difference between the start and the end ports is at least 255. You might run into conflicts if this difference is too low, because this range is shared with the OS. To see the configured dynamic port range, run netsh int ipv4 show dynamicport tcp. These ports aren't needed for Linux clusters.
Application. The application port range should be large enough to cover the endpoint requirement of your applications. This range should be exclusive from the dynamic port range on the machine, that is, the ephemeralPorts range as set in the configuration. Service Fabric uses these ports whenever new ports are required and takes care of opening the firewall for these ports on the nodes.
RDP. Optional, if RDP is required from the Internet or VirtualNetwork for jumpbox scenarios.
SSH. Optional, if SSH is required from the Internet or VirtualNetwork for jumpbox scenarios.
Custom endpoint. An example for your application to enable an internet accessible endpoint.
Note
For most rules with Internet as source please consider to restrict to your known network, ideally defined by CIDR block.
Outbound
Priority | Name | Port | Protocol | Source | Destination | Action | Mandatory |
---|---|---|---|---|---|---|---|
4010 | Resource Provider | 443 | TCP | Any | ServiceFabric | Allow | Yes |
4020 | Download Binaries | 443 | TCP | Any | AzureFrontDoor.FirstParty | Allow | Yes |
More information about the outbound security rules:
Resource Provider. Connection between UpgradeService and Service Fabric resource provider to receive management operations such as ARM deployments or mandatory operations like seed node selection or primary node type upgrade.
Download Binaries. The upgrade service is using the address download.microsoft.com to get the binaries, this relationship is needed for setup, re-image and runtime upgrades. In the scenario of an "internal only" load balancer, an additional external load balancer must be added with a rule allowing outbound traffic for port 443. Optionally, this port can be blocked after a successful setup, but in this case the upgrade package must be distributed to the nodes or the port has to be opened for the short period of time, afterwards a manual upgrade is needed.
Use Azure Firewall with NSG flow log and traffic analytics to track connectivity issues. The ARM template Service Fabric with NSG is a good example to start.
Note
The default network security rules should not be overwritten as they ensure the communication between the nodes. Network Security Group - How it works. Another example, outbound connectivity on port 80 is needed to do the Certificate Revocation List check.
Common scenarios needing additional rules
All additional scenarios can be covered with Azure Service Tags.
Azure DevOps
The classic PowerShell tasks in Azure DevOps (Service Tag: AzureCloud) need Client API access to the cluster, examples are application deployments or operational tasks. This doesn't apply to the ARM templates only approach, including ARM application resources.
Priority | Name | Port | Protocol | Source | Destination | Action | Direction |
---|---|---|---|---|---|---|---|
3915 | Azure DevOps | 19000 | TCP | AzureCloud | Any | Allow | Inbound |
Updating Windows
Best practice to patch the Windows operating system is replacing the OS disk by automatic OS image upgrades, no additional rule is required. The Patch Orchestration Application is managing in-VM upgrades where Windows Updates applies operating system patches, this needs access to the Download Center (Service Tag: AzureUpdateDelivery) to download the update binaries.
Priority | Name | Port | Protocol | Source | Destination | Action | Direction |
---|---|---|---|---|---|---|---|
4015 | Windows Updates | 443 | TCP | Any | AzureUpdateDelivery | Allow | Outbound |
API Management
The integration of Azure API Management (Service Tag: ApiManagement) need Client API access to query endpoint information from the cluster.
Priority | Name | Port | Protocol | Source | Destination | Action | Direction |
---|---|---|---|---|---|---|---|
3920 | API Management | 19080 | TCP | ApiManagement | Any | Allow | Inbound |
Application Networking
To run Windows container workloads, use open networking mode to make service-to-service communication easier.
Use a reverse proxy such as Traefik or the Service Fabric reverse proxy to expose common application ports such as 80 or 443.
For Windows Containers hosted on air-gapped machines that can't pull base layers from Azure cloud storage, override the foreign layer behavior, by using the --allow-nondistributable-artifacts flag in the Docker daemon.
Next steps
- Create a cluster on VMs or computers running Windows Server: Service Fabric cluster creation for Windows Server
- Create a cluster on VMs or computers running Linux: Create a Linux cluster
- Learn about Service Fabric support options
Athugasemdir
https://aka.ms/ContentUserFeedback.
Væntanlegt: Á árinu 2024 munum við hætta með GitHub-mál sem ábendingaleið fyrir efni og skipta því út fyrir nýtt ábendingakerfi. Frekari upplýsingar er að finna hér:Senda inn og skoða endurgjöf fyrir