Load Balancer balancing behavior

Question

Load Balancer balancing behavior

Frame, Grant 6

We have deployed a load balancer and attached a small number (3) of instances to handle incoming connections. We have set this up to use the default distribution mode which i understand is a 5 tuple. We then have 1 instance connecting to the Load Balancer. Per my understanding since each successive client connection will use a different client port that means that each connection gets its own unique hash. We were seeing that for a small number of connections - around 10-15 or so we say all going to 1 node attached to load balancer. They were not spread out over multiple nodes. So my question is how are the hashes split? are there just buckets and then hash determines which bucket the connection gets routed to? Is there a way to do a more round-robin type of split?

1 answer

Your answer

Answer 1

GitaraniSharma-MSFT 50,021 Microsoft Employee Moderator

Hello @Frame, Grant ,

Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.

I understand that you would like to know Azure load balancer distribution works and how to do a more round robin type of load balancing for your VMs.

Azure load balancer distribution is not round robin. It's hash based load balancing.

Azure Load Balancer supports two distribution modes for routing connections to your load-balanced applications:

Hash based
Source IP affinity

The default distribution mode for Azure Load Balancer is a five-tuple hash.
Hash-based mode has one configuration type:

None (hash-based) - Specifies that successive requests from the same client may be handled by any virtual machine.

Refer : https://learn.microsoft.com/en-us/azure/load-balancer/concepts
https://learn.microsoft.com/en-us/azure/load-balancer/distribution-mode-concepts

There is an insights called flow distribution which helps you visualize and manage the number of flows your backend instances are receiving and producing.

From the Insights blade of your Load Balancer, click on the View detailed metrics and then select the Flow Distribution tab.
Refer: https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-insights#flow-distribution

NOTE : Please note this feature is in Preview and the functional dependency view and preconfigured dashboard may change to improve this experience.

If you need a more round-robin type of load balancing, then I would suggest you to go with Azure Application gateway.

When the Application gateway selects the backend pool, it sends the request to one of the healthy backend servers in the pool (y.y.y.y). The health of the server is determined by a health probe. If the backend pool contains multiple servers, the application gateway uses a round-robin algorithm to route the requests between healthy servers. This load balances the requests on the servers.
Refer : https://learn.microsoft.com/en-us/azure/application-gateway/how-application-gateway-works

Kindly let us know if the above helps or you need further assistance on this issue.

----------------------------------------------------------------------------------------------------------------

Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

Frame, Grant 6 Reputation points

2022-11-03T00:55:16.573+00:00

Thanks for that info. My understanding is that Application Gateway is for servers which accept HTTP requests. Our service is only using TCP so i think we are not able to use that.

We will look into the flow distribution metrics you mentioned.

Another follow up question. We are right now using PrivateLink to connect from one subscription (where the client is) to another subscription (where the server) is. The PriavteLink Service then is connected to the Load Balancer. So is the client IP/port seen by the Load Balancer (and thus used in hash) the one from PrivateLink Service to Load Balancer or is it the client IP on other side of the Private Link (in other subscription)?
GitaraniSharma-MSFT 50,021 Reputation points Microsoft Employee Moderator

2022-11-04T16:39:39.62+00:00

Hello @Frame, Grant ,

I discussed your follow-up question with the Product Group team and below is their response:

Could you please send a visual or clarify your question a bit?
Normally the backend machines (where the application is running) will receive the NAT IP/Port when they receive traffic from Private Link. NAT IP is an IP that you select when creating a private link service. You don’t get client information unless you enable tcp proxy v2.

Hence, I would request you to share some more details on your setup and provide clarity on your question for further discussion on the same.

Regards,
Gita
Frame, Grant 6 Reputation points

2022-11-08T17:18:29.27+00:00

Hi, yes we are seeing the NAT IP on the backend machines. I was wondering if by enabling this 'tcp proxy v2' there would be more client ip/port combos so when hashed they spread out more equally among the backend machines.
We are in process of testing some things and using the flow distribution metrics to understand how the connections are being balanced.
GitaraniSharma-MSFT 50,021 Reputation points Microsoft Employee Moderator

2022-11-10T14:37:52.377+00:00

Hello @Frame, Grant ,

I will check regarding the following "by enabling this 'tcp proxy v2', would there be more client ip/port combos so when hashed they spread out more equally among the backend machines" and get back to you with an update.

Regards,
Gita
Frame, Grant 6 Reputation points

2022-11-10T16:57:47.103+00:00

Hi,

We looked into the flow distribution metrics and what we are seeing is that over time (say several hours) the flow is balanced. However over smaller amount of time (say 5-10mins) we do not see this balanced flow. E.g. We had 1 client which opened 10 connections through the Load Balancer and saw that all 10 connections went to only 1 node in target group for the Load Balancer. We kept these connections idle for a few minutes and then closed and re-established them. After 10 mins or so we started seeing new connections being routed to the other node in target group and moreover all new connections were only routed to that other node. The original node stopped getting new connections. This is why we see equal flow over time but not over smaller time frame.
We are using default distribution. What we are trying to do is open some small number of connections and get at least 1 to each node in the target group for load balancer. But that does not seem to be happening and we see that we cannot make connection to a node in target group.
GitaraniSharma-MSFT 50,021 Reputation points Microsoft Employee Moderator

2022-11-15T15:48:34.537+00:00

Hello @Frame, Grant ,

Thank you for the update.

Let me discuss this issue further with the Product Group team and get back to you.

Regards,
Gita
GitaraniSharma-MSFT 50,021 Reputation points Microsoft Employee Moderator

2022-11-24T12:41:07.407+00:00

Hello @Frame, Grant ,

Apologies for the delay in response. But it is taking longer than expected to get some updates on this issue.
Please allow me some more time and I will provide an update soon.

Regards,
Gita
GitaraniSharma-MSFT 50,021 Reputation points Microsoft Employee Moderator

2022-12-08T13:18:36.02+00:00

Hello @Frame, Grant ,

This particular scenario needs a specialized 1:1 remote session and access to the backend logs of your resources for further investigation.

The Product Group team mentioned that they would need to look into the particular setup and configuration to provide any further updates. Hence, if you have a support plan, I request you file a support ticket, else please do let us know, we will try and help you get a one-time free technical support.

Regards,
Gita

Share via

Load Balancer balancing behavior

1 answer

Your answer