Byte Count metrics clarification on private LB

Question

Hello,

I have setup a private Load Balancer with 1 backend server and only one LB rule. When checking the byte count metrics of it based on the direction, I notice high value for Out than In.

I would like to understand how is this possible? when there are no other outbounds from the backends that goes through this LB + the backends serve only the incoming traffic to it through the LB (meaning whatever In bytes will be served as Out bytes).

So would like to know what is this additional OUT bytes traffic which is being recorded in the metrics.

Any assistance would be helpful, thank you.

Answer 1

Hello @Alex ,

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others", I'll repost your solution in case you'd like to "Accept" the answer.

Issue:

You setup an internal Load Balancer with 1 backend server and only one LB rule and the backend serves only the incoming traffic to it through the LB and there is no outbound traffic initiated from it. But when checking the byte count metrics of the LB, based on the direction, you noticed high value for Out than In.

Solution:

I discussed this issue with the Azure load balancer Product Group team, and they requested more information related to your load balancer for further investigation. Hence, I requested you to share the details via an email.

You noticed that you have a support plan enabled on the Azure account, so you raised a support request and had a quick chat with the support regarding this and they clarified below things:

1. Load balancer itself will not measure/capture any metrics; it just acts like a virtual control plane facilitating the connections from client to the backend servers.
2. The data seen in ByteCount metrics is the actual data being processed by the backend servers. If the IN and OUT bytes are different, then the backend servers are responding with additional data in response to the requests and there isn't an ideal case where both IN and OUT should be close to equal at all times.
3. In your scenario, you have a LB with 1 backend server with coturn service, so additional guidance is to do tcpdump or packet trace the backend server network traffic and deduce further to get a conclusion on the additional OUT bytes seen in the metrics, if needed.
4. You also checked 2-3 more load balancers which had the similar setup and observed the same differences, like some had IN>OUT, some had OUT>IN, but never equal.

If you have any other questions or are still running into more issues, please let me know.

Thank you again for your time and patience throughout this issue.

---

Please don’t forget to close the thread by clicking "Accept the answer" wherever the information provided helps you, as this can be beneficial to other community members.