Thank you for reaching out.
I understand you want to know about how to differentiate if the request being sent to the load balancer is a health probe request or actual traffic.
When a TCP health probe is set up on Azure Load Balancer the TCP probes initiate a connection by performing a three-way open TCP handshake with the defined port. TCP probes terminate a connection with a four-way close TCP handshake. There is no payload associated with these probes.
How can I distinguish between these two incoming traffic whether it is normal or load balancer health check probe?
All the health probes are sent with source IP of 168.63.129.16. This way you can distinguish between the health probe and actual traffic. This is currently documented here.
Additional reference:
Hope this helps! Please let me know if you have any additional questions. Thank you!