If your Application Gateway is using HTTP/2, the TCP idle timeout is hardcoded to 180 seconds (3 minutes) by default and cannot be modified.
For HTTP/1.1, the Keep-Alive timeout is 120 seconds (non-configurable), and the TCP idle timeout is 4 minutes by default on public IPs—but this can be increased up to 30 minutes. If you're using a private IP, the idle timeout is fixed at 5 minutes.
Ref: What are the settings for Keep-Alive timeout and TCP idle timeout?
Check if the suggestion below can resolve your issue.
Check if HTTP/2 is enabled and test with it disabled temporarily.
$appgateway = Get-AzApplicationGateway -Name test -ResourceGroupName hm
$appgateway.EnableHttp2 = $false
Set-AzApplicationGateway -ApplicationGateway $appgateway
Consider enabling session affinity (cookie-based affinity) if not already configured enable by navigating to Backend settings and select your backend settings and enable
Once you enable the cookie-based affinity setting for the service, it directs subsequent traffic from the same user session to the same backend server for processing. follow the Enable cookie-based affinity with Application Gateway
I hope this helps to resolve your issue. Please feel free to ask any questions if the solution provided isn't helpful.
Please don’t forget to close the thread by clicking "Accept the answer" wherever the information provided helps you, as this can be beneficial to other community members.