Internal traffic is not supposed to hit the WAPs. The internal clients must use the ADFS farm directly (or through a load balancer, but a load balancer that redirects the traffic directly to the ADFS servers and not he WAPs).
In order to achieve this, the easier solution is to have a split-brain DNS (aka split-horizon). Let say the ADFS server IP is 10.0.0.1, the WAP IP is 1.1.1.1 and the FQDN of your ADFS deployment is adfs.contoso.com. When internal clients are using the internal DNS servers, they should resolve adfs.contoso.com into 10.0.0.1 and when clients are using the public DNS for the same record, they should resolve adfs.contoso.com into 1.1.1.1. This is also listed here in the Network section.