After discussions with Microsoft support it appears that port 8172 is enabled for backwards compatibility with old versions of MsDeploy. This port is being phased out and will be open sometimes and not other times.
The fix was for us to create a new resource group, app service plan and app services multiple times until we ended up a server that had the port closed. This was frustrating but ultimately it did resolve the issue.