Hi @Todd Elliott ,
You don't need a custom domain for each app service. Your custom domain will need be applied to your Application Gateway. Your reply URLs for AAD will need to match your https://<sitename>.azurewebsites.net address of your app service, as explained in the doc you referenced. For your multiple app services, you can override the host headers of the incoming requests to ensure it gets to the right app service.
Regards,
Ryan