@Gallatin 21V , Based on my understanding, you are looking to configure custom error pages.
Custom error pages are for errors that terminate on the Frontend (which means that the request that doesn't make to your app). Once the request gets to the app it is assumed the user can use whatever error page they want.
How-to configure a custom error page on App Service
App Service now enables the use of custom error pages for 403, 502, and 503 error codes so you can customize your users experience. It's available for both Linux and Windows workloads using at least a Premium SKU. The following is a tutorial on how to upload a custom error page to your Web App.
Custom error pages is currently available to try as a preview feature
Also, please checkout this discussion thread.
If the answer helped (pointed you in the right direction) > please click Accept Answer