Custom error pages are not supported out of the box with Azure App services. There is an open internal feedback item for this and I have created a new tracking item that I have bubbled up to product team along with your request. You are also encouraged to create your own feedback item on the Microsoft Ideas forum. https://feedback.azure.com/d365community
In B2C you can create a custom error page using a technical profile or a custom policy error page. But for regular Azure AD B2B, there is no way to specify custom error messages or error pages except in cases where the error message is passed back to your application.
You can also create Application Gateway custom error pages, but it's only supported for maintenance pages (to be displayed instead of a 502 error) and unauthorized access pages (to be displayed instead of a 403).