@New_Usr001 - Thank you for reaching out. Currently you can use Application Gateway, which allows you to create custom error pages instead of displaying default error pages. You can use your own branding and layout using a custom error page. Please refer to this documentation for further reference.
Review the information and see if it fits your scenario, if your ask is primarily for setting up custom error pages with in Azure App service, we don't have that out of box currently. There is an open feedback item, please review and upvote which would help our team to prioritize the issue accordingly.
If you have any other query, please feel free to revert back. Thank you