@MS Techie , Thanks for the question. If I have understood you correctly. Yes, the setup is very much possible and is a common requirement.
You can deploy multiple separate ‘WebApps’ under an ‘App Service Plan’. All those WebApps (/websites) will have their own separate default domain (FQDN) and also you can set custom domain for each of those WebApps.
Firstly, just to highlight on the terminology for more clarity:
App Service – Service which enables you to build and host web apps, mobile back ends, and RESTful APIs.
Apps– Your individual Apps (WebApps), these apps run in App Service Plan.
App Service Plan - An App Service Plan (ASP) defines a set of compute resources for a web app to run.
The ASP must be Shared plan or above tier to have ‘custom domain’ support. See, this doc. https://azure.microsoft.com/pricing/details/app-service/
Since you pay for the computing resources your App Service plan allocates, you can potentially save money by putting multiple apps into one App Service plan. You can continue to add apps to an existing plan as long as the plan has enough resources to handle the load. However, keep in mind that apps in the same App Service plan all share the same compute resources.
If those are separate WebApps, it would be easier to manage separate WebApps than via virtual subdirectories/paths or hostname, or sub-domains. Using App Service Plan feature (provision many apps under the tier) to save cost.
Azure Web Site App with multiple custom domains
To map a custom domain, see - Map App Service domain to your app
Kindly let me know if I have misunderstood your ask. Based on your requirement. Kindly see these discussion threads.