@MS Techie , Thanks for posting this question.
Windows/Active Directory authentication is not supported on App Service, because you cannot domain join an App Service worker.
However, you could use SQL authentication.
Managed identities in App Service make your app more secure by *eliminating secrets from your app**( connection strings)*.
In this tutorial, you will add managed identity to the sample web app you built: Tutorial: Secure Azure SQL Database connection from App Service using a managed identity
- Enable managed identities and grant SQL Database access to the managed identity
- Configure Entity Framework to use Azure AD authentication with SQL Database
- Connect to SQL Database from Visual Studio using Azure AD authentication
Additionally, kindly see this blog for more details.
Hope this helps!