Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
Wasting a lot of time searching solutions from internet, I fix the same issue on my own.
The only thing you need to do is to use HTTPS instead of HTTP. MSAL only support HPPTS.
It works in my scenario, and wish it helps.