How to authenticate with app-auth to crawl an azure web application?

Question

You can run the following to obtain a bearer token from login.microsoftonline.com:

Curl command example here: https://gist.github.com/nddipiazza/809ec76aac86afc49e71c6c8b0b6e44f

Then you can use this bearer token to access an Azure Web Site:

Curl command example here: https://gist.github.com/nddipiazza/db9a80827d14f8d6be16f615d8559b1d

This works on our sample microsoft online account that we created from scratch to run this test. Without the bearer token, we get authentication redirected. With bearer token, we are able to access the web page.

When we run this same thing on our corporate instance, it just throws a generic error:

"You do not have permission to view this directory or page." (401 Unauthorized)

Is there a link on the Azure Online Portal to obtain the information about the failure?

Related links I found maybe helpful but not sure if any of them are what I need:
https://learn.microsoft.com/en-us/answers/questions/93243/azure-function-app-secured-by-aad-you-do-not-have.html
https://stackoverflow.com/questions/48853599/how-to-debug-you-do-not-have-permission-to-view-this-directory-or-page

Answer 1

Suggest you to enable application logging, you should see authentication and authorization traces directly in your log files.
If you see an authentication error that you didn't expect, you can find all the details by looking in your existing application logs. If you enable failed request tracing, you can see exactly what role the authentication and authorization module may have played in a failed request.

Check this article for details on Authentication and authorization in Azure App Service and Azure Functions

Answer 2

@nicholas dipiazza If there is cold starts? it seems Always On request receives a 401. Could you use app setting WEBSITE_WARMUP_PATH = "/" for authorization/authentication to bypass the root URL.