Our Azure App Service application started to experience intermittent CORS errors

Question

Hi all,

I am new here, and I am hoping someone can help as we are completely at a loss with the problem we are experiencing suddenly with Azure App Service.

We have 2 Azure App Services, 1 for API and the other is an Web application (ReactJS). Both are hosted under different subdomain (api.domain.com and web.domain.com), and we manage our CORS and Referrer Policy via .NET Core Startup.cs, and not within Azure App Services. Both of these app services are hosted within the same App Service Plan (1 instance).

Our web application has been working for the past 3 years, but about a week ago our users started to report intermittent CORS errors. The error usually last about < 1min, and it goes away on its own, but it keeps coming back to different users every few minutes.

We have checked our recent releases but nothing seems to be the root cause this problem. Our team couldn't reproduce the CORS on our end (as it's quite random).

When we look at the Developer tool to analyze the HAR file, the only thing that looks suspicious is that the preflight call prior to the failed request always seem to be using port 80 for the remote address. We don't know whether this is the cause and why the browser would downgrade HTTPS to HTTP?

Our Referrer Policy is currently set as "strict-origin-when-cross-origin".

Wondering if anyone have experience something similar or if any experts have any suggestions on how to investigate to this problem we are having?

Any help is greatly appreciated.

Thank you in advance!

Allan

Answer 1

As of June 11, 2024 18:13 UTC the fix is pretty much completely done.

The fix goes by stamps so in order to know if your environment has the fix we would need to know the details of your environment to know in which stamp it is hosted.

One way to do it would be by creating a ticket and provide the information, that way we will know if your environment has been patched already or if is not we can patched. However as of 2024-06-11 about 88% of all the front ends are patched including nearly all multitenant.

Answer 2

The Microsoft Azure Team has investigated the issue you reported defined by intermittent 500 errors starting around mid-May 2024 from PUT/POST HTTP requests for your Microsoft.Web resources.

 

Upon investigation, engineers determined that the behavior was caused by a race condition in our FrontEnd load balancers. This race condition was triggered by one of our standard maintenance upgrades to our FrontEnds.

 

As soon as the issue was identified the engineering team worked on a fix and immediately started rolling it out. The fix for this issue has now been fully rolled out and all impacted scale units are patched.

 

We are always aiming to improve our stability and reliability. In this case that includes, but is not limited to:

• Code change to prevent this issue from happening again in the future
• Improving our perf + testing capabilities to have a bigger focus on such scenarios
• Enhancing our logging and monitoring systems to detect and alert us to any similar behavior as soon as possible

 

We sincerely apologize for any inconvenience this issue may have caused and appreciate your understanding as we continuously work to improve the reliability and performance of Azure services.

 

Regards,

 

The Microsoft Azure Team

Answer 3

Hi, As of today I am still facing this issue with HTTP 504 errors.

Our App services are hosted in France Central, the frontend app service is in Angular 16 and the backend app service is in .NET 8.

We just tried the fix to change our HTTP version from 2 to 1.1.

I am concerned about this fix you mentioned since we still had A LOT of 504 errors even this very morning.