Azure Front Door returns a 503 Service Unavailable when Range and Compression Headers are in the request

Question

We're seeing an issue with our Front Door configuration where requests from the Facebook crawler are consistently seeing a 503 Service Unavailable error for all requests.

This seems to be related to the combination of request headers used:

User-Agent: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
Accept-Encoding: deflate, gzip
Range: bytes=0-524288
Connection: close

Or in curl:

curl -v --compressed -H "Range: bytes=0-524288" -H "Connection: close" -A "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)" "$URL"

Requesting the page without the Accept-Encoding correctly returns a 206 Partial Content response.

Sending the full request to the origin/backend host directly (Azure App Service) also returns a valid response with the expected page content.

I've seen (and replied to) the same issue reported here: https://feedback.azure.com/forums/217313-networking/suggestions/38617117

Answer 1

Apologies all for not coming back on this.

@SaiKishor-MSFT Not really, no. The end result is that the social site (or anything else using the range header in combination with gzip encoding) causes Front Door to return a 503 when the backend is healthy and responding correctly. These are dynamic sites, so it's likely that headers or similar are slightly different between requests.

@Rich DeBourke Yes, we found the same, so I adjusted the rule to remove the Accepts-Encoding header if there was also a Range header. This ensures that all of these should work and get the correct 206 Partial response that is no larger than the requested value.

Answer 2

I have managed to implement the following work-around for now:

I've created a Rules Engine rule that removes the Accept-Encoding header when the User Agent begins with facebookexternal. This appears to be sufficient to stop Front Door from returning a 503 response, and at the moment the metadata on our pages fits within the requested range from Facebook, so this looks to be working at the moment.

It would be ideal if this could "just work" however.

Answer 3

@Ben Duguid

The most common cause of the 503 is usually a request timed out or the backend is unhealthy. While I understand that range and compression headers may have caused it, have you tried increasing the timeout value and test again?

Answer 4

Hi, the 503 response was happening within about 5-10 seconds, well within the default 30 second timeout.

I did try increasing the timeout to 120 seconds, but we still saw the 503 response within 5 seconds.

As mentioned, making the same request, with the range and accept-encoding headers, directly to the back-end pool resulted in a valid 206 Partial response, and stripping out the accept-encoding header from the request with a rule also succeeds through the front door, so this looks to be more of an issue within front door than with our application.

Answer 5

We're seeing the same behaviour with LinkedIn sharing as well, however that's still throwing some sort of 503, but only reporting "Unable to connect to server. Bad DNS, bad gateway, or invalid server address."

Again, sharing the backend pool directly works successfully, so this is not an obvious error with our application.