Azure Front Door returns a 503 Service Unavailable when Range and Compression Headers are in the request

Question

We're seeing an issue with our Front Door configuration where requests from the Facebook crawler are consistently seeing a 503 Service Unavailable error for all requests.

This seems to be related to the combination of request headers used:

User-Agent: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
Accept-Encoding: deflate, gzip
Range: bytes=0-524288
Connection: close

Or in curl:

curl -v --compressed -H "Range: bytes=0-524288" -H "Connection: close" -A "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)" "$URL"

Requesting the page without the Accept-Encoding correctly returns a 206 Partial Content response.

Sending the full request to the origin/backend host directly (Azure App Service) also returns a valid response with the expected page content.

I've seen (and replied to) the same issue reported here: https://feedback.azure.com/forums/217313-networking/suggestions/38617117

Answer 1

Apologies all for not coming back on this.

@SaiKishor-MSFT Not really, no. The end result is that the social site (or anything else using the range header in combination with gzip encoding) causes Front Door to return a 503 when the backend is healthy and responding correctly. These are dynamic sites, so it's likely that headers or similar are slightly different between requests.

@Rich DeBourke Yes, we found the same, so I adjusted the rule to remove the Accepts-Encoding header if there was also a Range header. This ensures that all of these should work and get the correct 206 Partial response that is no larger than the requested value.

Answer 2

Seems that it's the gzip encoding with the range header that's the problem:

User-Agent: LinkedInBot/1.0 (compatible; Mozilla/5.0; Apache-HttpClient +http://www.linkedin.com)
Range: bytes=0-524288
Connection: close
Accept-Encoding: deflate

Works as expected (206, although not compressed), while:

User-Agent: LinkedInBot/1.0 (compatible; Mozilla/5.0; Apache-HttpClient +http://www.linkedin.com)
Range: bytes=0-524288
Connection: close
Accept-Encoding: gzip

Returns the 503 error.

Removing the Range header, but including either of the compression encodings works as expected (200, compressed with gzip).

Answer 3

This appears to be happening on a number of clients, mostly when we've enabled the Rules Engine - for a couple of sites we have that don't have any rules in the Rules engine, we're not seeing this issue.

Answer 4

@Ben Duguid I did a little more investigation regarding this and here is the issue that causes this 503-

When range header is added, frontdoor will reach out to origin twice(or more). If origin returns inconsistent content, we’ll fail the request.

Hope this helps. Please let me know if you have any further questions. Thank you!