Why is Azure Front Door Premium not setting session affinity cookie on initial requests?

Question

Why is Azure Front Door Premium not setting session affinity cookie on initial requests?

Michael Glover 36

We are experiencing errors in our Azure App Service caused by our Azure Premium Front Door instance not setting its session affinity cookies, ASLBSA/ASLBSACORS, on the initial browser requests. I would expect to see a "set-cookie" response header for the origin cookie on the initial page load with Front Door, but that is usually not the case. Is this expected behavior from Front Door, or is it a bug? Specifically is there a path forward for us to use Front Door with our App Services and route to the correct app instance?

Background:

We have a stateful C# DotNet 6.0 application. It is deployed to a pair of Azure App Services in separate regions, with a Premium Azure Front Door in front of it. Both App Services have "ARR affinity" set to "on". The Front Door Origin Group is configured with "Session affinity/Enable session affinity" checked on.

Any initial request to the application will generate an AntiForgeryToken in the C# code that is associated with that individual web app instance. We set the AntiForgeryToken in the CSHTML using @Html.AntiForgeryToken() and validate it on subsequent calls using the ValidateAntiForgeryToken decorator on controller methods.

Any following HTTP requests must be routed to the web app instance where the AntiForgeryToken was initially set. If any following request is sent to a different web app instance that instance is unable to decrypt the AntiForgeryToken. This generates an HTTP 400 response caused by a System.Security.Cryptography.CryptographicException in the code.

The Azure App Services set an ARRAffinity cookie immediately on any initial request so any calls routed to the web app get sent to the correct web app instance if the app has been scaled to two or more instances. We have not experienced any issues with the request routing to individual instances once the call is routed to the web app.

Azure Premium Front Door routes requests to the correct web app by setting and reading the ASLBSA/ASLBSACORS cookies. We see that usually these cookies are not being set on the initial page load, but appear later after the intial document load when other dependent resources are being loaded. Occasionally, before the ASLBSA/ASLBSACORS are set, these requests are switched from one web app to another. This switch and, later, setting of the ASLBSA/ASLBSACORS cookies, causes Front Door to route following document requests to the web app where the AntiForgeryToken was not set.

Example of the error:

In this example both web apps are manually scaled to 1 instance. There can only be two values for the ARRAffinity cookie, one for each web app. From a new Chrome incognito window, on the initial page load, here is a screenshot of the document response headers.

· AntiForgery “set-cookie” response header value beginning with “CfDJ” ending with “esIA”, marked in red

· ARRAffinity “set-cookie” response header value beginning with “76f6”, marked in blue

· No ABSLCA cookies present or set in the initial response.

OriginInvalid01

Screenshot of the request headers, for reference:

Screenshot 2023-02-17 114835

"Cookies" view of the initial request:

OriginInvalid03

At this point the AntiForgeryToken has been set on the web app instance that corresponds to the ARRAffinity cookie "76f6". There is not yet any tracking by Front Door on which of the two web apps this instance is on.

After the initial document load there are various resources that load. Here is a screenshot of the last resource to be loaded after the initial document load, the "favicon.ico":

· AntiForgery "set-cookie" response header value beginning with "9fXo" ending with “esIA”, marked in red. Still set from the initial document load.

· ARRAffinity "set-cookie" response header value beginning with "33f4", marked in blue. This is a different cookie than the initial load.

· At this point the ASLBSA/ASLBSACORS cookies are present, beginning with "0003" and ending with "afbc", marked in green.

OriginInvalid02

"Cookies" view of the final resource request:

OriginInvalid04

The initial document load request went to the web app that had instance "76f6". This instance created the AntiForgeryToken. Before the final resource load called from the document, the ASLBSA cookie had been set, but was pointing to the web app that the initial request did not go to. The ARRAffinity token had been reset to "33f4". The AntiForgeryToken cannot be decrypted or used by the "33f4" instance. Any following requests sent to Front Door will be routed to that web app and instance, and will cause an error.

Second browser request:

Screenshot 2023-02-17 115108

"Cookies" view of second browser request:

Screenshot 2023-02-17 115123

From the browser:

Screenshot 2023-02-17 115041

Application Insights, showing the System.Security.Cryptography.CryptographicException generated on the web app instance that did not intially set the AntiForgetToken:

Screenshot 2023-02-17 115733

AzureDiagnostic log entry from Front Door:

Screenshot 2023-02-17 115526

We have seen a few instances where the ASLBSA/ASLBSACORS cookies are explicitly set in the first call. This is the behavior I would expect.

Screenshot 2023-02-17 110336

GitaraniSharma-MSFT 50,096 Reputation points Microsoft Employee Moderator

2023-03-06T12:27:15.4333333+00:00

@Michael Glover , Could you please provide an update on this post? Kindly let us know if the below helped or you need further assistance on this issue.

Accepted answer

1 additional answer

Your answer

GitaraniSharma-MSFT 50,096 Reputation points Microsoft Employee Moderator

2023-03-06T12:27:15.4333333+00:00

@Michael Glover , Could you please provide an update on this post? Kindly let us know if the below helped or you need further assistance on this issue.

Answer 1

GitaraniSharma-MSFT 50,096 Microsoft Employee Moderator

Hello @Michael Glover ,

Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.

I understand that you are experiencing errors in your Azure App Service caused by Azure Front Door Premium instance not setting its session affinity cookies, ASLBSA/ASLBSACORS, on the initial browser requests.

Looking at the screenshot of the document response headers on the initial page load provided by you, I see that the response contains a cache-control header = no-cache, no-store.

User's image

As mentioned in the Azure Front Door Premium Session Affinity doc,

Session affinity will be established in the following circumstances beyond the standard non-cacheable scenarios:

The response must include the Cache-Control header of no-store.
If the response contains an Authorization header, it must not be expired.
The response is an HTTP 302 status code.
And lastly, the backend pools must be HEALTHY.

Below is an example of Successful session affinity:

User's image

Now, let's see the points when session affinity will not occur.

Session affinity will NOT occur:

if the HTTP Response is a 304 (not modified)
if the backend sends a CACHABLE Response (Example: Blank field)
Cache Control-Header = NO-CACHE. This is because “no-cache doesn’t mean “don’t cache”, it means it must revalidate with the server before using the cached resource.

Cache-control explanation link: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control

Another important point: Public proxies may interfere with session affinity. This is because establishing a session requires Front Door to add a session affinity cookie to the response, which cannot be done if the response is cacheable as it would disrupt the cookies of other clients requesting the same resource. To protect against this, session affinity will not be established if the origin sends a cacheable response when this is attempted. If the session has already been established, it does not matter if the response from the origin is cacheable.

I see that you've mentioned "We have seen a few instances where the ASLBSA/ASLBSACORS cookies are explicitly set in the first call. This is the behavior I would expect". Though the screenshot is missing, I believe it works for you sometimes.

When I load the site on my end, I can see the ASLBSA/ASLBSACORS cookies present or set in the initial response.

Below is the screenshot of the initial page load response headers on my MS edge new in-private window:

User's image

I also tried to load the site on a new chrome In-Cognito browser, and I can still see the ASLBSA/ASLBSACORS cookies present or set in the initial response. Screenshot below:

User's image

I tried to load the site multiple times (every time with a new browser window, also tried with a colleague's machine) and each time I can see the ASLBSA/ASLBSACORS cookies set in the initial response.

I'm not really sure if it is the Cache-Control header on your end or some public proxy but were there any changes made? Could you please confirm if it is still not working on your end?

If it is still an issue on your end, could we make sure that the response only has the Cache-Control header of no-store and try again?

And if the issue still happens, the best approach would be to raise a support request and engage the backend team for assistance. So, if you have a support plan, I request you file a support ticket, else please do let us know, we will try and help you get a one-time free technical support.

Kindly let us know if the above helps or you need further assistance on this issue.

Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

Michael Glover 36 Reputation points

2023-02-24T17:14:34.61+00:00

Thank you for the quick response! I have updated the broken images in the original question.

As you pointed out I believe the "Cache-Control" response header of "no-cache" coming from our application is likely causing our issue. The AntiForgeryToken code is setting the "no-cache" header, but I believe we have a workaround. I'll come back and update after we implement and test the fix to confirm that was the issue.

We hadn't made any changes to the site when you tested it. I did some further testing on multiple machines and noticed that when the browser sends a request header of "cache-control : no-cache" on the initial request the response doesn't set the ASLBSA cookie, but if that request header isn't present, it will set it every time on the initial request. This information may help someone troubleshooting similar issues in the future.

From a machine that sets the cookie every time on the initial request:

From a machine that does not set the affinity cookie every time on the initial request:
GitaraniSharma-MSFT 50,096 Reputation points Microsoft Employee Moderator

2023-03-02T07:45:07.85+00:00

Hello @Michael Glover , just checking in to see if there are any updates on this issue. Were you able to implement the workaround to remove the "Cache-Control" response header of "no-cache" coming from your application?
Michael Glover 36 Reputation points

2023-03-06T22:35:59.6333333+00:00
@GitaraniSharma-MSFT thank you for checking with us! We’re still testing our workaround, but I do have some additional information.

The “no-cache” headers aren’t affecting whether or not the ASLBSA cookie on the initial request. The “no-cache” in the request header I commented on was a red herring. We also have examples of the cookie being set and not being set when the “no-cache” response header is present and not present.

The difference on when the ASLBSA cookie gets set on the initial request or later is somehow tied to the TLS cipher used to encrypt the connection. The TLS cipher negotiated during the connection varies based on the Front Door POP, as different servers on different POPs have different TLS preferences.

Every time we connect to chahtaachvffastaging.choctawnation.com using Front Door we negotiate TLSv1.2 to one of two ciphers:

TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

These show up in the Front Door Access Log in the securityCipher_s field as:

TLS_ECDH_EPHEM_AES_256_SHA_384

ECDHE-RSA-AES128-GCM-SHA256

The ASLBSA cookie is never set during the initial request when the connection uses TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 as the cipher.

From Chrome developer tools, incognito window, “Security” tab:

ASLBSA Cookie not being set on the initial response:

The ASLBSA cookie is always set during the initial request when the connection sets TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 as the cipher.

ASLBSA cookie set on the initial response:

If you retest and check the TLSv1.2 cipher your browser is using it will correspond to the cookie being set or not set. I believe this is why you weren't able to reproduce the issue locally.

Some Azure Front Door POPs always negotiate to one cipher or the other, and a few negotiate both. This behavior seems to be affected by the server TLS cipher preferences, which are different between POPs and sometimes different for the various servers that server traffic in a single POP.

Statistics from our production site, receiving traffic nationwide:

To be clear this is the SSL connection between the client and Front Door. Front Door then makes an SSL connection from Front Door to the app service. We tried restricting the app service to certain ciphers to see if that would affect the cipher used by Front Door but it had no effect. From reading the documentation it appears that there is no way to restrict the cipher in a Premium Front Door instance.

You can use Nmap to confirm which ciphers are supported by the Front Door POP servers:

I’ve been able to determine some of the current IPs of the POP servers by changing internet connections, using Azure VMs based in different regions, and looking at global DNS propagation for chahtaachvffa.choctawnation.com. I can confirm that the IP is tied to the POP by hitting the URL and seeing which POP server shows up in the logs while looking at WireShark for the IP returned by the browser’s DNS query. It seems strange that the TLS cipher would be causing this issue. It also seems strange that the preferred TLS ciphers from different Front Door POPs, and even the servers inside a POP have different preferences. Is there anything that you know of that we might have overlooked that we could do to always use the TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 cipher?
GitaraniSharma-MSFT 50,096 Reputation points Microsoft Employee Moderator

2023-03-08T07:18:55.72+00:00

Hello @Michael Glover , thank you for the update. TLS ciphers affecting session affinity seems a bit weird. To be honest, I've not seen such an issue before.

I tried accessing the site again and I can see the cipher AES_256_GCM as you mentioned.

However, as of today, Azure Front Door doesn’t support disabling or configuring specific cipher suites for your profile.

Refer: https://learn.microsoft.com/en-us/azure/frontdoor/end-to-end-tls?pivots=front-door-standard-premium#supported-cipher-suites

I believe creating a support request for this issue would be the right approach. The support team can engage the backend team for further troubleshooting and debugging of this issue. So, if you have a support plan, I request you file a support ticket, else please do let us know, we will try and help you get a one-time free technical support.

Regards,

Gita
Michael Glover 36 Reputation points

2023-03-08T17:53:43.9733333+00:00

I'll do that, thank you for your help!

Answer 2

Michael Glover 36

For the benefit of anyone else who may come along with this issue, we were able to mitigate the affinity cookie issue by implemented DotNetCore Shared Data protection. It allowed us to share the encryption keys between app service instances, even in different regions, so we are no longer required to land at a specific app service instance. We still have our affinity cookies set at the app service level and the Front Door level, but when we experience the "wrong origin" from the Front Door origin cookie not setting we are still able to decrypt the AntiForgeryToken on the app service instance we land at.

GitaraniSharma-MSFT 50,096 Reputation points Microsoft Employee Moderator

2023-03-29T12:12:37.5733333+00:00

@Michael Glover , thank you for sharing the mitigation steps.
Nat Wallbank 45 Reputation points

2023-06-27T14:40:22.3833333+00:00

I had this issue too in the past few days and realised it was because the first request was usually to my /signin endpoint, which is public. Authenticated endpoints didn't seem to have the same issue.

I found that setting Cache-Control: no-store for all relevant endpoints resolved the issue.

Share via

Why is Azure Front Door Premium not setting session affinity cookie on initial requests?

1 additional answer

Your answer