Blob file is accessible from Cache without SAS token [using Azure Front Door and CDN Profile]

Sachin K 0

Hi,

I am using Azure Blob storage to store the file. To fetch the file I generate a URL with SAS token. Also, I am using the Azure Front Door and CDN Profile to cache the file. The file is accessible with the URL with SAS token. If I call the URL again it will get cached (I got X-Cache : TCP_HIT in the response header).

Problem
After caching the file, If I call the same URL without SAS token the file is accessible.

Question
How can I restrict the access of files from the cache without using the SAS token?

Below I have shared the caching settings in Azure Front Door and CDN Profile
Capture

1 answer

GitaraniSharma-MSFT 49,586 Reputation points Microsoft Employee

2024-01-19T13:22:08.14+00:00
Hello @Sachin K ,

Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.

I understand that your Azure Storage Blob file setup with Azure CDN is accessible from Cache without SAS token, and you would like to restrict the access of files from the cache. We have a document for using Azure CDN with SAS, which you can refer in this case:

https://learn.microsoft.com/en-us/azure/cdn/cdn-sas-storage-support

There are two options recommended for using SAS with Azure CDN:

Option 1: Using SAS with pass-through to blob storage from Azure CDN

Fine-tune the cache duration either by using caching rules or by adding Cache-Control headers at the origin server. Because Azure CDN treats the SAS token as a plain query string, as a best practice you should set up a caching duration that expires at or before the SAS expiration time. Otherwise, if a file is cached for a longer duration than the SAS is active, the file may be accessible from the Azure CDN origin server after the SAS expiration time has elapsed. If this situation occurs, and you want to make your cached file inaccessible, you must perform a purge operation on the file to clear it from the cache.

Option 2: Using CDN security token authentication with a rewrite rule:

To use Azure CDN security token authentication, you must have an Azure CDN Premium from Edgio profile. This option is the most secure and customizable.

I believe you are using option 1, so as suggested, you should set up a caching duration that expires at or before the SAS expiration time.

Select your Azure CDN endpoint, select Caching rules, then select Cache every unique URL from the Query string caching list.

And then go to the Rules engine of the Azure CDN endpoint and add the Global rule to set/override the cache expiration settings.

You should set the cache behavior to Override and set up a caching duration that expires at or before the SAS expiration time.

Override: Ignore origin-provided cache duration; use the provided cache duration instead. This setting doesn't override cache-control: no-cache.

Refer: https://learn.microsoft.com/en-us/azure/cdn/cdn-how-caching-works#default-caching-behavior

https://learn.microsoft.com/en-us/azure/cdn/cdn-caching-rules#caching-behavior-settings

Kindly let us know if the above helps or you need further assistance on this issue.

Please don’t forget to "Accept the answer" wherever the information provided helps you, this can be beneficial to other community members.
Please sign in to rate this answer.
Sachin K 0 Reputation points

2024-01-19T15:15:55.9333333+00:00

Hello @GitaraniSharma-MSFT ,
Thank you for your valuable response. Could you please check whether my following understandings are correct for the given options?

Option 1
I don't want to allow access to the file from the cache without SAS token, not even until the cache expiry time. So I cannot choose it.

**Option 2
'**Cache every unique URL' will not hit the cache because when URL is generated each time SAS token would be different. So I have to drop this too.

The problem at the Application level
Whenever the file in Blob storage is modified read the file from Blob storage and cache it and then for every successive request read it from the cache.

Is it possible, If so what settings should I follow?

Sachin K 0 Reputation points

2024-01-19T15:16:44.0066667+00:00

Please let me know If additional information is required. Thanks !!

GitaraniSharma-MSFT 49,586 Reputation points Microsoft Employee

2024-01-22T10:23:00.6966667+00:00

Hello @Sachin K ,

I don't want to allow access to the file from the cache without SAS token, not even until the cache expiry time.

If you don't want to allow access to the file from the cache without SAS token, then the better option would be to disable caching altogether.

Azure CDN/AFD won't validate the Access Token. It will be validated only if you directly access the storage account.

Cache every unique URL will not hit the cache because when URL is generated each time SAS token would be different.

This should not be the case with SAS token. Azure CDN treats the SAS token as a plain query string.

With Azure Content Delivery Network (CDN), you can control how files are cached for a web request that contains a query string. In a web request with a query string, the query string is that portion of the request that occurs after a question mark (?). A query string can contain one or more key-value pairs, in which the field name and its value are separated by an equals sign (=). Each key-value pair is separated by an ampersand (&). For example, http://www.contoso.com/content.mov?field1=value1&field2=value2. If there's more than one key-value pair in a query string of a request, their order doesn't matter.

Cache every unique URL: In this mode, each request with a unique URL, including the query string, is treated as a unique asset with its own cache. For example, the response from the origin server for a request for example.ashx?q=test1 is cached at the POP node and returned for subsequent caches with the same query string. A request for example.ashx?q=test2 is cached as a separate asset with its own time-to-live setting.

This mode is only recommended to NOT use in case when the query string contains parameters that will change with every request, such as a session ID or a username, because it will result in a low cache-hit ratio.

Refer: https://learn.microsoft.com/en-us/azure/cdn/cdn-how-caching-works#introduction-to-caching

https://learn.microsoft.com/en-us/azure/cdn/cdn-query-string

So, when a user with the SAS token requests the file access, the response from the origin server will be cached at the POP node and will be returned for subsequent caches with the same query string. When the query string changes, a new request is sent to the origin server again and then cached.

Could you please clarify what do you mean by "when URL is generated each time SAS token would be different"?

Also, your ask seems to contradict with your problem:

Problem: After caching the file, If I call the same URL without SAS token the file is accessible. Question: How can I restrict the access of files from the cache without using the SAS token?

Since your existing query string mode says: Ignore query strings, the CDN point-of-presence (POP) node passes the query strings from the requestor to the origin server on the first request and caches the asset. All subsequent requests for the asset that are served from the POP ignore the query strings until the cached asset expires.

So, this is an expected behavior in this default mode.

If you don't want to access the files without SAS tokens, then you should not use caching at all.

Or you could try to use the below query string mode in your rule:

Bypass caching for query strings: In this mode, requests with query strings aren't cached at the CDN POP node. The POP node retrieves the asset directly from the origin server and passes it to the requestor with each request.

Kindly let us know if the above helps or you need further assistance on this issue.

Regards,

Gita Sharma

Sachin K 0 Reputation points

2024-01-22T10:37:55.0266667+00:00

Hello @GitaraniSharma-MSFT ,

Thanks for the help. I understood the information provided.
Could you please provide a solution to the problem mentioned below? What settings or configurations I should set?

Whenever the file in Blob storage is modified read the file from Blob storage and cache it and then for every successive request read it from the cache.

GitaraniSharma-MSFT 49,586 Reputation points Microsoft Employee

2024-01-22T12:11:55.52+00:00

Hello @Sachin K ,

A cached resource isn't compared to the version on the origin server every time it's accessed. Instead, as long as a cached resource is considered to be fresh, it's assumed to be the most current version and is sent directly to the client. A cached resource is considered to be fresh when its age is less than the age or period defined by a cache setting.

You can control how caching is performed in a CDN by sending cache-directive headers. Cache-directive headers are HTTP headers, which are typically added by the origin server.

Two headers can be used to define cache freshness: Cache-Control and Expires. Cache-Control is more current and takes precedence over Expires, if both exist. There are also two types of headers used for validation (called validators): ETag and Last-Modified. ETag is more current and takes precedence over Last-Modified, if both are defined.

When the cache is stale, HTTP cache validators are used to compare the cached version of a file with the version on the origin server. Azure CDN Standard/Premium from Edgio supports both ETag and Last-Modified validators by default, while Azure CDN Standard from Microsoft supports only Last-Modified by default.

A cache validates a file using Last-Modified by sending an If-Modified-Since header with a date and time in the request. The origin server compares that date with the Last-Modified header of the latest resource. If the resource hasn't been modified since the specified time, the server returns status code 304 (Not Modified) in its response. If the resource has been modified, the server returns status code 200 (OK) and the updated resource.

Refer: https://learn.microsoft.com/en-us/azure/cdn/cdn-how-caching-works#resource-freshness

Azure storage blob supports Last-Modified HTTP header.

Last-Modified: The date/time when the blob was last modified. The date format follows RFC 1123. Any write operation on the blob (including updates on the blob's metadata or properties) changes the last modified time of the blob.

https://learn.microsoft.com/en-us/rest/api/storageservices/put-blob?tabs=microsoft-entra-id#response-headers

https://learn.microsoft.com/en-us/rest/api/storageservices/set-blob-properties?tabs=microsoft-entra-id#response-headers

So, in your configured global and custom caching rules, you should specify the following Caching behavior setting:

Set if missing: It honors origin-provided cache-directive headers, if they exist; otherwise, use the provided cache duration.

For the Override and Set if missing Caching behavior settings, valid cache durations range between 0 seconds and 366 days. For a value of 0 seconds, the CDN caches the content, but must revalidate each request with the origin server.

Refer: https://learn.microsoft.com/en-us/azure/cdn/cdn-caching-rules#caching-behavior-settings

So, you may set a Global rule to configure cache duration as below:

And configure your caching rule to set the query string caching behavior as below:

The above settings will make sure the below:

It will honor origin-provided cache-directive headers, if they exist; otherwise, the CDN will cache the content, but will revalidate each request with the origin server. If the file has been modified on the origin server, the cache updates its version of the resource.

And requests with query strings will not be cached at the CDN POP node. The POP node will retrieve the asset directly from the origin server and pass it to the requestor with each request.

Kindly let us know if the above helps or you need further assistance on this issue.

Regards,

Gita

Sachin K 0 Reputation points

2024-01-23T05:05:25.3+00:00

Hi @GitaraniSharma-MSFT

I could not find the settings mentioned below in Azure Front Door. Could you please provide settings and details related to AFD and CDN profile

GitaraniSharma-MSFT 49,586 Reputation points Microsoft Employee

2024-01-23T06:20:29.22+00:00

Hello @Sachin K ,

Could you please confirm if you are using Azure CDN or Azure Front Door?

Also, please share the provider/SKU of the respective product (for example: if you are using Azure CDN, then is it Standard Microsoft/Standard Edgio/Premium Edgio and if it is Azure Front Door, then is it Classic/Standard/Premium SKU)?

Regards,

Gita

Sachin K 0 Reputation points

2024-01-23T06:25:04.3066667+00:00

Hi @GitaraniSharma-MSFT ,

Sorry for the confusion. I am using Azure Front Door [Pricing Tier: Azure Front Door Standard].

Thanks & Regards

GitaraniSharma-MSFT 49,586 Reputation points Microsoft Employee

2024-01-23T07:32:05.06+00:00

Hello @Sachin K ,

Azure Front Door with CDN documents are as below:

https://learn.microsoft.com/en-us/azure/frontdoor/scenario-storage-blobs

https://learn.microsoft.com/en-us/azure/frontdoor/integrate-storage-account

Caching with Azure Front Door:

https://learn.microsoft.com/en-us/azure/frontdoor/front-door-caching?pivots=front-door-standard-premium

Cache behavior and duration can be configured in Rules Engine. Rules Engine caching configuration always overrides the route configuration.

So, you need to go to the Azure Front Door Rule Sets and configure a new rule as below:

Azure Front Door doesn't have a Bypass caching for query strings mode, so you can only choose from Ignore Query String or Use Query String or Ignore Specified Query Strings and Include Specified Query Strings.

You can set the query string behavior to either ignore or cache every unique URL and then set the cache duration to 0.

When Cache behavior is set to Override if origin missing, you must specify the cache duration to use. The maximum duration is 366 days. For a value of 0 seconds, the CDN caches the content, but must revalidate each request with the origin server.

Refer: https://learn.microsoft.com/en-us/azure/frontdoor/front-door-rules-engine-actions?pivots=front-door-standard-premium&tabs=portal#RouteConfigurationOverride

Kindly let us know if the above helps or you need further assistance on this issue.

Regards,

Gita

Sachin K 0 Reputation points

2024-01-23T11:48:52.31+00:00

Hi @GitaraniSharma-MSFT ,

If I chose 'Ignore Query String' at Azure Front Door as 'Query Caching behaviour', the Cache behavior is set to Override if origin missing and Cache duration to 0.

Does the following scenario work

When the Client requests for the File, the Cache will fetch the file from the Origin Server if the original file at the Origin server is modified or return the File from the Cache itself if no modification happens at the Origin server.

GitaraniSharma-MSFT 49,586 Reputation points Microsoft Employee

2024-01-23T14:50:59.28+00:00

Yes @Sachin K , the above configuration will work.

Sachin K 0 Reputation points

2024-01-24T08:42:17.9366667+00:00

Hi @GitaraniSharma-MSFT ,

I have made following settings

And I have tested the blob URL to fetch the file

Sample URL
https://{baseUrl}/{container}/{foldername}%5Cfilename.jpg?sv=2021-01-01&st=2022-01-24T08%3A29%3A46Z&se=2022-01-22T08%3A45%3A46Z&sr=b&sp=r&sig=Xhzsave6ygbkRVXnVS0DED3nJRyCSIyfoMUmcg9VgvY%3D

Step 1 : First call on URL

Step 2 : Second call on the same URL

Step : 3 Call the newly generated URL for the same file

On Step 3, should I get X-Cache : TCP_HIT, If the file is reading from the cache? but I'm getting TCP_MISS. So that means for each newly generated URL the image is taken from the Origin server rather than the cache*. Actually my requirement is the file should be taken from the cache even if URL is changed.*

GitaraniSharma-MSFT 49,586 Reputation points Microsoft Employee

2024-01-25T13:18:14.9866667+00:00

Hello @Sachin K ,

I've reached out to you offline in an email. Could you please respond to the email, so that we can take a closer look at the issue?

Regards,

Gita

Sachin K 0 Reputation points

2024-01-27T05:30:35.3233333+00:00

Hi @GitaraniSharma-MSFT ,
Sorry, I didn't receive any email. Could you please check?

GitaraniSharma-MSFT 49,586 Reputation points Microsoft Employee

2024-01-30T08:47:49.8866667+00:00

Hello @Sachin K , I've sent another email. Please check your inbox and respond accordingly.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.