Hi, we have an Azure Container App that is located in UK South and it sends queries to a table storage account that is also located in UK South. When the container app is configured to query the same dataset in a North Europe storage account the response times are much faster. Is there any reason why the table storage service located in UK South would be noticeably slower than the same service operating in North Europe? Thanks, Steve

Hi Steve, Are both the same type of storage account with same redundancy settings? For example, are both GPv2? Do both accounts handling about the same number of requests and storing about the same amount of data? Are you seeing the difference in speed reflected in the monitoring metrics? For example, if you run tests against each and compare Success E2E Latency and Success Server Latency what are you seeing? Based on information provided in comments below I recommend you open a new support case via the Azure portal. A support engineer can investigate the storage backend in UK South to find out why it is so much slower than North Europe for same workload. Please click Accept Answer if the above and related comments have been helpful. Thanks. -TP

UK South Table Storage performance noticeably slower than North Europe

Accepted answer

TP 102.6K Reputation points

2023-07-28T12:10:29.06+00:00

Hi Steve,

Are both the same type of storage account with same redundancy settings? For example, are both GPv2?

Do both accounts handling about the same number of requests and storing about the same amount of data?

Are you seeing the difference in speed reflected in the monitoring metrics? For example, if you run tests against each and compare Success E2E Latency and Success Server Latency what are you seeing?

Based on information provided in comments below I recommend you open a new support case via the Azure portal. A support engineer can investigate the storage backend in UK South to find out why it is so much slower than North Europe for same workload.

Please click Accept Answer if the above and related comments have been helpful.

Thanks.

-TP
Please sign in to rate this answer.
Steve 45 Reputation points

2023-07-28T12:57:46.15+00:00

Hi TP, thanks for taking the time to respond.

Both storage account kinds are GPv2.

The storage account in UK south has GZRS enabled for redundancy.

The storage account in North Europe has ZRS enabled for redundancy.

Both accounts store the same data and receive the same volume of transactions.

Success E2E Latency for the UK South Table Storage Account is 105.09ms(average over 24hr)

Success E2E Latency for the North Europe Storage Account is 47.87ms(average over 24hr)

The storage account in UK South has hierarchical namespace enabled for blobs but I w.uldn't expect this to effect table storage performance.

I have also setup a service endpoint(Microsoft.Storage.Global) on the VNET that is used by the Container App Environment to ensure that requests from the Container App running in UK South to table storage in North Europe are routed optimally via the Azure backbone network.

Thanks,

Steve

TP 102.6K Reputation points

2023-07-28T13:02:30.3333333+00:00

Is Success Server Latency running near the same for both? I'm curious since it doesn't include any network latency. I know, in theory UK South should have less network latency since it is in same region.

Steve 45 Reputation points

2023-07-28T13:12:19.25+00:00

For UK south success latency is 2ms below E2E latency

For North Europe its 5ms behind E2E latency.

Steve 45 Reputation points

2023-07-28T13:20:24.0266667+00:00

Would GZRS add additional latency? The storage account in North Europe only has ZRS.

TP 102.6K Reputation points

2023-07-28T13:23:51.62+00:00

Okay, thanks. I think you should open a case with Azure support via the portal and have them investigate the backend. It could be for some unknown reason the storage stamp you are on in UK South is running slower than North Europe.

In general I try to fix things without involving support as much as possible, but based on the information you have provided there is no clear reason that I can see why UK South would be so much slower. Of course there is option to upgrade to Cosmos DB but that involves greater cost and may not suit your needs.

TP 102.6K Reputation points

2023-07-28T13:28:15.9366667+00:00

GZRS should not add latency because the replication to the secondary region is asynchronous. The saving to all three zones is synchronous so adds delay, but the replication to the paired Azure region (the "G" in GZRS) is asynchronous.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

UK South Table Storage performance noticeably slower than North Europe

0 additional answers

Your answer