When Redis uses the advanced pricing layer, after cleaning up some keys, the current 400k key count occupies 1.7GB of memory. By exporting RDB, the RDB file is only a few tens of MB. Even if restored through the Azure portal, it only occupies the same size as the RDB file. There is a gap of about 40 times or more in the middle. Can you provide suggestions and an explanation of the reasons for this difference. How did such a large difference come about.

Hi Sun Tongxin3 - Thanks for reaching out. There can be difference between the current size of data in cache versus when it is Exported. However, once you export that data back, it should ideally reflect the size of cache data which was there previously unless there is any kind of data deletion that happened just before it was Exported. This might need a support ticket for a closer look at the metrics and the backend but let me try to check if there are any other possibilities and shall keep you posted.

The difference between the dataset size of Redis resources and the size after export and restoration is too large

Accepted answer

Amrinder Singh 5,155 Reputation points Microsoft Employee

2024-07-02T08:42:30.55+00:00

Hi Sun Tongxin3 - Thanks for reaching out.

There can be difference between the current size of data in cache versus when it is Exported. However, once you export that data back, it should ideally reflect the size of cache data which was there previously unless there is any kind of data deletion that happened just before it was Exported.

This might need a support ticket for a closer look at the metrics and the backend but let me try to check if there are any other possibilities and shall keep you posted.
Please sign in to rate this answer.
Sun Tongxin3 20 Reputation points

2024-07-03T02:07:29.9833333+00:00

Hi Amrinder Singh,

I am glad to receive your reply, and I can confirm that this is also my doubt.

I tried to parse RDB files （about 40MB） using the redis check rdb command line tool. The size of the use-mem seen (about 200MB) is also inconsistent with the used_memory_dataset (about 1.6GB) information viewed through info in the running Redis resource.

Firstly, before cleaning up the number of keys in the early stage, I thought it might be a compression issue, but after cleaning up the big keys later, the remaining usage of this dataset made me very distressed.

The size of the RDB exported from Redis data structure through the check tool is inconsistent with the size in the info, and the size of the restored RDB is also inconsistent with the size seen in the check.

Will there be additional data structure optimization adjustments during the process of RDB reconstruction, such as the possibility of redundancy in the data structure filled in earlier stages, resulting in significant differences in optimization when exporting and refilling?

We are unable to view the complete configuration of Azure Redis, and I would like to know if there is a solution to resolve the current false occupation.

Amrinder Singh 5,155 Reputation points Microsoft Employee

2024-07-04T12:27:48.5+00:00

Hi Sun Tongxin3 - Thanks for your patience.

I tried re-producing the scenario but unfortunately couldn't replicate the same.

At the time you imported the data back, in case there is any kind of node maintenance happens at the same time, there is another possibility of some data loss happening in that scenario.

I would recommend raising a support ticket in order to review the backend metrics and logs in order to understand what exactly happened in here.

Hope that helps!
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

The difference between the dataset size of Redis resources and the size after export and restoration is too large

0 additional answers

Your answer