What's the possible reason leading difference peak in Server Load, Processor Time and CPU usage?

Shengyu Xiong 20 Reputation points Microsoft Employee
2024-07-07T13:52:03.0666667+00:00

I am now working on a purge task to clear persistent redis region, and when I trigger task, I notice there exist difference in server load, processor time and CPU usage in different region.

Region A

Configuration

configuration Premium 26 GB (2 × 13 GB)

Dashboards

the peak indicates the trigger time

User's image

there is no obvious increase in Server Load, Processor Time and CPU usage at that period

  • I trigger 3 times, every time dashboards are similar

Condition

  • operation per second is 1.3-1.4 k before trigger
  • get operation is slightly higher than set operation

User's image

Region B

Configuration

configuration Premium 26 GB (2 × 13 GB)

Dashboards

the peak indicates the trigger time

User's image

Volume

  • Total keys ~6.5 million, purge task clear ~5.5 million keys
  • Region A and Region B key distribution are similar

Condition

  • operation per second is 1.8-1.9 k before trigger
  • get operation is double than set operation

User's image

Purge Task Logic

// List of keys pattern that need get from redis
string[] patterns;

 using (ConnectionMultiplexer connection = await ConnectionMultiplexer.ConnectAsync(config))
 {
    // get server
    Iserver server = connection.GetServers().// filter server and check server logic, then get a serer
    List<Task> tasks = new List<Task>(patterns.Length);

    foreach (var pattern in patterns)
    {
        tasks.Add(RedisPersistentKeyPurgeAction(connection, server, pattern));
    }
    await Task.WhenAll(tasks).ConfigureAwait(false);
}

private async Task RedisPersistentKeyPurgeAction(ConnectionMultiplexer connection, IServer server, string pattern)
{ 
     // batchExpire keys from redis
    List<string> batchExpireBuffer = new List<string>(50);

    var db = connection.GetDatabase();
    await using var keys = server.KeysAsync(pattern: pattern).GetAsyncEnumerator();
    bool isLastkey = !await keys.MoveNextAsync();

    while (! isLastKey) 
    {
        // every proccessed 100 keys, there will be a sleep
        await Task.Delay(200);

        // every 50 keys or current has reached last key matched, then batch set default time to live to those persistent keys
        if (statisfy some condtion)
         {
             await BatchSetExpiry(batchExpireBuffer, expiry, db);
             batchExpireBuffer.Clear();
          }
         // if it length of batchExpireBuffer < 50 
         if (batchExpireBuffer.Count < 50)
         {
             batchExpireBuffer.Add(keys.Current.ToString());
         }
         // other logic of iterator
         ....
    }
}

 private Task BatchSetExpiry(List<string> setExpiryList, int expiry, IDatabase db)
 {
         IBatch batch = db.CreateBatch();

         foreach (var key in setExpiryList)
         {
            // expiry is default expire time, 12 hours
             batch.KeyExpireAsync(key, TimeSpan.FromSeconds(expiry));
         }

         batch.Execute();
       // omit other logic, e.g. exception handling
        return Task.CompletedTask;
 }
Azure Cache for Redis
Azure Cache for Redis
An Azure service that provides access to a secure, dedicated Redis cache, managed by Microsoft.
251 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.