What's the possible reason leading difference peak in Server Load, Processor Time and CPU usage?
Shengyu Xiong
20
Reputation points Microsoft Employee
I am now working on a purge task to clear persistent redis region, and when I trigger task, I notice there exist difference in server load, processor time and CPU usage in different region.
Region A
Configuration
configuration Premium 26 GB (2 × 13 GB)
Dashboards
the peak indicates the trigger time
there is no obvious increase in Server Load, Processor Time and CPU usage at that period
- I trigger 3 times, every time dashboards are similar
Condition
- operation per second is 1.3-1.4 k before trigger
- get operation is slightly higher than set operation
Region B
Configuration
configuration Premium 26 GB (2 × 13 GB)
Dashboards
the peak indicates the trigger time
Volume
- Total keys ~6.5 million, purge task clear ~5.5 million keys
- Region A and Region B key distribution are similar
Condition
- operation per second is 1.8-1.9 k before trigger
- get operation is double than set operation
Purge Task Logic
// List of keys pattern that need get from redis
string[] patterns;
using (ConnectionMultiplexer connection = await ConnectionMultiplexer.ConnectAsync(config))
{
// get server
Iserver server = connection.GetServers().// filter server and check server logic, then get a serer
List<Task> tasks = new List<Task>(patterns.Length);
foreach (var pattern in patterns)
{
tasks.Add(RedisPersistentKeyPurgeAction(connection, server, pattern));
}
await Task.WhenAll(tasks).ConfigureAwait(false);
}
private async Task RedisPersistentKeyPurgeAction(ConnectionMultiplexer connection, IServer server, string pattern)
{
// batchExpire keys from redis
List<string> batchExpireBuffer = new List<string>(50);
var db = connection.GetDatabase();
await using var keys = server.KeysAsync(pattern: pattern).GetAsyncEnumerator();
bool isLastkey = !await keys.MoveNextAsync();
while (! isLastKey)
{
// every proccessed 100 keys, there will be a sleep
await Task.Delay(200);
// every 50 keys or current has reached last key matched, then batch set default time to live to those persistent keys
if (statisfy some condtion)
{
await BatchSetExpiry(batchExpireBuffer, expiry, db);
batchExpireBuffer.Clear();
}
// if it length of batchExpireBuffer < 50
if (batchExpireBuffer.Count < 50)
{
batchExpireBuffer.Add(keys.Current.ToString());
}
// other logic of iterator
....
}
}
private Task BatchSetExpiry(List<string> setExpiryList, int expiry, IDatabase db)
{
IBatch batch = db.CreateBatch();
foreach (var key in setExpiryList)
{
// expiry is default expire time, 12 hours
batch.KeyExpireAsync(key, TimeSpan.FromSeconds(expiry));
}
batch.Execute();
// omit other logic, e.g. exception handling
return Task.CompletedTask;
}
Sign in to answer