Training
Module
Improve performance with a cache in a .NET Aspire project - Training
In this module, you'll learn about caches in a .NET Aspire cloud-native app and how to use them to optimize the performance of your microservices.
This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
In this article, you'll learn about various caching mechanisms. Caching is the act of storing data in an intermediate-layer, making subsequent data retrievals faster. Conceptually, caching is a performance optimization strategy and design consideration. Caching can significantly improve app performance by making infrequently changing (or expensive to retrieve) data more readily available. This article introduces the two primary types of caching, and provides sample source code for both:
Important
There are two MemoryCache
classes within .NET, one in the System.Runtime.Caching
namespace and the other in the Microsoft.Extensions.Caching
namespace:
While this article focuses on caching, it doesn't include the System.Runtime.Caching
NuGet package. All references to MemoryCache
are within the Microsoft.Extensions.Caching
namespace.
All of the Microsoft.Extensions.*
packages come dependency injection (DI) ready, both the IMemoryCache and IDistributedCache interfaces can be used as services.
In this section, you'll learn about the Microsoft.Extensions.Caching.Memory package. The current implementation of the IMemoryCache is a wrapper around the ConcurrentDictionary<TKey,TValue>, exposing a feature-rich API. Entries within the cache are represented by the ICacheEntry, and can be any object
. The in-memory cache solution is great for apps that run on a single server, where all the cached data rents memory in the app's process.
Tip
For multi-server caching scenarios, consider the Distributed caching approach as an alternative to in-memory caching.
The consumer of the cache has control over both sliding and absolute expirations:
Setting an expiration will cause entries in the cache to be evicted if they're not accessed within the expiration time allotment. Consumers have additional options for controlling cache entries, through the MemoryCacheEntryOptions. Each ICacheEntry is paired with MemoryCacheEntryOptions which exposes expiration eviction functionality with IChangeToken, priority settings with CacheItemPriority, and controlling the ICacheEntry.Size. Consider the following extension methods:
To use the default IMemoryCache implementation, call the AddMemoryCache extension method to register all the required services with DI. In the following code sample, the generic host is used to expose DI functionality:
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
HostApplicationBuilder builder = Host.CreateApplicationBuilder(args);
builder.Services.AddMemoryCache();
using IHost host = builder.Build();
Depending on your .NET workload, you might access the IMemoryCache
differently, such as constructor injection. In this sample, you use the IServiceProvider
instance on the host
and call generic GetRequiredService<T>(IServiceProvider) extension method:
IMemoryCache cache =
host.Services.GetRequiredService<IMemoryCache>();
With in-memory caching services registered, and resolved through DI, you're ready to start caching. This sample iterates through the letters in the English alphabet 'A' through 'Z'. The record AlphabetLetter
type holds the reference to the letter, and generates a message.
file record AlphabetLetter(char Letter)
{
internal string Message =>
$"The '{Letter}' character is the {Letter - 64} letter in the English alphabet.";
}
Tip
The file
access modifier is used on the AlphabetLetter
type, as it's defined within and only accessed from the Program.cs file. For more information, see file (C# Reference). To see the full source code, see the Program.cs section.
The sample includes a helper function that iterates through the alphabet letters:
static async ValueTask IterateAlphabetAsync(
Func<char, Task> asyncFunc)
{
for (char letter = 'A'; letter <= 'Z'; ++letter)
{
await asyncFunc(letter);
}
Console.WriteLine();
}
In the preceding C# code:
Func<char, Task> asyncFunc
is awaited on each iteration, passing the current letter
.To add items to the cache call one of the Create
, or Set
APIs:
var addLettersToCacheTask = IterateAlphabetAsync(letter =>
{
MemoryCacheEntryOptions options = new()
{
AbsoluteExpirationRelativeToNow =
TimeSpan.FromMilliseconds(MillisecondsAbsoluteExpiration)
};
_ = options.RegisterPostEvictionCallback(OnPostEviction);
AlphabetLetter alphabetLetter =
cache.Set(
letter, new AlphabetLetter(letter), options);
Console.WriteLine($"{alphabetLetter.Letter} was cached.");
return Task.Delay(
TimeSpan.FromMilliseconds(MillisecondsDelayAfterAdd));
});
await addLettersToCacheTask;
In the preceding C# code:
addLettersToCacheTask
delegates to IterateAlphabetAsync
and is awaited.Func<char, Task> asyncFunc
is argued with a lambda.MemoryCacheEntryOptions
is instantiated with an absolute expiration relative to now.AlphabetLetter
object is instantiated, and passed into Set along with letter
and options
.For each letter in the alphabet, a cache entry is written with an expiration, and post eviction callback.
The post eviction callback writes the details of the value that was evicted to the console:
static void OnPostEviction(
object key, object? letter, EvictionReason reason, object? state)
{
if (letter is AlphabetLetter alphabetLetter)
{
Console.WriteLine($"{alphabetLetter.Letter} was evicted for {reason}.");
}
};
Now that the cache is populated, another call to IterateAlphabetAsync
is awaited, but this time you'll call IMemoryCache.TryGetValue:
var readLettersFromCacheTask = IterateAlphabetAsync(letter =>
{
if (cache.TryGetValue(letter, out object? value) &&
value is AlphabetLetter alphabetLetter)
{
Console.WriteLine($"{letter} is still in cache. {alphabetLetter.Message}");
}
return Task.CompletedTask;
});
await readLettersFromCacheTask;
If the cache
contains the letter
key, and the value
is an instance of an AlphabetLetter
it's written to the console. When the letter
key is not in the cache, it was evicted and its post eviction callback was invoked.
The IMemoryCache
comes with many convenience-based extension methods, including an asynchronous GetOrCreateAsync
:
The entire sample app source code is a top-level program and requires two NuGet packages:
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
HostApplicationBuilder builder = Host.CreateApplicationBuilder(args);
builder.Services.AddMemoryCache();
using IHost host = builder.Build();
IMemoryCache cache =
host.Services.GetRequiredService<IMemoryCache>();
const int MillisecondsDelayAfterAdd = 50;
const int MillisecondsAbsoluteExpiration = 750;
static void OnPostEviction(
object key, object? letter, EvictionReason reason, object? state)
{
if (letter is AlphabetLetter alphabetLetter)
{
Console.WriteLine($"{alphabetLetter.Letter} was evicted for {reason}.");
}
};
static async ValueTask IterateAlphabetAsync(
Func<char, Task> asyncFunc)
{
for (char letter = 'A'; letter <= 'Z'; ++letter)
{
await asyncFunc(letter);
}
Console.WriteLine();
}
var addLettersToCacheTask = IterateAlphabetAsync(letter =>
{
MemoryCacheEntryOptions options = new()
{
AbsoluteExpirationRelativeToNow =
TimeSpan.FromMilliseconds(MillisecondsAbsoluteExpiration)
};
_ = options.RegisterPostEvictionCallback(OnPostEviction);
AlphabetLetter alphabetLetter =
cache.Set(
letter, new AlphabetLetter(letter), options);
Console.WriteLine($"{alphabetLetter.Letter} was cached.");
return Task.Delay(
TimeSpan.FromMilliseconds(MillisecondsDelayAfterAdd));
});
await addLettersToCacheTask;
var readLettersFromCacheTask = IterateAlphabetAsync(letter =>
{
if (cache.TryGetValue(letter, out object? value) &&
value is AlphabetLetter alphabetLetter)
{
Console.WriteLine($"{letter} is still in cache. {alphabetLetter.Message}");
}
return Task.CompletedTask;
});
await readLettersFromCacheTask;
await host.RunAsync();
file record AlphabetLetter(char Letter)
{
internal string Message =>
$"The '{Letter}' character is the {Letter - 64} letter in the English alphabet.";
}
Feel free to adjust the MillisecondsDelayAfterAdd
and MillisecondsAbsoluteExpiration
values to observe the changes in behavior to the expiration and eviction of cached entries. The following is sample output from running this code. Due to the non-deterministic nature of .NET events, your output might be different.
A was cached.
B was cached.
C was cached.
D was cached.
E was cached.
F was cached.
G was cached.
H was cached.
I was cached.
J was cached.
K was cached.
L was cached.
M was cached.
N was cached.
O was cached.
P was cached.
Q was cached.
R was cached.
S was cached.
T was cached.
U was cached.
V was cached.
W was cached.
X was cached.
Y was cached.
Z was cached.
A was evicted for Expired.
C was evicted for Expired.
B was evicted for Expired.
E was evicted for Expired.
D was evicted for Expired.
F was evicted for Expired.
H was evicted for Expired.
K was evicted for Expired.
L was evicted for Expired.
J was evicted for Expired.
G was evicted for Expired.
M was evicted for Expired.
N was evicted for Expired.
I was evicted for Expired.
P was evicted for Expired.
R was evicted for Expired.
O was evicted for Expired.
Q was evicted for Expired.
S is still in cache. The 'S' character is the 19 letter in the English alphabet.
T is still in cache. The 'T' character is the 20 letter in the English alphabet.
U is still in cache. The 'U' character is the 21 letter in the English alphabet.
V is still in cache. The 'V' character is the 22 letter in the English alphabet.
W is still in cache. The 'W' character is the 23 letter in the English alphabet.
X is still in cache. The 'X' character is the 24 letter in the English alphabet.
Y is still in cache. The 'Y' character is the 25 letter in the English alphabet.
Z is still in cache. The 'Z' character is the 26 letter in the English alphabet.
Since the absolute expiration (MemoryCacheEntryOptions.AbsoluteExpirationRelativeToNow) is set, all the cached items will eventually be evicted.
One common strategy for caching data, is updating the cache independently from the consuming data services. The Worker Service template is a great example, as the BackgroundService runs independent (or in the background) from the other application code. When an application starts running that hosts an implementation of the IHostedService, the corresponding implementation (in this case the BackgroundService
or "worker") start running in the same process. These hosted services are registered with DI as singletons, through the AddHostedService<THostedService>(IServiceCollection) extension method. Other services can be registered with DI with any service lifetime.
Important
The service lifetime's are very important to understand. When you call AddMemoryCache to register all of the in-memory caching services, the services are registered as singletons.
Imagine you're developing a photo service that relies on third-party API accessible via HTTP. This photo data doesn't change very often, but there is a lot of it. Each photo is represented by a simple record
:
namespace CachingExamples.Memory;
public readonly record struct Photo(
int AlbumId,
int Id,
string Title,
string Url,
string ThumbnailUrl);
In the following example, you'll see several services being registered with DI. Each service has a single responsibility.
using CachingExamples.Memory;
HostApplicationBuilder builder = Host.CreateApplicationBuilder(args);
builder.Services.AddMemoryCache();
builder.Services.AddHttpClient<CacheWorker>();
builder.Services.AddHostedService<CacheWorker>();
builder.Services.AddScoped<PhotoService>();
builder.Services.AddSingleton(typeof(CacheSignal<>));
using IHost host = builder.Build();
await host.StartAsync();
In the preceding C# code:
HttpClient
instance is registered for the CacheWorker
class with AddHttpClient<TClient>(IServiceCollection).CacheWorker
class is registered with AddHostedService<THostedService>(IServiceCollection).PhotoService
class is registered with AddScoped<TService>(IServiceCollection).CacheSignal<T>
class is registered with AddSingleton.host
is instantiated from the builder, and started asynchronously.The PhotoService
is responsible for getting photos that match given criteria (or filter
):
using Microsoft.Extensions.Caching.Memory;
namespace CachingExamples.Memory;
public sealed class PhotoService(
IMemoryCache cache,
CacheSignal<Photo> cacheSignal,
ILogger<PhotoService> logger)
{
public async IAsyncEnumerable<Photo> GetPhotosAsync(Func<Photo, bool>? filter = default)
{
try
{
await cacheSignal.WaitAsync();
Photo[] photos =
(await cache.GetOrCreateAsync(
"Photos", _ =>
{
logger.LogWarning("This should never happen!");
return Task.FromResult(Array.Empty<Photo>());
}))!;
// If no filter is provided, use a pass-thru.
filter ??= _ => true;
foreach (Photo photo in photos)
{
if (!default(Photo).Equals(photo) && filter(photo))
{
yield return photo;
}
}
}
finally
{
cacheSignal.Release();
}
}
}
In the preceding C# code:
IMemoryCache
, CacheSignal<Photo>
, and ILogger
.GetPhotosAsync
method:
Func<Photo, bool> filter
parameter, and returns an IAsyncEnumerable<Photo>
._cacheSignal.WaitAsync()
to release, this ensures that the cache is populated before accessing the cache._cache.GetOrCreateAsync()
, asynchronously getting all of the photos in the cache.factory
argument logs a warning, and returns an empty photo array - this should never happen.yield return
.Consumers of this service are free to call GetPhotosAsync
method, and handle photos accordingly. No HttpClient
is required as the cache contains the photos.
The asynchronous signal is based on an encapsulated SemaphoreSlim instance, within a generic-type constrained singleton. The CacheSignal<T>
relies on an instance of SemaphoreSlim
:
namespace CachingExamples.Memory;
public sealed class CacheSignal<T>
{
private readonly SemaphoreSlim _semaphore = new(1, 1);
/// <summary>
/// Exposes a <see cref="Task"/> that represents the asynchronous wait operation.
/// When signaled (consumer calls <see cref="Release"/>), the
/// <see cref="Task.Status"/> is set as <see cref="TaskStatus.RanToCompletion"/>.
/// </summary>
public Task WaitAsync() => _semaphore.WaitAsync();
/// <summary>
/// Exposes the ability to signal the release of the <see cref="WaitAsync"/>'s operation.
/// Callers who were waiting, will be able to continue.
/// </summary>
public void Release() => _semaphore.Release();
}
In the preceding C# code, the decorator pattern is used to wrap an instance of the SemaphoreSlim
. Since the CacheSignal<T>
is registered as a singleton, it can be used across all service lifetimes with any generic type—in this case, the Photo
. It is responsible for signaling the seeding of the cache.
The CacheWorker
is a subclass of BackgroundService:
using System.Net.Http.Json;
using Microsoft.Extensions.Caching.Memory;
namespace CachingExamples.Memory;
public sealed class CacheWorker(
ILogger<CacheWorker> logger,
HttpClient httpClient,
CacheSignal<Photo> cacheSignal,
IMemoryCache cache) : BackgroundService
{
private readonly TimeSpan _updateInterval = TimeSpan.FromHours(3);
private bool _isCacheInitialized = false;
private const string Url = "https://jsonplaceholder.typicode.com/photos";
public override async Task StartAsync(CancellationToken cancellationToken)
{
await cacheSignal.WaitAsync();
await base.StartAsync(cancellationToken);
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
logger.LogInformation("Updating cache.");
try
{
Photo[]? photos =
await httpClient.GetFromJsonAsync<Photo[]>(
Url, stoppingToken);
if (photos is { Length: > 0 })
{
cache.Set("Photos", photos);
logger.LogInformation(
"Cache updated with {Count:#,#} photos.", photos.Length);
}
else
{
logger.LogWarning(
"Unable to fetch photos to update cache.");
}
}
finally
{
if (!_isCacheInitialized)
{
cacheSignal.Release();
_isCacheInitialized = true;
}
}
try
{
logger.LogInformation(
"Will attempt to update the cache in {Hours} hours from now.",
_updateInterval.Hours);
await Task.Delay(_updateInterval, stoppingToken);
}
catch (OperationCanceledException)
{
logger.LogWarning("Cancellation acknowledged: shutting down.");
break;
}
}
}
}
In the preceding C# code:
ILogger
, HttpClient
, and IMemoryCache
._updateInterval
is defined for three hours.ExecuteAsync
method:
"https://jsonplaceholder.typicode.com/photos"
, and maps the response as an array of Photo
objects.IMemoryCache
under the "Photos"
key._cacheSignal.Release()
is called, releasing any consumers who were waiting for the signal.Consumers in the same process could ask the IMemoryCache
for the photos, but the CacheWorker
is responsible for updating the cache.
In some scenarios, a distributed cache is required—such is the case with multiple app servers. A distributed cache supports higher scale-out than the in-memory caching approach. Using a distributed cache offloads the cache memory to an external process, but does require extra network I/O and introduces a bit more latency (even if nominal).
The distributed caching abstractions are part of the Microsoft.Extensions.Caching.Memory
NuGet package, and there is even an AddDistributedMemoryCache
extension method.
Caution
The AddDistributedMemoryCache should only be used in development and/or testing scenarios, and is not a viable production implementation.
Consider any of the available implementations of the IDistributedCache
from the following packages:
Microsoft.Extensions.Caching.SqlServer
Microsoft.Extensions.Caching.StackExchangeRedis
NCache.Microsoft.Extensions.Caching.OpenSource
The distributed caching APIs are a bit more primitive than their in-memory caching API counterparts. The key-value pairs are a bit more basic. In-memory caching keys are based on an object
, whereas the distributed keys are a string
. With in-memory caching, the value can be any strongly-typed generic, whereas values in distributed caching are persisted as byte[]
. That's not to say that various implementations don't expose strongly-typed generic values but that would be an implementation detail.
To create values in the distributed cache, call one of the set APIs:
Using the AlphabetLetter
record from the in-memory cache example, you could serialize the object to JSON and then encode the string
as a byte[]
:
DistributedCacheEntryOptions options = new()
{
AbsoluteExpirationRelativeToNow =
TimeSpan.FromMilliseconds(MillisecondsAbsoluteExpiration)
};
AlphabetLetter alphabetLetter = new(letter);
string json = JsonSerializer.Serialize(alphabetLetter);
byte[] bytes = Encoding.UTF8.GetBytes(json);
await cache.SetAsync(letter.ToString(), bytes, options);
Much like in-memory caching, cache entries can have options to help fine-tune their existence in the cache—in this case, the DistributedCacheEntryOptions.
There are several convenience-based extension methods for creating values, that help to avoid encoding string
representations of objects into a byte[]
:
To read values from the distributed cache, call one of the get APIs:
AlphabetLetter? alphabetLetter = null;
byte[]? bytes = await cache.GetAsync(letter.ToString());
if (bytes is { Length: > 0 })
{
string json = Encoding.UTF8.GetString(bytes);
alphabetLetter = JsonSerializer.Deserialize<AlphabetLetter>(json);
}
Once a cache entry is read out of the cache, you can get the UTF8 encoded string
representation from the byte[]
There are several convenience-based extension methods for reading values, that help to avoid decoding byte[]
into string
representations of objects:
There is no way to update the values in the distributed cache with a single API call, instead, values can have their sliding expirations reset with one of the refresh APIs:
If the actual value needs to be updated, you'd have to delete the value and then re-add it.
To delete values in the distributed cache, call one of the remove APIs:
Tip
While there are synchronous versions of the aforementioned APIs, please consider the fact that implementations of distributed caches are reliant on network I/O. For this reason, it is preferred more often than not to use the asynchronous APIs.
.NET feedback
.NET is an open source project. Select a link to provide feedback:
Training
Module
Improve performance with a cache in a .NET Aspire project - Training
In this module, you'll learn about caches in a .NET Aspire cloud-native app and how to use them to optimize the performance of your microservices.