Events
Take the Microsoft Learn Challenge
Nov 19, 11 PM - Jan 10, 11 PM
Ignite Edition - Build skills in Microsoft Azure and earn a digital badge by January 10!
Register nowThis browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
In this article, you'll learn how to use the Azure SDK for .NET pagination functionality to work efficiently and productively with large data sets. Pagination is the act of dividing large data sets into pages, making it easier for the consumer to iterate through smaller amounts of data. Starting with C# 8, you can create and consume streams asynchronously using Asynchronous (async) streams. Async streams are based on the IAsyncEnumerable<T> interface. The Azure SDK for .NET exposes an implementation of IAsyncEnumerable<T>
with its AsyncPageable<T>
class.
All of the samples in this article rely on the following NuGet packages:
For the latest directory of Azure SDK for .NET packages, see Azure SDK latest releases.
Clients instantiated from the Azure SDK for .NET can return the following pageable types.
Type | Description |
---|---|
Pageable<T> |
A collection of values retrieved in pages |
AsyncPageable<T> |
A collection of values asynchronously retrieved in pages |
Most of the samples in this article are asynchronous, using variations of the AsyncPageable<T>
type. Using asynchronous programming for I/O-bound operations is ideal. A perfect use case is using the async APIs from the Azure SDK for .NET as these operations represent HTTP/S network calls.
To iterate over an AsyncPageable<T>
using the await foreach
syntax, consider the following example:
async Task IterateSecretsWithAwaitForeachAsync()
{
AsyncPageable<SecretProperties> allSecrets = client.GetPropertiesOfSecretsAsync();
await foreach (SecretProperties secret in allSecrets)
{
Console.WriteLine($"IterateSecretsWithAwaitForeachAsync: {secret.Name}");
}
}
In the preceding C# code:
AsyncPageable<SecretProperties>
object.await foreach
loop, each SecretProperties
is asynchronously yielded.secret
is materialized, its Name
is written to the console.To iterate over an AsyncPageable<T>
when the await foreach
syntax isn't available, use a while
loop.
async Task IterateSecretsWithWhileLoopAsync()
{
AsyncPageable<SecretProperties> allSecrets = client.GetPropertiesOfSecretsAsync();
IAsyncEnumerator<SecretProperties> enumerator = allSecrets.GetAsyncEnumerator();
try
{
while (await enumerator.MoveNextAsync())
{
SecretProperties secret = enumerator.Current;
Console.WriteLine($"IterateSecretsWithWhileLoopAsync: {secret.Name}");
}
}
finally
{
await enumerator.DisposeAsync();
}
}
In the preceding C# code:
AsyncPageable<SecretProperties>
object.IAsyncEnumerator<SecretProperties>
.If you want control over receiving pages of values from the service, use the AsyncPageable<T>.AsPages
method:
async Task IterateSecretsAsPagesAsync()
{
AsyncPageable<SecretProperties> allSecrets = client.GetPropertiesOfSecretsAsync();
await foreach (Page<SecretProperties> page in allSecrets.AsPages())
{
foreach (SecretProperties secret in page.Values)
{
Console.WriteLine($"IterateSecretsAsPagesAsync: {secret.Name}");
}
// The continuation token that can be used in AsPages call to resume enumeration
Console.WriteLine(page.ContinuationToken);
}
}
In the preceding C# code:
AsyncPageable<SecretProperties>
object.IAsyncEnumerable<Page<SecretProperties>>
.await foreach
.IReadOnlyList<T>
that's iterated over with a synchronous foreach
.The System.Linq.Async
package provides a set of LINQ methods that operate on IAsyncEnumerable<T> type. Because AsyncPageable<T>
implements IAsyncEnumerable<T>
, you can use System.Linq.Async
to query and transform the data.
Use ToListAsync
to convert an AsyncPageable<T>
to a List<T>
. This method might make several service calls if the data isn't returned in a single page.
async Task ToListAsync()
{
AsyncPageable<SecretProperties> allSecrets =
client.GetPropertiesOfSecretsAsync();
List<SecretProperties> secretList = await allSecrets.ToListAsync();
secretList.ForEach(secret =>
Console.WriteLine($"ToListAsync: {secret.Name}"));
}
In the preceding C# code:
AsyncPageable<SecretProperties>
object.ToListAsync
method is awaited, which materializes a new List<SecretProperties>
instance.Take
can be used to get only the first N
elements of the AsyncPageable
. Using Take
will make the fewest service calls required to get N
items.
async Task TakeAsync(int count = 30)
{
AsyncPageable<SecretProperties> allSecrets =
client.GetPropertiesOfSecretsAsync();
await foreach (SecretProperties secret in allSecrets.Take(count))
{
Console.WriteLine($"TakeAsync: {secret.Name}");
}
}
System.Linq.Async
provides other methods that provide functionality equivalent to their synchronous Enumerable
counterparts. Examples of such methods include Select
, Where
, OrderBy
, and GroupBy
.
When using the System.Linq.Async
package, beware that LINQ operations are executed on the client. The following query would fetch all the items just to count them:
// ⚠️ DON'T DO THIS! 😲
int expensiveSecretCount =
await client.GetPropertiesOfSecretsAsync()
.CountAsync();
Warning
The same warning applies to operators like Where
. Always prefer server-side filtering, aggregation, or projections of data if available.
The System.Linq.Async
package is primarily used to provide observer pattern capabilities over IAsyncEnumerable<T>
sequences. Asynchronous streams are pull-based. As their items are iterated over, the next available item is pulled. This approach is in juxtaposition with the observer pattern, which is push-based. As items become available, they're pushed to subscribers who act as observers. The System.Linq.Async
package provides the ToObservable
extension method that lets you convert an IAsyncEnumerable<T>
to an IObservable<T>
.
Imagine an IObserver<SecretProperties>
implementation:
sealed file class SecretPropertyObserver : IObserver<SecretProperties>
{
public void OnCompleted() =>
Console.WriteLine("Done observing secrets");
public void OnError(Exception error) =>
Console.WriteLine($"Error observing secrets: {error}");
public void OnNext(SecretProperties secret) =>
Console.WriteLine($"Observable: {secret.Name}");
}
You could consume the ToObservable
extension method as follows:
IDisposable UseTheToObservableMethod()
{
AsyncPageable<SecretProperties> allSecrets =
client.GetPropertiesOfSecretsAsync();
IObservable<SecretProperties> observable = allSecrets.ToObservable();
return observable.Subscribe(
new SecretPropertyObserver());
}
In the preceding C# code:
AsyncPageable<SecretProperties>
object.ToObservable()
method is called on the AsyncPageable<SecretProperties>
instance, returning an IObservable<SecretProperties>
.observable
is subscribed to, passing in the observer implementation, returning the subscription to the caller.IDisposable
. When it's disposed, the subscription ends.Pageable<T>
is a synchronous version of AsyncPageable<T>
that can be used with a normal foreach
loop.
void IterateWithPageable()
{
Pageable<SecretProperties> allSecrets = client.GetPropertiesOfSecrets();
foreach (SecretProperties secret in allSecrets)
{
Console.WriteLine($"IterateWithPageable: {secret.Name}");
}
}
Important
While this synchronous API is available, use the asynchronous API alternatives for a better experience.
.NET feedback
.NET is an open source project. Select a link to provide feedback:
Events
Take the Microsoft Learn Challenge
Nov 19, 11 PM - Jan 10, 11 PM
Ignite Edition - Build skills in Microsoft Azure and earn a digital badge by January 10!
Register nowTraining
Module
Store and iterate through sequences of data using Arrays and the foreach statement in C# - Training
Learn to create array variables and iterate through elements of the array.