Edit

Querying based on runtime state (C#)

  • Article

Note

Make sure you add using System.Linq.Expressions; and using static System.Linq.Expressions.Expression; at the top of your .cs file.

Consider code that defines an IQueryable or an IQueryable<T> against a data source:

string[] companyNames = [
    "Consolidated Messenger", "Alpine Ski House", "Southridge Video",
    "City Power & Light", "Coho Winery", "Wide World Importers",
    "Graphic Design Institute", "Adventure Works", "Humongous Insurance",
    "Woodgrove Bank", "Margie's Travel", "Northwind Traders",
    "Blue Yonder Airlines", "Trey Research", "The Phone Company",
    "Wingtip Toys", "Lucerne Publishing", "Fourth Coffee"
];

// We're using an in-memory array as the data source, but the IQueryable could have come
// from anywhere -- an ORM backed by a database, a web request, or any other LINQ provider.
IQueryable<string> companyNamesSource = companyNames.AsQueryable();
var fixedQry = companyNames.OrderBy(x => x);

Every time you run this code, the same exact query will be executed. This is frequently not very useful, as you may want your code to execute different queries depending on conditions at run time. This article describes how you can execute a different query based on runtime state.

IQueryable / IQueryable<T> and expression trees

Fundamentally, an IQueryable has two components:

  • Expression—a language- and datasource-agnostic representation of the current query's components, in the form of an expression tree.
  • Provider—an instance of a LINQ provider, which knows how to materialize the current query into a value or set of values.

In the context of dynamic querying, the provider will usually remain the same; the expression tree of the query will differ from query to query.

Expression trees are immutable; if you want a different expression tree—and thus a different query—you'll need to translate the existing expression tree to a new one, and thus to a new IQueryable.

The following sections describe specific techniques for querying differently in response to runtime state:

  • Use runtime state from within the expression tree
  • Call additional LINQ methods
  • Vary the expression tree passed into the LINQ methods
  • Construct an Expression<TDelegate> expression tree using the factory methods at Expression
  • Add method call nodes to an IQueryable's expression tree
  • Construct strings, and use the Dynamic LINQ library

Use runtime state from within the expression tree

Assuming the LINQ provider supports it, the simplest way to query dynamically is to reference the runtime state directly in the query via a closed-over variable, such as length in the following code example:

var length = 1;
var qry = companyNamesSource
    .Select(x => x.Substring(0, length))
    .Distinct();

Console.WriteLine(string.Join(",", qry));
// prints: C, A, S, W, G, H, M, N, B, T, L, F

length = 2;
Console.WriteLine(string.Join(",", qry));
// prints: Co, Al, So, Ci, Wi, Gr, Ad, Hu, Wo, Ma, No, Bl, Tr, Th, Lu, Fo

The internal expression tree—and thus the query—haven't been modified; the query returns different values only because the value of length has been changed.

Call additional LINQ methods

Generally, the built-in LINQ methods at Queryable perform two steps:

You can replace the original query with the result of an IQueryable<T>-returning method, to get a new query. You can do this conditionally based on runtime state, as in the following example:

// bool sortByLength = /* ... */;

var qry = companyNamesSource;
if (sortByLength)
{
    qry = qry.OrderBy(x => x.Length);
}

Vary the expression tree passed into the LINQ methods

You can pass in different expressions to the LINQ methods, depending on runtime state:

// string? startsWith = /* ... */;
// string? endsWith = /* ... */;

Expression<Func<string, bool>> expr = (startsWith, endsWith) switch
{
    ("" or null, "" or null) => x => true,
    (_, "" or null) => x => x.StartsWith(startsWith),
    ("" or null, _) => x => x.EndsWith(endsWith),
    (_, _) => x => x.StartsWith(startsWith) || x.EndsWith(endsWith)
};

var qry = companyNamesSource.Where(expr);

You might also want to compose the various subexpressions using a third-party library such as LinqKit's PredicateBuilder:

// This is functionally equivalent to the previous example.

// using LinqKit;
// string? startsWith = /* ... */;
// string? endsWith = /* ... */;

Expression<Func<string, bool>>? expr = PredicateBuilder.New<string>(false);
var original = expr;
if (!string.IsNullOrEmpty(startsWith))
{
    expr = expr.Or(x => x.StartsWith(startsWith));
}
if (!string.IsNullOrEmpty(endsWith))
{
    expr = expr.Or(x => x.EndsWith(endsWith));
}
if (expr == original)
{
    expr = x => true;
}

var qry = companyNamesSource.Where(expr);

Construct expression trees and queries using factory methods

In all the examples up to this point, we've known the element type at compile time—string—and thus the type of the query—IQueryable<string>. You may need to add components to a query of any element type, or to add different components, depending on the element type. You can create expression trees from the ground up, using the factory methods at System.Linq.Expressions.Expression, and thus tailor the expression at run time to a specific element type.

Constructing an Expression<TDelegate>

When you construct an expression to pass into one of the LINQ methods, you're actually constructing an instance of Expression<TDelegate>, where TDelegate is some delegate type such as Func<string, bool>, Action, or a custom delegate type.

Expression<TDelegate> inherits from LambdaExpression, which represents a complete lambda expression like the following:

Expression<Func<string, bool>> expr = x => x.StartsWith("a");

A LambdaExpression has two components:

  • A parameter list—(string x)—represented by the Parameters property.
  • A body—x.StartsWith("a")—represented by the Body property.

The basic steps in constructing an Expression<TDelegate> are as follows:

  • Define ParameterExpression objects for each of the parameters (if any) in the lambda expression, using the Parameter factory method.

    ParameterExpression x = Expression.Parameter(typeof(string), "x");

  • Construct the body of your LambdaExpression, using the ParameterExpression(s) you've defined, and the factory methods at Expression. For instance, an expression representing x.StartsWith("a") could be constructed like this:

    Expression body = Call(
    x,
    typeof(string).GetMethod("StartsWith", new[] { typeof(string) })!,
    Constant("a")
);

  • Wrap the parameters and body in a compile-time-typed Expression<TDelegate>, using the appropriate Lambda factory method overload:

    Expression<Func<string, bool>> expr = Lambda<Func<string, bool>>(body, x);

The following sections describe a scenario in which you might want to construct an Expression<TDelegate> to pass into a LINQ method, and provide a complete example of how to do so using the factory methods.

Scenario

Let's say you have multiple entity types:

record Person(string LastName, string FirstName, DateTime DateOfBirth);
record Car(string Model, int Year);

For any of these entity types, you want to filter and return only those entities that have a given text inside one of their string fields. For Person, you'd want to search the FirstName and LastName properties:

string term = /* ... */;
var personsQry = new List<Person>()
    .AsQueryable()
    .Where(x => x.FirstName.Contains(term) || x.LastName.Contains(term));

But for Car, you'd want to search only the Model property:

string term = /* ... */;
var carsQry = new List<Car>()
    .AsQueryable()
    .Where(x => x.Model.Contains(term));

While you could write one custom function for IQueryable<Person> and another for IQueryable<Car>, the following function adds this filtering to any existing query, irrespective of the specific element type.

Example

// using static System.Linq.Expressions.Expression;

IQueryable<T> TextFilter<T>(IQueryable<T> source, string term)
{
    if (string.IsNullOrEmpty(term)) { return source; }

    // T is a compile-time placeholder for the element type of the query.
    Type elementType = typeof(T);

    // Get all the string properties on this specific type.
    PropertyInfo[] stringProperties =
        elementType.GetProperties()
            .Where(x => x.PropertyType == typeof(string))
            .ToArray();
    if (!stringProperties.Any()) { return source; }

    // Get the right overload of String.Contains
    MethodInfo containsMethod = typeof(string).GetMethod("Contains", new[] { typeof(string) })!;

    // Create a parameter for the expression tree:
    // the 'x' in 'x => x.PropertyName.Contains("term")'
    // The type of this parameter is the query's element type
    ParameterExpression prm = Parameter(elementType);

    // Map each property to an expression tree node
    IEnumerable<Expression> expressions = stringProperties
        .Select(prp =>
            // For each property, we have to construct an expression tree node like x.PropertyName.Contains("term")
            Call(                  // .Contains(...) 
                Property(          // .PropertyName
                    prm,           // x 
                    prp
                ),
                containsMethod,
                Constant(term)     // "term" 
            )
        );

    // Combine all the resultant expression nodes using ||
    Expression body = expressions
        .Aggregate(
            (prev, current) => Or(prev, current)
        );

    // Wrap the expression body in a compile-time-typed lambda expression
    Expression<Func<T, bool>> lambda = Lambda<Func<T, bool>>(body, prm);

    // Because the lambda is compile-time-typed (albeit with a generic parameter), we can use it with the Where method
    return source.Where(lambda);
}

Because the TextFilter function takes and returns an IQueryable<T> (and not just an IQueryable), you can add further compile-time-typed query elements after the text filter.

var qry = TextFilter(
        new List<Person>().AsQueryable(), 
        "abcd"
    )
    .Where(x => x.DateOfBirth < new DateTime(2001, 1, 1));

var qry1 = TextFilter(
        new List<Car>().AsQueryable(), 
        "abcd"
    )
    .Where(x => x.Year == 2010);

Add method call nodes to the IQueryable's expression tree

If you have an IQueryable instead of an IQueryable<T>, you can't directly call the generic LINQ methods. One alternative is to build the inner expression tree as above, and use reflection to invoke the appropriate LINQ method while passing in the expression tree.

You could also duplicate the LINQ method's functionality, by wrapping the entire tree in a MethodCallExpression that represents a call to the LINQ method:

IQueryable TextFilter_Untyped(IQueryable source, string term)
{
    if (string.IsNullOrEmpty(term)) { return source; }
    Type elementType = source.ElementType;

    // The logic for building the ParameterExpression and the LambdaExpression's body is the same as in the previous example,
    // but has been refactored into the constructBody function.
    (Expression? body, ParameterExpression? prm) = constructBody(elementType, term);
    if (body is null) {return source;}

    Expression filteredTree = Call(
        typeof(Queryable),
        "Where",
        new[] { elementType},
        source.Expression,
        Lambda(body, prm!)
    );

    return source.Provider.CreateQuery(filteredTree);
}

In this case, you don't have a compile-time T generic placeholder, so you'll use the Lambda overload that doesn't require compile-time type information, and which produces a LambdaExpression instead of an Expression<TDelegate>.

The Dynamic LINQ library

Constructing expression trees using factory methods is relatively complex; it is easier to compose strings. The Dynamic LINQ library exposes a set of extension methods on IQueryable corresponding to the standard LINQ methods at Queryable, and which accept strings in a special syntax instead of expression trees. The library generates the appropriate expression tree from the string, and can return the resultant translated IQueryable.

For instance, the previous example could be rewritten as follows:

// using System.Linq.Dynamic.Core

IQueryable TextFilter_Strings(IQueryable source, string term) {
    if (string.IsNullOrEmpty(term)) { return source; }

    var elementType = source.ElementType;

    // Get all the string property names on this specific type.
    var stringProperties = 
        elementType.GetProperties()
            .Where(x => x.PropertyType == typeof(string))
            .ToArray();
    if (!stringProperties.Any()) { return source; }

    // Build the string expression
    string filterExpr = string.Join(
        " || ",
        stringProperties.Select(prp => $"{prp.Name}.Contains(@0)")
    );

    return source.Where(filterExpr, term);
}

See also