Dynamic Method Bags

  • Article

By Bill Wagner

The new dynamic features in C# 4.0 are usually discussed as an interop feature. You can write code using dynamic objects to call libraries defined in IronPython, or IronRuby, or COM objects. Just because a feature was defined for one use doesn’t mean that’s all it’s good for. I work with dynamic typing to explore new ways to solve problems without ever leaving C#. In this article, I’ll show you a bit of C# trickery that combines generics, expression trees, and dynamic to create a method bag. A method bag is a type that lets developers add new methods at runtime, and then call those methods through dynamic dispatch in C#.

The end goal of this exercise is to write code like this:

// Create an object that contains some new methods:
var newType = new MethodBag();
newType.SetMethod("Write", () => Console.WriteLine("Hello World"));
newType.SetMethod("Display", (string parm) => Console.WriteLine(parm));
newType.SetMethod("IsValid", () => true);
newType.SetMethod("Square", (int num) => num * num);
newType.SetMethod("Sequence", () => from n in Enumerable.Range(1, 100)
                               where n % 5 == 2
                               select n * n);

Once you’ve created an object and created methods on that object, you can call those just as though they’d been defined at compile time:

// start acting dynamic:
dynamic dispatcher = newType;
dispatcher.Write();
var result = dispatcher.IsValid();
Console.WriteLine(result);
dispatcher.Display("This is a message");
var result2 = dispatcher.Square(5);
Console.WriteLine(result2);
var sequence = dispatcher.Sequence();
foreach (var num in sequence)
    Console.WriteLine(num);

There are several pieces to this puzzle. Let’s go into the pieces one at a time. Each of the techniques is something you’ve seen before: dynamic dispatch, generics, and expression trees. Putting them together makes it possible to add named methods to a type and runtime and then use those methods whenever a dynamic type is expected.

The first task is to understand how dynamic dispatch occurs in C#. I defined this functionality in a class called MethodBag. The MethodBag class is derived from System.Dynamic.DynamicObject, in order to simplify the implementation of the dynamic behavior. You need to implement the interface System.Dynamic.IDynamicMetaObjectProvider in order to create a C# type that contains dynamic behavior. That’s a simple interface definition; you need to create one method that returns a DynamicMetaObject. Implementing a DynamicMetaObject, however, is quite a bit of work. Whenever you can, you should leverage the work done in the System.Dynamic.DynamicObject class by deriving your class from it. DynamicObject implements IDynamicMetaObjectProvider for you. The DynamicMetaObject created by the DynamicObject uses a number of virtual methods declared in DynamicObject to provide the dynamic behavior. Overriding any or all of those methods creates the dynamic behavior your type needs. You can see one example of this in Ander’s talk from the 2008 PDC (https://channel9.msdn.com/pdc2008/TL16/) and on Phil Haack’s blog (https://haacked.com/archive/2009/08/26/method-missing-csharp-4.aspx). In both cases, they have overridden TrySetMember and TryGetMember to create property bag in C#. I needed to override TryInvokeMember in order to implement the MethodBag and provide dynamic dispatch to methods at runtime. Here’s part of the code. I’ll explain the MethodDescription type later in this article. For now, it’s enough to understand that MethodDescription contains information about the code that creates a given method.

using global::System;
using global::System.Collections.Generic;
using global::System.Dynamic;
using global::System.Linq.Expressions;
public class MethodBag : DynamicObject
{
    public override bool TryInvokeMember(InvokeMemberBinder binder,
        object[] args, out object result)
    {
        result = null;
        if (!methods.ContainsKey(binder.Name))
            return false;
        // Described later
        MethodDescription method = methods[binder.Name];
        if (method.NumberOfParameters != args.Length)
            return false;
        result = method.Invoke(args);
        return true;
    }
}

TryInvokeMember returns a Boolean flag that indicates whether or not this method did actually perform the dynamic method invocation. If it can’t perform whatever work is needed, it should return false. In this class, that’s what happens if the caller doesn’t define a method, or if the method has the wrong number of parameters.  Once those checks pass, I’ll use a helper class (the MethodDescription class) to invoke that code a caller defined for the named method.

You are handling dynamic invocation, so you need to validate everything yourself. TryInvokeMember passes the parameters for the method in the args array (of type System.Object). If the method has a return type other than void, that returned object gets stored in the result object.  

Validating parameters is much easier if you create a strongly typed API for client code to define member methods. That way, inside the MethodBag, you can determine the number and types of parameters. The number of combinations for parameter types is incredibly large: as you add methods you add more method parameters. Creating generic methods means that you can avoid making specific versions for each or all of those combinations.

You’ll create overloads of SetMethod() for each number of parameters you want to support, including variations for Action and Func with each number of parameters. Inside each of those methods, you’ll store some of the information about the about the expression. Later, you’ll use that information to more easily invoke that expression.

The implementation is in the MethodDescription class and a few derived classes. MethodDescription defines the functionality you’ll need for any method added to the method bag. You’ll need to access the number of parameters. You’ll need to get and set the embedded Expression that defines the behavior for the dynamic method. Finally, you’ll need a way to execute the method at runtime. Here’s the definition of the MethodDescription class:

private abstract class MethodDescription
{
    internal abstract int NumberOfParameters
    {
        get;
    }
    internal Expression target
    {
        get;
        set;
    }
    internal abstract object Invoke(object[] parms);
}

You create derived classes for methods with different numbers of parameters, and for methods that do or do not return a value. Inside each of those derived classes, you’ll create a specific version of Invoke() that passes the correct number of parameters to a delegate created from the original expression. Inside each override of Invoke(), you’ll make use of C#’s dynamic support to do all the type coercing. After all, you’re implementing a dynamic object. Whether you check the types of all the parameters, or you let the C# runtime dynamic binder do it, you’ll get the same result.

Let’s walk through the methods that implement dynamic dispatch for a lambda that represents a System.Action method. You’ll set a method like this:

newType.SetMethod("Write", () => Console.WriteLine("Hello World"));

Anytime someone calls ‘dynamicThing.Write()’, you want to print “Hello World” to the console. The lambda expression is passed to SetMethod() as an expression. Inside SetMethod, it creates a new ActionDescription object which stores the expression:

public void SetMethod(string name, Expression<Action> lambda)
{
    var desc = new ActionDescription { target = lambda };
    methods.Add(name, desc);
}

Finally, Invoke() must compile and invoke that target.

internal override object Invoke(object[] parms)
{
    var target2 = target as Expression<Action>;
    target2.Compile().Invoke();
    return null;
}

Invoke() casts the target expression as an Expression<Action>. Then, it compiles and executes the resulting action. Because a System.Action has a void return type, this version of Invoke() sets the return value to null.

I said earlier that Invoke() will leverage the dynamic subsystem to manage some of the complexity involved in invoking these expressions. That starts to happen as soon as you support any expressions that take parameters or return values. The version of Invoke() that corresponds to an Action<T> looks like this:

internal override object Invoke(object[] parms)
{
    dynamic target2 = target;
    target2.Compile().Invoke(parms[0]);
    return null;
}

Invoke() now makes use of the dynamic subsystem. First, it creates a local variable that is statically typed to be dynamic. That enables me to compile and execute the resulting delegate without needing to create a new overload for every possible type that could be used for T in Action<T>. You can’t create a generic version of Invoke(). You’d need to create overloads for every number of parameters you’d want to support. Furthermore, you’d need determine which of the generic methods to call from TryInvokeMember() based on the number of parameters in the method call. It would end up being much more code in order to get just single parameter methods. Leveraging dynamic support easily extends to any number of parameters, and processing the return values.

Download the source code from the LINQ farm.

Download the source code directly.

The MethodBag does not support many of the features of a hand written class. I only coded the versions that take one and two parameters to save space. That should be enough to understand the technique and extend it to higher numbers of parameters.

There are other limitations that are too much work to overcome. If you run into these limitations, consider it a strong indication that you should create a hand coded type to implement the behavior you need.  Most obviously, I have not implemented state in the method bag. It’s actually harder than you might think to implement state by combining a MethodBag with the PropertyBag implemented in the references I cited earlier. That’s because expression trees cannot contain objects that are statically typed as dynamic. There simply isn’t a good syntax to create methods that rely on internal state on the same dynamic object.

I did not implement overloads, methods with the same name that contain different parameter types. It would not be that hard to extend the MethodBag I’ve shown with internal storage that could store method names with different numbers of parameters in different internal collections. That would enable some overloading features. Creating overloads that contain the same number of parameters (but different types) would be quite a bit more difficult. At that point, you would need much more internal storage mechanisms. If you really need multiple overloads, you must view that as a strong indication that you should create your own class.

You can only add public methods. You can’t define methods with any other access modifiers. Nor can you define methods that are virtual or abstract. That’s really not that big of a limitation, because you can’t derive from a specific shape of MethodBag. Also, you can’t specify that a MethodBag implements an interface.

You’ll also notice that I didn’t implement any type validation on the parameters to any of the methods. That’s because it really won’t make any difference to users. Either way, calling a dynamic method with the wrong parameter types will throw an exception. Whether I add code to check each parameter type, or if I simply let the C# runtime binder do the same checks.

Even with these limitations, using dynamic to create methods at runtime does have its place. You can create an object, add methods, and use it anywhere you can use any dynamic object. You could create MethodBags that will run queries for dynamic websites. A MethodBag could create the methods needed based on a users’ preference and dynamically generate the data. You could create a MethodBag to pass to libraries created in dynamic languages. You could create a MethodBag that implements the APIs needed by that dynamic library. It’s not nearly as full featured as a class, but it is a way to pass several method definitions to other APIs.