Share via

The One True Object (Part 2)

  • Article

The core of the DLR's type system is based on passing messages to objects. This isn't exactly a new idea, but focuses on what has always been the intellectual core of object-oriented systems. This simple notion doesn't explicitly talk about types at all, but instead focuses on objects and messages. Every dynamic and static language has its own notion of what a type is - from C#'s static single-inheritance types to Python's dynamic multiple-inheritance types to JavaScript's prototypes. Trying to reconcile all of these different systems at the type level is ridiculously complicated - although maybe something to tackle for DLR v2. Despite their differences at the type level, all of these languages share tremendous commonalities if you view them at the object-based message-passing level.

In order to support a broad range of languages in this kind of a type system, we need to have a standard set of clearly defined messages that all objects can respond to. The set needs to be rich enough to capture all of the unique properties of the different languages that we're working with while at the same time be sufficiently common so that code written in different languages can work together. This is a balance that I'm sure we're going to need to adjust as we bring more languages to the DLR - but I think we have an excellent start and lots of success with our initial four languages. Here is our current set of operations:

  • [Get|Set|Delete]Member(name, case-sensitivity) - gets, sets or deletes a named member on an object
  • Call/CreateInstance(argument modifiers) - standard call or 'new' call to an object with arguments.  Modifiers include parameter names, support for expanding argument list or keyword dictionaries and a marker for an argument representing an implicit this.
  • SimpleOperation(OperationKind enumeration) - catch-all for simple common operations like add and subtract as well as indexing, etc.
  • Convert(Type) - converts an object to a given static type if possible

Given this set of messages, we also need to have some mechanisms in place to allow objects of different types to respond to these messages. While the DLR lets developers treat objects uniformly whether they come from a static or a dynamic language, under the hood we need to use different mechanisms to implement this message passing for the two different cases.

Standard CLR static types

When an object is of a Type as defined by a statically typed language, the object's behavior is completely described by this Type. We use the runtime Type of the object to determine the correct behavior. For example, with the GetMember message, this will mean looking for a field, property or method with the given name on the object's Type and returning the approriate member for the object if found and otherwise throwing an exception.

The DLR will use all of the existing well known attributes specified in the Common Language Subset (CLS) in order to map members of a type onto a DLR operation. For example, we'll recognize add operator methods to use when responding to an add message. In addition, we will make standard objects behave as expected, for example, any delegate object will respond properly to a Call message.

In addition to these existing well-specified operators, the DLR adds some additional custom attributes that effectively extend the existing set of CLS operations to include support for overriding standard DLR messages. A good example of this is the static attribute that will let a type override member lookup to respond to a GetMember message. This lets a static type support name-based lookup when it is being called from a dynamic language. In our current DLR hosting work, we've found several excellent places to use this attribute. One good example is the FindControl method on page objects. With this attibute we can make working with pages feel more natural to a dynamic language:

 page.FindControl("text1") --> page.text1

In fact, we use a slightly more complicated method to override the GetMember behavior of the ASP.NET page objects. We can't modify the actual types because we want to run on existing shipping versions of ASP.NET. This means that we need to extend these types externally. To do this we build on the new feature of extension methods that was added to C#3 and VB9 as part of the suite of features that support LINQ. Extension methods let developers define methods to be added to a type outside of the definition of that type itself. These extension methods can be imported like a standard namespace and will then behave as if they were originally defined on the types that they apply to.

The DLR uses a slightly extended version of extension methods. First, we add support for properties and operator methods. Second, we allow extension methods to be provided at other scopes than just the per-file scope used by C# and VB.NET. For example, we allow languages to choose to apply extension methods to a given type for all code written in that language. This is the way that we support the standard Python methods on core CLR types such as string so that Python programmers can call "s.title()" even though System.String has no method with that name. This same mechanism also allows us to support special Python members on objects such as "__class__" or "__getitem__" by modifying the view of the type from Python code - rather than modifying the type itself. This lets all of our languages that want to use an immutable string object share the same actual instances freely while using extension methods to customize the view of that type to be appropriate to each language.

Fully dynamic types

For types that are defined by a dynamic language, we provide a more dynamic way to respond these messages. These dynamic types will implement a special interface - IDynamicObject - that can provide customized handling for all dynamic operations. One of the key steps in implementing a dynamic language on the DLR is to implement this interface for the standard object type used by a given language. This means that the IronPython implementation implements this interface on Python-defined classes to support the correct dynamic behavior following Python's rules for multiple inheritance and method resolution order. Similarly, the DLR-based JavaScript implementation implements this interface on JavaScript functions/objects in order to implement the correct prototype-based inheritance for that language. Each language doesn't need to know anything about the details of the type system in the other languages, but just needs to know what messages to pass to those objects and counts on the other languages correctly implementing their side of the contract.

One very obvious thing missing from the current DLR is a standard default implementation of this interface. This would be useful to people who wanted to implement simple scripting languages on the DLR who weren't trying to precisely match the semantics of an existing language. This would also be useful for libraries that wanted to create a generic expando object to pass to consumers. I expect that we'll provide this in the future, but at the moment our attention remains focused on the cases where we need to capture the exact semantics of existing dynamic languages where the interface approach provides the needed flexibility.

Obvious things that we haven't gotten to yet

A few obvious questions at this point may be how the different languages can both use these standard messages and yet retain all of the details of their own unique semantics. The simple trick here is that while the messages are standard, each language has to decide which messages to pass for a given piece of source code. This gives sufficent flexibility - although I expect you'll want to hear more details about that. The second obvious question may be that the mechanisms as described above sound awfully slow with a lot of introspection and interface calls going on at runtime. This description tries to capture the semantics of message passing in the DLR - not the implementation. It's the DLR's job to make this run fast and we're quite confident that we do this well today and will keep doing better for many tomorrows, but there's a much bigger explanation owed here as well.

Note for those wanting to explore the source code

I'd still caution most people to wait a few months to really dive into these bits as our design is actively being refactored and documentation is still mostly non-existent. However, if you'd like to look at the code you can find the DLR bits in the IronPython 2.0alpha1 release under the Microsoft.Scripting and Microsoft.Scripting.Vestigial projects. In case it's not obvious, the one with "Vestigial" in its name isn't expected to be around for much longer. The set of standard messages described in this post are found in the Microsoft.Scripting.Actions namespace - where these "messages" are called "Actions". The interface IDynamicObject is currently misnamed. The current version of that interface will soon be removed and the oddly named interface IActionable will be renamed to IDynamicObject. For now, you should look at IActionable.

Comments

  • Anonymous
    May 04, 2007
    I'm enjoying this series so far. I don't have much to add to the discussion, but I like the direction you are going. I'm mostly intersted in IronRuby, so of course I wonder how this architecture will deal with Ruby's pecularities. Especially the concept of "metaclasses" (aka "eigenclass") and modules.

  • Anonymous
    May 04, 2007
    The comment has been removed

  • Anonymous
    May 04, 2007
    DLR is so cool. I'm expecting for the next release of IronPython.

  • Anonymous
    May 04, 2007
    Sun is considering adding a new bytecode "invokedynamic" to the JVM and hopes that it'll somehow speed up method dispatch in dynamic languages. What's your take on this? Could some VM support really give a big performance boost, considering that the actual lookup still has to happen in software (ie. the language implementation)? Maybe with some VM supported (polymorphic) inline cache or other methods from Strongtalk/Self (type feedback, ...)?

  • Anonymous
    May 04, 2007
    What is the overhead involved in the message passing?  How does it compare with a simple function call?  I'm dying to see the implementation!

  • Anonymous
    May 04, 2007
    Great series Jim! I very much enjoyed meeting you and Jon Lam at Mix. p.s. You're the 300th blog I'm subscribed to. I should send you a prize or something. ;)

  • Anonymous
    May 05, 2007
    Dynamic Language Runtime(DLR)。DLR和IronPython全部开源，如果你微软这样的动作吃惊,请看看Microsoft 的 OpenSource Licence,可以到codeplex下载。新的动态语言运行时（Dynamic Language Runtime，DLR）向CLR中加入了一小部分核心特性，使之得到显著改善。它向平台中加入了一系列明确为动态语言需求所设计的服务，包括同享的动态类型系统、标准托管模型（Standard Hosting Model），以及轻松生成快速动态代码的支持

  • Anonymous
    May 06, 2007
    Jim Hugunin's Thinking Dynamic has a series of blog entries on a new level of support for dynamic languages

  • Anonymous
    May 07, 2007
    It's time for a New and Notable Update when my flagged posts in FeedDemon gets past a screenfull

  • Anonymous
    May 07, 2007
    I'm very interested in the work that the DLR is kicking up, and where it might lead. http://alblue.blogspot.com/2007/05/jvm-or-clr.html The one question I have is given Microsoft's past record of platform compatibility, how are they going to make the DLR available to the world? I guess the Silverlight technology is going to be cross platform to start with, but is there going to be any kind of guarantee that it will continue? That, more than anything else, is what will keep people away from the technology (though for Microsoft shops, it's a godsend).

  • Anonymous
    May 10, 2007
    I've been thinking about providing a unified runtime support for dynamic languages on the JVM for quite some time - I work on two such projects, the Rhino JavaScript interpreter and on the FreeMarker template engine, and keep in touch with people working on JRuby and Jython as well. My opinion is that instead of providing interfaces on objects (and thus internalizing the knowledge required for cross-language interop into object implementations), it is a more flexible approach to define interfaces for metaobject protocols, have each language ship its implementation of it (I call these metaobject protocol implementations "object navigators" or just "navigators"), and allow languages to use other languages' metaobject protocol implementations to access and manipulate those languages' native objects, in addition to using its own MOP implementation. The article is at http://www.szegedi.org/articles/wrappersOrNavigators.html if you're interested.

  • Anonymous
    May 15, 2007
    Excellent work Jim.  Writing .NET 2.0 desktop apps in the one true language has been a very satisfying experience.  Clearly for WPF the Python/Xaml interface has been a bit of a road block so I'm delighted you good folks are putting this effort in. Could the Xaml parser from Silverlight be used on the desktop?  If not, do you think that this would be a WPF 3.5 thing or an IronPython 2.0 thing?

  • Anonymous
    September 03, 2007
    Scott's last post had a link to a new opensource Lisp project for the DLR, called IronLisp (in homeage

  • Anonymous
    December 11, 2007
    It's time for a New and Notable Update when my flagged posts in FeedDemon gets past a screenfull