CLR Inside Out

.NET Application Extensibility

Jack Gudenkauf and Jesse Kaplan

Code download available at:  CLR Inside Out 2007_02.exe(197 KB)

Contents

Building an Extensible Host
Discovery
Activation, Isolation, Security, and Sandboxing
Lifetime Management
Add-In Development
Wrap-Up

The Microsoft .NET Framework has made it very easy to take components built by different developers and companies and integrate them into your own applications. But it's only easy if you know which components you're building against. If those components are not known at build time, which is typically the case for add-ins, then things become much more difficult. Developers often have questions when extending their applications. For example, should the add-ins be stored in a database or on disk? Should they reflect on the add-in for a known interface so as to acquire the type for activation? What is the best way to use AppDomain, AppDomainManager, and AppDomainSetup?

We will address these and other questions in two installments of CLR Inside Out by introducing the new System.AddIn namespace in the Base Class Library (BCL), available in the next release of Visual Studio® (code-named "Orcas"). You may download the libraries prior to the full release of Visual Studio "Orcas" via the Community Technology Preview. You can also find information in the common language runtime (CLR) Add-In team blog.

In this series of columns we will walk you through the development lifecycle of adding extensibility to an application and then versioning it over time. This first column will cover all the challenges developers face when they first add extensibility to their application and how our APIs make their lives easier. Next month, we will focus on the longer-term problems of keeping add-ins working even as the host changes: we will take the sample application built in this column, change it significantly for its next version, and walk you through the steps needed to keep the original add-ins working.

So let's start talking about extensibility challenges and the beauty of APIs. With the addition of the System.AddIn namespace and guidance on building a managed add-in model, the CLR provides a solution that solves problems common to the first version of an application and guidance for ensuring the future compatibility of your applications. See the "Extensibility Terminology" sidebar to better understand the terms we'll use.

Extensibility Terminology

Host An application that supports extensibility. A host may be a client or server (for example, Excel, Word, Paint.NET, SQL Server, and Exchange Server).

Extensibility A mechanism for manipulating host application objects or extending host functionality, sometimes referred to as automation. Generally made available via an object model published as part of a host's SDK.

Software Development Kit (SDK) A set of public interfaces (APIs) and associated documentation. Allows the host application to provide services to an add-in and vice versa. This SDK may be provided by different parties in different situations. For example, the host may provide the SDK to enable host-specific add-ins, an add-in may provide the SDK to let many different hosts use it (for example, Windows Media® Player), a third party may provide the SDK to let many different hosts, and add-ins communicate with each other.

Add-In (also called Add-On, Extension, Plug-In, Snap-In) A customization; an assembly loaded by a host. Also, something that automates a host application (such as a client). Something that provides additional functionality to a host (service). An add-in is custom code, usually written by a third party, that is dynamically loaded and activated, by a host application, based upon some context (such as host startup or document loading). The add-in extends the host application via the host's public API (for example, an object model or managed class library) which was made available to the add-in via an SDK.

Pipeline The machinery that enables the host and add-in to communicate over a version-resilient, secure protocol.

Building an Extensible Host

After a host has defined an object model that it wants to expose, there are three main phases in a host's interaction with its add-ins: discovery, activation, and lifetime management. Discovery involves finding the available add-ins and providing the host or user with enough information to select which add-ins to use. Activation involves loading and starting the selected add-ins in an appropriate isolated security sandbox. Lifetime management refers to keeping an add-in, and the objects it exchanges with its host, alive for as long as the application has references to it while giving the host the ability to shut the add-in down immediately when necessary. Unless specified otherwise, all of the code here is part of the Program.cs file in SampleExtensibleCalculator.csproj available for download from the MSDN®Magazine Web site.

Before we go into detail about using an add-in at run time, let's talk about the SDK and defining the object model that a host provides to its add-ins. Our add-in model supports a wide range of scenarios, but typically they fall into one of three categories.

In the first category, the add-in provides a service to the host application. For example, a mail server might host add-ins for virus scanning, spam filtering, or IP protection; a word processor might have need for a spell-check add-in; and a Web browser might host add-ins for handling certain file types. Our Calculator example illustrates this scenario. If you look at the add-in code found in SampleCalculatorAddIn.cs, you will see that the add-in is providing the service implementation of Add, Subtract, Multiply, and Divide to the host.

In the second category, the host offers its behavior to the add-in and lets the add-in define how it automates the host. In this scenario, the host is really providing a service to the add-in. Most Microsoft® Office add-ins fall under this category: upon initialization of the add-in, the Office application will pass its root object to the add-in, allowing it to automate the host.

Using this model, a host can let third parties extend it in ways the application never considered. We've seen Microsoft Excel® add-ins that automatically replace text symbols with real-time stock quotes, and Microsoft Word add-ins that add virtual links to phone numbers in documents that start up a Voice over IP (VoIP) program and call a phone number. Sometimes you see entire applications built as one large add-in to an existing application. There are several CRM (customer relationship management) applications whose entire client front-end is contained inside Microsoft Outlook®.

The final category represents add-ins that use the host mainly for screen real-estate rather than any host-specific functionality. Search tool bars are an example of this: the same toolbar may add itself to your browser, the windows taskbar, and your e-mail client while providing the same functionality regardless of its host.

In our sample application, our host provides an SDK to add-ins in order to use them as a service; in this case, it will use the add-in to implement simple calculator functions. In our model there are actually three, often very similar, definitions of the object model. We define these components in what we call a "pipeline": the host's view, the add-in's view, and the contracts that cross the isolation boundary. We'll discuss the pipeline in more detail in the future, but for now let's just look at the host's view.

The following code is part of the host view, rather than the core host code, so you can find it in CalculatorView.cs in hostCalculatorView.csproj:

public abstract class Calculator 
{
    public abstract double Add(double a, double b);    
    public abstract double Subtract(double a, double b);
    public abstract double Multiply(double a, double b);
    public abstract double Divide(double a, double b);
}

As you can see, the host's view is simply an abstract base class that defines the functionality it expects from its add-ins. The host will program against this view directly and never be aware there is an isolation boundary between it and its add-in or that the add-in may have a very different view of the host.

Discovery

The first step in using add-ins in your application is discovering them: the application needs a way to find the available add-ins and get enough information about them to decide which ones to load and activate. Depending on the application, the host will typically perform this step at application start-up, when a document is loaded, or upon a user request.

Some of the high-level discoverability requirements include the ability to find particular types of add-ins in a specified location, providing enough information to let the host or user decide which add-ins it wants to use without having to activate them first, and intelligently caching so that the list of available add-ins does not have to be recomputed, and make it possible for add-ins to update the cache at installation time. To achieve these ends, you can follow the steps we took. First, define where you want to look for add-ins. In this sample, we expect the AddIns and components to be installed in the current directory, but often applications will choose several different locations (for example, the user's application data directory).

String addInRoot = Environment.CurrentDirectory;

Next, update your add-in store so that it is aware of newly installed add-ins and pipeline components. Our AddInStore class is used in determining what can connect to what. This information makes up what we call the pipeline.

The store contains all the components that make up the pipeline between the host and its add-ins. The AddInStore may be updated at run time using the AddInStore.Update APIs and may also be updated at add-in installation time, using our command-line tool, in order to enhance the discovery startup performance. The following code checks to see if new AddIns have been installed. If no new add-ins or pipeline components have been installed since this was last called, this returns very quickly:

AddInStore.Update(addInRoot);

Once the cache has been updated, the host will want to find all the available add-ins that can be connected to the view (abstract base class) that the host built against. This code looks for Calculator AddIns in our root directory and stores the results:

Collection<AddInToken> tokens = 
    AddInStore.FindAddIns(typeof(Calculator), addInRoot);

The AddInStore.FindAddIns method takes in the type of add-in you are looking for and the path where they are stored and returns a collection of AddInTokens. It is important to note that no add-in assemblies have been loaded at this point and no add-in code has been executed. Each token describes a unique add-in and four properties (Name, Version Number, Publisher, and Description) provided by the add-in to allow the application/user to make decisions about which add-ins to load. In our calculator scenario, we built a ChooseCalculator method to display the tokens to the user, allowing them to select which they want to use:

AddInToken calcToken = ChooseCalculator(tokens);

Activation, Isolation, Security, and Sandboxing

Once the application has decided which add-ins it wants to use, it then needs to activate them. The host needs, from its perspective, an instance that implements the requested abstract base class (that is, the view). In reality, however, the host usually needs more than that: it really wants the add-in isolated from the host and other add-ins, and it wants to be able to sandbox that add-in with a particular permission set.

An isolated add-in assembly may not directly access code or resources in another assembly. The .NET Framework uses AppDomains as the in-process unit of isolation. For those who are unfamiliar with AppDomains, you can think of them as lightweight processes. Typically the host will run its code in one AppDomain, usually the default AppDomain that is created when the runtime starts up, and will want each add-in to have its own AppDomain. By isolating add-ins from the host and from each other, you gain several important advantages:

  • Each add-in is in its own folder, which has its own application base and loads its own assemblies without conflicting with the host or other add-ins. This also enables add-ins to have their own configuration file which may be used for functionality specific to add-ins.
  • The host can catch failures in one add-in and shut it down (and possibly restart) without impacting the host or other add-ins in the host process.
  • The add-in and its dependencies can be unloaded (more on this in the Lifetime Management section of this column).
  • Add-ins can be sandboxed to a particular security permission set (more on this just ahead).

It is important to note, however, that when developing the host and the add-in itself you are programming directly against the abstract base classes that make up the host and add-in views and therefore you don't have to worry about the constraints of the isolation boundary. In this version, we support AppDomain level isolation and may add process level isolation in future versions. For more information on using AppDomains as a security boundary for hosting untrusted add-ins. For more on add-ins and AppDomains, see AppDomain's and Addin's.

The host needs to consider the security context with which to load and activate the add-in. A host application may be running in full trust but may wish to run add-ins with less permission. The preexisting security and AppDomain creation APIs are very powerful and let you tweak many different settings as to how the AppDomain is sandboxed. In most cases the host will want to grant the add-in either FullTrust, intranet, or Internet security settings and rely on the framework to decide what that means. Our APIs make it very easy to choose one of those three settings but we also give you the ability to fine-tune the setup parameters if needed. Putting it all together in one line of code, you get the following, which activates the selected AddInToken in a new AppDomain and sandboxes the assembly in the internet zone:

Calculator calculator = 
    calcToken.Activate<Calculator>(AddInSecurityLevel.Internet);

This single line of code returns to the host an activated instance of the selected add-in. So what happens behind the scenes to make this happen?

The AddIn framework creates an AppDomain and loads the appropriate assemblies in the AppDomain. Next, we give the code running in that AppDomain the security permissions appropriate for the specified security level (in this case corresponding to the Internet permission set). Then we set the ApplicationBase to be the location of the actual add-in assembly, to allow it to find its dependencies and resources. We then set the config file for the AppDomain to be the one sitting next to the add-in DLL. Finally, we hook up required pipeline components that connect the host to the add-in.

There is another overload that lets you specify a custom permission set for the add-ins and yet another that lets you provide your own AppDomain. This will allow you to fine-tune the AppDomain and security settings for the add-in or even group add-ins that have the same lifetime, security context, and/or enable performance benefits of re-using an AppDomain.

Figure 1 provides an end-to-end hosting that uses what we've covered thus far.

Figure 1 Hosting Add-Ins from Discovery to Activation

static void Main()
{
    // In this sample we expect the AddIns and components to 
    // be installed in the current directory
    String addInRoot = Environment.CurrentDirectory;
            
    // Check to see if new AddIns have been installed
    AddInStore.Rebuild(addInRoot);
            
    // Look for Calculator AddIns in our root directory and 
    // store the results
    Collection<AddInToken> tokens = 
        AddInStore.FindAddIns(typeof(Calculator), addInRoot);
            
    // Ask the user which AddIn they would like to use
    AddInToken calcToken = ChooseCalculator(tokens);
            
    // Activate the selected AddInToken in a new AppDomain set sandboxed 
    // in the internet zone
    Calculator calculator = 
        calcToken.Activate<Calculator>(AddInSecurityLevel.Internet);
            
    // Run the read-eval-print loop
    RunCalculator(calculator);
}

Lifetime Management

An essential part of any add-in model is the method used for managing the lifetime of the objects within it. COM lifetime management was entirely based on reference counting, while CLR lifetime management is based on a garbage collector, and CLR remoting uses a sponsor/timeout approach. In our system, we need to go across AppDomains or process boundaries and in these scenarios, the CLR's garbage collector is no longer able to provide adequate support. Therefore, we need to provide a system of lifetime management that makes it completely transparent to the host and add-in developers, and makes it as easy as possible for the pipeline (object model) developers. We provide a solution for lifetime management that will be discussed in more detail in next month's column.

A host application may be a long-running service, have memory constraints, or need to be able to unload add-ins at run time for these and other reasons. The runtime itself only allows the unloading of assemblies when you unload the AppDomain they were loaded in: it doesn't have the traditional DllUnload. This is yet another way the AppDomain boundary adds value: it lets you reclaim system resources without having to cycle the process.

If the host wants to shut down an add-in immediately, it doesn't have to worry about keeping track of which add-in belongs in which AppDomain. Instead we provide a class called the AddInController that takes care of it for you. As long as you have a reference to the add-in you want to shut down, you can do it in one line of code:

// Retreive the AddInController for my add-in and then shut it down
AddInController.GetAddInController(calculator).Shutdown();

Aside from the host's ability to shut down its add-ins eagerly, our model enables the host and add-ins to act as if their lifetime management was completely controlled by a garbage collector: if they want something to be released, they just null out the references or dispose it. Behind the scenes we actually manage lifetime with a combination of a reference-counting system and the runtime's remoting services.

Add-In Development

From the developer's perspective, the mechanics of the add-in system are hidden. The host's object model interaction, and/or the means by which the add-in offers extensibility to the host, should be as transparent as possible. Writing an add-in using our model is very easy: we employ a custom attribute that defines the add-in assembly entry in its assembly and provides descriptive information about the add-in to the host. Once the attribute is applied, all the add-in has to do is implement the methods defined in its abstract base class.

Note that in the sample code for the add-in (found in SampleCalculatorAddIn.cs, as part of SampleCalculatorAddIn.csproj in the code download), the add-in developer does not see the Pipeline code. The mechanism for handling the remoting and lifetime management, for instance, is abstracted away (see Figure 2).

Figure 2 Simple Calculator Add-In

namespace SampleCalculatorAddIn
{
    [AddIn("Sample Calculator AddIn", Version="1.0.0.0")]
    public class SampleCalculatorAddIn : Calculator
    {
        public override double Add(double a, double b)
        {
            return a + b;
        }
        public override double Subtract(double a, double b)
        {
            return a-b;
        }
        public override double Multiply(double a, double b)
        {
            return a * b;
        }
        public override double Divide(double a, double b)
        {
            return a / b;
        }
    }
}

As you can see, the add-in model contributes almost no overhead to the add-in developer. All the developer needs to describe the add-in to the host is to use the custom attribute and then implement the functionality the host required of it, in this case the simple arithmetic operations of a calculator.

Wrap-Up

Our goal in designing this model was to enable dynamic composition of version resilient, isolatable components. Our primary concerns were to ensure the add-in developer experience would not be complicated by the mechanics involved in meeting this goal and to make the host's job as easy as possible.

Next month we will discuss, in greater detail, the pipeline and utilization of the model to enable compatibility as your host and add-in version independently. In the meantime, please pay a visit to our team blog for more samples and to give us your feedback.

Send your questions and comments to clrinout@microsoft.com.

Jack Gudenkauf is a Software Architect on the CLR team at Microsoft. He is currently focused on engaging with ISV/VAR/SI customers to ascertain .NET architectural adoption issues. Jack has held positions including Developer Manager, MIS Director, software engineering, and consulting services. Previously, he ran his own consulting company, The Logical Choice.

Jesse Kaplan is the Program Manager for Extensibility and Add-Ins for the CLR team at Microsoft. His past responsibilities include compatibility and managed/native interoperability.