C#: Writing extendable applications using on-the-fly compilation
Sometime back I had posted about writing applications that can load plugins using late binding. Users can drop assemblies in a specific folders which is scanned by an host application at startup. Using reflection the host can locate classes that implement a specific interface and instantiate those classes as plugins. The drawback is that the host processes plugin assemblies and you need to store/maintain the corresponding sources elsewhere.
This can be extended using on the fly compilation. In this technique the user drops source files in a specific folder and the host-application either at startup or on user request compiles the sources in-memory and loads the generated assembly. The sources can be in any of the .NET languages like C#, VB.NET or J#. The whole of this can be trivially accomplished using the Microsoft.CSharp and the System.CodeDom.Compiler namespaces from System.dll.
We can call these sources as scripts as they are available as sources. However, in the truest sense they are not scripts because they are not interpreted but are compiled and executed. They are more of plugins where the step of compilation is done by the host application.
There are several security issues to be considered before using this technique and so you might want to read the drawbacks section at the end of this post before trying this out.
The Common Contract
The first and one of the most important things to decide is the Document Object Model or the DOM that the host-application exposes. The DOM is used by the script to manipulate the host-application. Similarly it's also important to decide the interface that the script exposes so that the host application can locate the script class in the script-source and instantiate the class. The DOM and the script interface together form the SW contract that both parties use to communicate with each other.
It's best to define the DOM and the script interface in a common extension dll and make both the host application and all scripts refer to it. The DOM is also defined in terms of an interface which the host application implements. For our purpose let's consider the simple DOM/script-interface combination.
namespace ScriptableApplication.Extensibility
{
public delegate void ClickEvent(int x, int y); public interface IHost { string Title { get; set; } Color BackgroundColor { get; set; } event ClickEvent Click; } public interface IScript { string Name {get;} string Description { get;} void Initialize(IHost host); void Close(); }}
Here the host exposes a simple DOM using which the scripts can modify the host-applications title text and background color. It also exposes an event which is fired each time user clicks on the host-form. Scripts can subscribe to the event to get click notification along with the coordinates of the click.
Each script needs to have at least one class that implements the IScript interface. The host application compiles the script on the fly and using reflection looks for any class that implements this interface and instantiates that class. It then calls the Initialize method passing it the IHost pointer which the script can use to manipulate the host DOM.
On the Fly Compilation
This is what makes the whole technique work. .NET framework exposes all the compilers and associated framework in a very well designed namespace. This can be used to locate any compiler registered with a given file extension and use it to compile the sources.
private CompilerResults LoadScript(string filepath)
{
string language = CSharpCodeProvider.GetLanguageFromExtension(
Path.GetExtension(filepath));
CodeDomProvider codeDomProvider =
CSharpCodeProvider.CreateProvider(language);
CompilerParameters compilerParams = new CompilerParameters();
compilerParams.GenerateExecutable = false;
compilerParams.GenerateInMemory = true;
compilerParams.IncludeDebugInformation = false;
string extAssembly = Path.Combine(
Path.GetDirectoryName(Application.ExecutablePath),
"Extensibility.dll");
compilerParams.ReferencedAssemblies.Add(extAssembly);
compilerParams.ReferencedAssemblies.Add("System.dll");
compilerParams.ReferencedAssemblies.Add("System.Drawing.dll");
compilerParams.ReferencedAssemblies.Add("System.Windows.Forms.dll"); return codeDomProvider.CompileAssemblyFromFile(compilerParams,
filepath);
}
In the first part we use the .NET framework to locate the relevant compiler based on the extension of the script file. In case of unsupported language System.Configuration.ConfigurationErrorsException is thrown.
After we get the code-dom provider we create the compiler parameters. The GenerateInMemory flag is set to indicate the assembly is not generated on disk but in-memory.
Since the script needs to refer to the extensibility dll and some bare minimal .NET framework dlls, they are added to the ReferencedAssemblies collection. After that the compilation is done and the result of the compilation is returned as a CompilerResults object.
In case compilation fails the failures are available and are enumerated as follows
CompilerResults result = LoadScript(filePath);
if (result.Errors.HasErrors)
{
StringBuilder errors = new StringBuilder();
string filename = Path.GetFileName(filePath);
foreach (CompilerError err in result.Errors)
{
errors.Append(string.Format("\r\n{0}({1},{2}): {3}: {4}",
filename, err.Line, err.Column,
err.ErrorNumber, err.ErrorText));
}
string str = "Error loading script\r\n" + errors.ToString();
throw new ApplicationException(str);
}
This gives detailed failure messages along with rom/column numbers of the failure.
Loading the plugins
If there are no compilation errors then the compiled in-memory assembly is available in result.CompiledAssembly. The host uses reflection to search through all the types in the assembly that implements IScript interface and loads that type and calls IScript methods.
GetPlugins(result.CompiledAssembly);
private void GetPlugins(Assembly assembly)
{
foreach (Type type in assembly.GetTypes())
{
if (!type.IsClass || type.IsNotPublic) continue;
Type[] interfaces = type.GetInterfaces();
if (((IList<Type>)interfaces).Contains(typeof(IScript)))
{
IScript iScript = (IScript)Activator.CreateInstance(type);
iScript.Initialize(m_host);
// add the script details to a collection
ScriptDetails.Add(string.Format("{0} ({1})\r\n",
iScript.Name, iScript.Description));
}
}
}
With this we are done initializing the plugins
Implementing the scripts
Implementing the scripts is simple and can be done in any .NET language. The following example is a script in C#.
using System;
using ScriptableApplication.Extensibility;
using System.Drawing;
public class Script : IScript
{
int i = 0;
IHost host = null;
public string Name {get {return "CoolScript";}}
public string Description { get {return "Coolest script";}}
public void Initialize(IHost host)
{
this.host = host;
host.Click +=
delegate(int x, int y)
{
host.Title = string.Format("Clicked on {0}, {1}",
x, y);
Random autoRand = new Random(); host.BackgroundColor =
Color.FromArgb(autoRand.Next(0, 255),
autoRand.Next(0, 255),
autoRand.Next(0, 255));
}; }
public void Close()
{
}
}
The script stores the IHost reference and subscribes to the IHost.Click event. In the event handler it sets the host dialogs title to the position of the click and changes the background-color to some random generated color.
Similarly scripts can be written in VB.NET as follows
Imports Microsoft.VisualBasic
Imports System
Imports System.Drawing
Imports ScriptableApplication.Extensibility
PublicClass Class1
Implements IScript
Public Sub Initialize(ByVal host As IHost)
Implements IScript.Initialize
host.Title = "VB Script loaded"
End Sub
Public Sub Close() Implements IScript.Close
End Sub
Public ReadOnly Property Description() As String
Implements IScript.Description
Get
Description = "I'm a nice VB script"
End Get
End Property
Public ReadOnly Property Name() As String
Implements IScript.Name
Get
Name = "VB Script"
End Get
End Property
End Class
Sample project
You can download the complete sample from here. It contains the following
Extensibility project: This contains only one source file Interfaces.cs which contains definition for the IScript and IHost interfaces. This project generates the Extensibility.dll.
ScriptableApplication project: This contains ScriptHandler.cs which has the complete implementation of on the fly compilation and loading of the dlls. MainForm.cs which implements the host application.
Scripts: Contains two sample scripts script.cs and script.vb
Drawbacks
There are two huge drawbacks to this technique, both of which related to security. If any user with lesser privilege has access to the plugins folder or can direct the host-application to pick up a script from any folder then the user can execute any code he wants using the credentials of the host application. In my case I used this to create motion detection algorithm plugins. This was a personal use test code and I ensured that only I and other admins of the box had write permission to the plugins folder, and so it is kind-of safe for use. However, if you want to use this in a program that's distributed outside you need to plug this security-flaw.
I had initially assumed that I can apply assembly level security restrictions on the generated assembly to disallow file and network access using something like [assembly: FileIOPermission(SecurityAction.RequestRefuse, Unrestricted = true)] and I'll be done. I am not an expert on security related issues and so I asked around on what is the best option in securing this scenario. Shawn Farkas suggested that the best option would be to use sand-boxed AppDomains. However since the plugin will be in an AppDomain different from the AppDomain in which the host application exists, there'll be performance cost due to cross appdomain calls that needs marshalling.
The other issue is error conditions in the script. If there is an unhandled exception in the script it'll bring the whole application down.