• Article

November 2009

Volume 24 Number 11

SharePoint’s Sandbox - Developing, Deploying, and Monitoring Sandboxed Solutions in SharePoint 2010

By Paul Stubbs | November 2009

SharePoint is a rich development platform with a large community actively developing solutions. The challenge, however, has always been the balance between creating solutions and deploying them in a way that you can trust will not damage or impair the SharePoint farm. Farm administrators are responsible for maintaining the health and integrity of the SharePoint farm, and often this means putting complicated, time-consuming processes in place to test and validate solutions deployed to the farm. This need is counter to the rapid application model used to create SharePoint solutions, and it complicates the deployment of third-party solutions. A new feature in SharePoint 2010, called sandboxed solutions, addresses many of these concerns, enabling farm administrators to feel comfortable that the SharePoint farm is safe, giving site collection administrators the authority to manage the applications in their site collection, and providing developers with the flexibility to create solutions they know will be deployed in a safe and rapid manner.

In this article, I’ll describe how sandboxed solutions in SharePoint 2010 provide a framework for safe and rapid deployment of solutions. You’ll learn how farm administrators can monitor solutions and how site collection administrators install and manage solutions and features. You’ll also see how to develop a sandboxed Web Part. Let’s start with an understanding of how solutions are deployed to the sandbox.

Deploying Sandboxed Solutions

The structure of a sandboxed solution is very similar to a farm solution, which generally runs with full-trust permissions. The main differences lie in how a sandboxed solution is deployed and in the process that hosts the solution (which determines whether a solution is a sandboxed solution or a farm solution). This means that the same sandboxed solution can run with full trust when it’s deployed at the farm level and with partial trust when it’s deployed at the site collection level.

A SharePoint solution is a deployable package that can contain features and assemblies. The solution package is a .cab-based file with a .wsp extension. Visual Studio 2010 SharePoint project templates create .wsp package files. A solution might contain a number of SharePoint features, and these features provide functionality such as Web Parts, list definitions, modules, and event receivers.

Site collection administrators now have the authority to upload, activate, delete, and manage sandboxed solutions using the new Solution Gallery, which is a repository of sandboxed solutions. The gallery also enables a site collection administrator to monitor a solution’s usage against a resource quota. (You will see how in detail later in the article.) The Solution Gallery is simply a standard SharePoint list that stores the .wsp file. It is located under the _catalogs folder at _catalogs/solutions. The gallery is added to your site collection when you upgrade from a SharePoint 2007 site collection. You open the Solution Gallery, shown in Figure 1, by first clicking Site Settings on the Site Actions menu. Then, on the Site Settings page, click Solutions under the Galleries section.


Figure 1 SharePoint Solution Gallery

Deploying a sandboxed solution to the site collection is as simple as uploading the .wsp file to the gallery and activating it. Clicking the Upload Solution button gives you the option to upload a single .wsp file or multiple files. When the upload document form opens, browse for the location of the .wsp file or files you want to deploy.

Next, you are prompted to activate the solution. When you activate a solution, site-collection-scoped features are automatically activated as well, but you must go to each site to activate site-scoped features from the Manage Site Features page. The Solution Gallery lists a solution’s name and modified date as well as its activated status and the resources consumed by the solution. You can delete a solution from the gallery only after you have deactivated it. If you want to copy a solution from one site collection to another, save the .wsp file to disk and upload it to another Solution Gallery.

You can also use new solution and feature upgrade infrastructure features in SharePoint 2010 to upgrade sandboxed solutions. To upgrade a solution, you need to create a .wsp file that has a new file name but the same solution ID. The easiest way to do this is to open your project in Visual Studio and make the changes, after which you can change the package name by using the package designer. Double-click the Package node in your solution to open the package designer, and then change the name of the package. You can now upload the sandboxed solution package as you would a new package. SharePoint detects that a package with the same solution ID is installed and prompts you to upgrade. After you upgrade, you’ll see that the new version is activated and the old version deactivated, as shown in Figure 2. If you browse to the Web Part installed by the solution, you can see the new version running.


Figure 2 Upgraded Sandboxed Solution

Building a Sandboxed Web Part

A sandboxed solution looks and behaves like a farm solution, but a sandboxed solution’s assembly must be marked to allow partially trusted callers. The Sandboxed Solution property is configured in the properties window of the project. Remember that a sandboxed solution can be deployed as a full-trust solution and gain the expanded capabilities that come with running in full trust because nothing in the .wsp solution file specifies it as a sandboxed solution or a farm solution. This is determined by how the solution is deployed.

One of the most common types of sandboxed solutions are Web Parts. In this section, I’ll describe how to create a sandboxed solution with a Web Part and point out some the issues you should be aware of. First, there are two types of Web Parts in Visual Studio—visual Web Parts and standard Web Parts. A visual Web Part contains an .ascx control that you can use to visually design the look of the Web Part. Unfortunately, you cannot use visual Web Parts in the sandbox because sandboxed solutions can’t deploy files to the Web front end, which is where the .ascx files need to be deployed. This means that you must create a standard Web Part.

Start by selecting Empty Project in the New Project dialog box under the SharePoint node for C# or Visual Basic. When you create a SharePoint solution, the project wizard asks whether you are creating a sandboxed solution, which is the default, or a full-trust solution. (See Figure 3.) This option applies to SharePoint solutions that can be deployed to the Solutions Gallery. Click Finish after verifying the path to your site collection.


Figure 3 New SharePoint Project Wizard Dialog Box

To add a Web Part to the solution, choose Add New Item from the project main menu, and then select the Web Part project template item from the list. At this point, you have a fully functional sandboxed Web Part that you can deploy.

During the development of a sandboxed solution, Visual Studio automatically deploys and activates the solution in the Solution Gallery when you press F5. If you browse to the Solution Gallery, you can see the solution you just created and deployed. In Figure 4, the solution has the default name SharePointProject1.


Figure 4 SharePoint Project Deployed to Solution Gallery by Visual Studio

As you can with other types of Visual Studio projects, you can set a breakpoint in the code for a sandboxed solution. However, Visual Studio won’t recognize a breakpoint at this point because the Web Part has not been added to a page. You’ll see an empty red circle in the left margin for the line of code that has the breakpoint, as shown in Figure 5, indicating that the assembly that contains the Web Part has not been loaded in the attached process. When you add the Web Part to a page, Visual Studio detects that the assembly has been loaded and will stop on the breakpoint. Even though you haven’t added any real code to the project yet, you can see that everything is in place and working correctly.


Figure 5 Debugging SharePoint Solutions in Visual Studio

Knowing a little about how SharePoint sandboxing is implemented can help you understand how to debug sandboxed solutions. You might be familiar with debugging full-trust solutions, which run under the IIS worker process used by SharePoint called w3wp.exe. Sandboxed solutions do not run under this process. They run under the sandboxed worker process called SPUCWorkerProcess.exe, which you can see in Figure 6. Visual Studio automatically attaches the debugger to this process when you press F5. If you are debugging a solution that’s already deployed, you need to attach to this process manually.


Figure 6 Attach to SPUCWorkerProcess to Debug Sandboxed Solutions

Now let’s look at a couple of examples of how code runs in a sandboxed solution. The first is an example of code that works in the sandbox. Add the following code to the CreateChildControls method in your Web Part, just after the base.CreateChildControls method. You can see that you can access the SPLists collection to return the number of SPList objects in the site. Lists and libraries in sandboxed solutions generally work as they do in full-trust solutions. You are just restricted to the current site collection.

protected override void CreateChildControls()
    {
      base.CreateChildControls();

      Label ListCount = new Label();
      ListCount.Text = 
        String.Format("There are {0} Lists",
        SPContext.Current.Web.Lists.Count);
      Controls.Add(ListCount);
    }

Another example, which is blocked in the sandbox, is working with SPSecurity. Cut and paste the following code into your Web Part and deploy it to the sandbox:

protected override void CreateChildControls()
    {
      base.CreateChildControls();

      SPSecurity.RunWithElevatedPrivileges(
        delegate
        {
          Label ListCount = new Label();
          ListCount.Text =
            String.Format("There are {0} Lists",
            SPContext.Current.Web.Lists.Count);
          Controls.Add(ListCount);
        });
    }

You won’t receive an exception until the code executes in the sandbox process. The exception thrown is “Could not load type 'CodeToRunElevated' from assembly 'Microsoft.SharePoint, Version=14.900.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c'.” This exception is not a security exception but a missing type exception, and it occurs because the SPSecurity class is not part of the sandboxed SharePoint API. You might wonder how there can be a missing type runtime exception if Visual Studio compiled the solution without any exceptions. Visual Studio compiles against the full version of the SharePoint API, but at runtime, SharePoint swaps out the full API for the sandboxed API, and this API is what your code runs against. Visual Studio can help you write valid code by filtering IntelliSense for you when you create a sandboxed solution. You can see in Figure 7 that the SPSecurity class does not appear in the list. However, if you cut and paste code from a full-trust solution, that code might not work in the sandbox, and Visual Studio has no way to validate this.


Figure 7 IntelliSense Filtering Helps You Create Valid Sandboxed Solutions

Working in the Sandbox

You have seen how to deploy and build sandboxed solutions. Now let’s explore what can you do in the sandbox.

Sandboxed solutions have access to a large subset of functionality in the Microsoft.SharePoint namespace. Essentially all the classes below SPSite are available. This means that SPSite, SPWeb, SPList, and SPListItem are all available. In a sandboxed solution, you can create Web Parts, list definitions and instances, content types and fields, modules, declarative workflows, and event receivers. The complete sandboxed API is documented in the SharePoint SDK.

 In general, the sandbox process prevents you from accessing data outside the site collection where the solution has been deployed. This means, for example, that you can’t access the Internet to make Web service calls, you can’t access a hard drive to read or write files, and you can’t access code that is not marked to allow partially trusted callers. You also can’t deploy files to disk or add assemblies to the GAC in a sandboxed solution, and security-related functionality, such as running RunWithElevatedPriviledges and other SPSecurity methods, is not allowed.

But what you can do is read and write to lists and libraries within the same site collection. The sandbox provides enough functionality to build most of the applications that are required at a site level. At times, however, you might want to enable your sandboxed solutions to reach outside the sandbox to perform a trusted operation, such as calling a Web service or accessing a database. The best way to reach out of the sandbox is by using the Business Connectivity Services (BCS) to create an external content type. You can then read and write to the data source from the sandboxed solution. Another, more advanced, way to reach outside the sandbox is to create a class that runs in a full-trust process outside the sandboxed worker process to proxy calls. This proxy class is deployed as a farm solution and is callable from the sandboxed solution. Although it is outside the scope of this article to describe how to create a full-trust proxy, you should be aware of the functionality.

Monitoring Solutions

Farm administrators have the responsibility to keep the SharePoint farm healthy and secure, and they now have the tools to monitor and set quotas on sandboxed solutions. SharePoint provides farm administrators with a UI in Central Administration to assign resource quotas to each site collection. You can access the quotas from the Central Administration page under the Application Management section. Then click Configure Quotas and Locks in the Site Collections section. You can also navigate directly to the page at /_admin/sitequota.aspx in your central administration site.

The Quotas and Locks page, shown in Figure 8, allows you to select the site collection to work with. In the Site Quota Information section, you can set the User Solutions Resources Quota properties. You can enable the maximum daily limit and set the amount of points to allow. The default is 300 points. Points are calculated based on a number of factors, which I will explain in a moment. You can also enable an e-mail alert for when you reach a certain number of points, which is set to 100 by default. The page also tells you the current usage for the day and the average usage over the last 14 days. These two items provide the same values that a site collection administrator can monitor at the top of the Solutions Gallery.


Figure 8 Farm Administrators Control the Resource Quotas

Farm administrators can also adjust the solution resource quotas by using Windows PowerShell. For example, PowerShell makes it very easy to iterate through every site collection and reset the values back to their defaults. Using the Get-SPSite command and piping it to a foreach statement changes all the values with a single line of code. To run this code, open a SharePoint 2010 Management Console window from the Start menu and type the following commands:

Get-SPSite | foreach-object {$_.Quota.UserCodeMaximumLevel = 300}
Get-SPSite | foreach-object {$_.Quota.UserCodeWarningLevel = 100}

If you want to see only what the current quota values are, you can run the following PowerShell command:

Get-SPSite | foreach-object {$_.Quota}

This command outputs the quota properties for every site collection in the farm. PowerShell formats the output of the quota properties for the site collection as a simple list, such as this:

QuotaID                : 0
StorageMaximumLevel         : 0
InvitedUserMaximumLevel     : 0
StorageWarningLevel         : 0
UserCodeWarningLevel        : 100
UserCodeMaximumLevel        : 300
UpgradedPersistedProperties :

UserCodeWarningLevel and UserCodeMaximumLevel are the two quota properties that control sandboxed solutions. You will see the terms “user code” and “user solutions” to refer to sandboxed code and solutions in some of the administration pages and in the SharePoint object model. These properties allow you to adjust the number of points assigned to each site collection. You can also adjust the algorithm that is used to calculate what a point is worth. Points are calculated based on various metrics, and these metrics are designed to monitor the resource usage of the server to accurately reflect the true health of the server. SharePoint contains 14 metrics that contribute to the quota points.

  1. AbnormalProcessTerminationCount
  2. CPUExecutionTime
  3. CriticalExceptionCount
  4. InvocationCount
  5. PercentProcessorTime
  6. ProcessCPUCycles
  7. ProcessHandleCount
  8. ProcessIOBytes
  9. ProcessThreadCount
  10. ProcessVirtualBytes
  11. SharePointDatabaseQueryCount
  12. SharePointDatabaseQueryTime
  13. UnhandledExceptionCount
  14. UnresponsiveprocessCount

Each metric, called a ResourceMeasure, contains a ResourcesPerPoint property. The ResourcesPerPoint value is divided by the resources consumed to calculate the points used for that category. For example, AbnormalProcessTerminationCount has a ResourcesPerPoint value of 1. Every time a sandboxed solution terminates abnormally, 1 point is added. If you want to increase the penalty for a sandboxed solution that terminates, you can set ResourcesPerPoint to another value, such as 2. You can use 0 if you are not concerned about this metric. Other ResourceMeasures might not have a one-to-one correspondence with an event. The CPUExecutionTime metric prevents a sandboxed solution from consuming too many CPU cycles. The default ResourcesPerPoint value is 3,600 for CPUExecutionTime.

Another value is AbsoluteLimit, which is incremented each time a solution gets a point. If a solution exceeds the AbsoluteLimit value, it is terminated even if the daily usage limit has not been reached. Exceeding the value of AbsoluteLimit affects only the single solution, unlike the daily usage limit. These two levels of quotas, daily limit and absolute limit, work together to protect the health of the farm. I strongly recommend that you not modify ResourceMeasures, but as a developer or an administrator, you should understand exactly how these mechanisms work.

Although SharePoint does not provide an administration page for adjusting the quota metrics, you can see and change the values using the SharePoint object model or Windows PowerShell. The following PowerShell script outputs all 14 of the ResourceMeasure objects in a list that contains all the properties for each ResourceMeasure object. Figure 9 shows the sample output from the two values AbnormalProcessTerminationCount and CPUExecutionTime.

[System.Reflection.Assembly]::Load("Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c")
$spusercodeservice = [Microsoft.SharePoint.Administration.SPUserCodeService]::Local
$spusercodeservice.ResourceMeasures

Figure 9 Sample Output for ResourceMeasure Objects

MinimumThreshold       : 0
ResourcesPerPoint       : 1
AbsoluteLimit               : 1
Name                    : AbnormalProcessTerminationCount
TypeName                : Microsoft.SharePoint.Administration.SPResourceMeasure
DisplayName                 : AbnormalProcessTerminationCount
Id                  : 878646dd-2460-4e88-83cd-67e938fb069c
Status                  : Online
Parent                  : SPUserCodeService Name=SPUserCodeV4
Version                 : 2308
Properties               : {}
Farm                    : SPFarm Name=SharePoint_Config
UpgradedPersistedProperties     : {}

MinimumThreshold        : 0.1
ResourcesPerPoint       : 3600
AbsoluteLimit               : 60
Name                    : CPUExecutionTime
TypeName                : Microsoft.SharePoint.Administration.SPResourceMeasure
DisplayName              : CPUExecutionTime
Id                  : 418cbb0f-d191-4633-9177-7a4568d11d8d
Status                  : Online
Parent                  : SPUserCodeService Name=SPUserCodeV4
Version                 : 2265
Properties              : {}
Farm                    : SPFarm Name=SharePoint_Config
UpgradedPersistedProperties     : {}

You can also adjust the values programmatically by using the SharePoint object model. You can get a reference to the sandbox service from the static property called Local on the SPUserCodeService object and iterate through the ResourceMeasures collection. Here’s an example:

SPUserCodeService userCodeService = 
            SPUserCodeService.Local;

      SPResourceMeasureCollection rmc =
        userCodeService.ResourceMeasures;

      foreach (SPResourceMeasure resourceMeasure in rmc)
      {
        Console.WriteLine(resourceMeasure.Name);
      }

Validating Solutions in the Sandbox

The SharePoint sandbox framework provides an additional way for farm administrators to monitor and validate solutions that run in the sandbox. Farm administrators can deploy solution validators that run when a solution is uploaded to a Solution Gallery. Administrators can create validators that allow only code signed with a particular certificate to run, for example, or they can create a validator to allow only Web Parts. Another good use for validators is to log and catalog solutions in the farm as they are activated. You can see how this simple but powerful tool helps farm administrators get a handle on the solutions running in the farm.

Each solution validator is called when a solution is activated. If the validator is updated, solutions are validated again the next time they are executed. You create a validator inside your SharePoint farm feature project by adding a class that derives from SPSolutionValidator. Your validator class must have a System.Runtime.InteropServices.Guid attribute assigned to it. You can use the Create GUID tool from the Visual Studio Tools menu to create a GUID value. This value is used as the ProviderID property.

Next you add a constructor that takes a reference to UserCodeService. This constructor calls the base class constructor, passing the UserCodeService and a string name for the validator. In this constructor, you must assign a “signature" property to the validator. The default validator that ships with Windows uses a value of 1. If you are building the example, you can use 1234. The signature property is used by SharePoint to determine whether the validator has changed. In a production validator, you would want this value to be a hash that contains the version information of the validator.

Next you override the ValidateSolution and ValidateAssembly methods. The ValidateSolution method is called once for each solution, and ValidateAssembly is called once for each assembly in each solution. The ValidateSolution method is passed an SPSolutionValidationProperties object, which contains a number of properties used to validate the solution. The most important property is the Valid property. This property is set to False by default, so you must set it to True or the solution will fail validation. You can also set the ValidationErrorMessage and the ValidationErrorUrl to send back information about why a solution fails validation, and you can use the Files collection to get a reference to all the files in the solution package. The ValidateAssembly instance is also passed a reference to the SPSolutionValidatorProperties in addition to a SPSolutionFile, which is a reference to the assembly in the solution package. Figure 10 shows an example of a solution validator.

Figure 10 A Solution Validator

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.InteropServices;
using Microsoft.SharePoint.Administration;
using Microsoft.SharePoint.UserCode;

namespace SandboxSolutionValidator
{
  [Guid("DAC77CB7-7511-4E7E-8427-B6A57C5F49F7")]
    class SandboxSolutionValidator:SPSolutionValidator
    {
    [Persisted]
    List<string> PutSomeStringsHere;

    private const string validatorName = "Sandboxed solution Validator";

    public SandboxSolutionValidator(
      SPUserCodeService userCodeService) :
      base(validatorName, userCodeService)
    {
      this.Signature = 1234;
    }

    public override void ValidateSolution(
      SPSolutionValidationProperties properties)
    {
      //System.Diagnostics.Debugger.Launch();
      base.ValidateSolution(properties);
       
      //Determine whether the solution is valid
      properties.Valid = true;

      //Iterate over each file in the solution
      foreach (SPSolutionFile file in 
                properties.Files) { }

      //Set the error message and Url if it’s not valid
      properties.ValidationErrorMessage =
        "The solution is not valid";
      properties.ValidationErrorUrl =
        "https://moss.Contoso.com/MyErrorPage.apx";
    }

    public override void ValidateAssembly(
      SPSolutionValidationProperties properties, 
      SPSolutionFile assembly)
    {
      //System.Diagnostics.Debugger.Launch();
      base.ValidateAssembly(properties, assembly);

      properties.Valid = true;
    }
    }
}

You must add the solution validator to the SPUserCodeService SolutionValidators collection. You can do this by using Windows PowerShell, but the recommended way is to deploy the validator by using a farm-level feature. In the feature, you use a feature receiver to add the validator to the collection. In the FeatureActivated event (see Figure 11), you get a reference to the farm’s SPUserCodeService. You use the SPUserCodeService to create an instance of your validator and then add the instance to the SolutionValidators collection of the UserCodeService. These operations can be a little difficult to debug, but you can add the Debugger.Launch commands to your code. This starts an instance of Visual Studio and attaches your code to the correct process. Be sure you remove these commands when you have finished developing the validator.

Figure 11 Deploying a Validator

public override void FeatureActivated(
        SPFeatureReceiverProperties properties)
    {
      //System.Diagnostics.Debugger.Launch();

      SPUserCodeService userCodeService =
            SPUserCodeService.Local;

      SPSolutionValidator validator = new SandboxSolutionValidator(userCodeService);

      userCodeService.SolutionValidators.Add(
            validator);
    }

You can use PowerShell to list the validators that are installed in the farm, which is a quick way for administrators to verify which validators are installed. The technical preview version of SharePoint 2010 ships with a default solution validator. This validator does nothing other than set the valid property to True. Enter the following PowerShell script to list the installed validators. You can see the output in Figure 12.

[System.Reflection.Assembly]::Load(
"Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c");
$spusercodeservice = [Microsoft.SharePoint.Administration.SPUserCodeService]::Local;
$spusercodeservice.solutionvalidators;


Figure 12 Installed Solution Validators

You have seen a small example of how to create, deploy, and validate sandboxed code in your farm. This example should enable you to create rich and interesting validation code that provides your farm with great flexibility and security.

Sandboxed Is the Default

As I mentioned earlier, the default SharePoint project type in Visual Studio is a sandboxed solution. This should be the default in your mind as well—you should be thinking about creating your SharePoint solutions to run within the sandboxed environment. You have seen that sandboxed applications are very capable, can do most of the tasks that a business user needs, and provide the ability to break out in a safe way when you need to. The real benefit is the speed at which you can deploy your solutions into the enterprise. You no longer need to follow a grueling process of testing and validation for every solution you create, although good software practices should always be followed. Site collection administrators will be able to control their own collections and run the right solutions for their site. Farm administrators will be free from the tasks of controlling and managing all the requests for solution installs and be able to focus on the overall health of the farm with a set of rich monitoring tools.

In the end, I predict that you will see an explosion of SharePoint solutions. You’ll see an increased number of enterprise solutions written and deployed and a huge increase in the SharePoint community because solutions will be more readily available and easy to find, install, and run.

Paul Stubbs is a Microsoft technical evangelist for SharePoint and Office who focuses on the information worker development community for SharePoint and Office, Silverlight, and Web 2.0 social networking. He has authored three books about solution development with Microsoft Office, SharePoint, and Silverlight and several articles for MSDN Magazine. He has also spoken at Microsoft Tech-Ed and TechReady conferences. In addition, Paul has worked as a senior program manager with the Visual Studio Tools for Office team in Redmond, Washington. Read his blog at blogs.msdn.com/pstubbs.