SharePoint has always had a variety of server-side extensibility points. When a developer is creating a custom application or service there are many extensibility points that can be leveraged: event handlers, web parts, web pages, web services, workflow activities, InfoPath forms, and more. In previous versions of SharePoint, only a farm administrator had permissions to deploy new server-side functionality to the servers.

Beginning with the SharePoint 2010 release, we started allowing site administrators to deploy certain types of customizations and applications without requiring the involvement of a farm administrator. To protect all of the server-side resources from malicious code, inefficient code, and naive overconsumption of resources, we execute these types of solutions in the new Sandboxed Code Service. The sandbox provides a constrained, monitored, scalable runtime environment where code can be executed. Farm administrators get to choose what server(s) in the farm they want the Sandbox Code Service to run on. Site administrators get to choose what solution(s) they want to allow in their site collections.

Whenever code is executing within the sandbox, it is limited in the scope of what it can do. For example, the code will not have network access or direct SQL access. This is so that we can protect the rest of the servers in SharePoint farm as well as other non-SharePoint servers that may be on the same network. You can, of course, access and manipulate data stored within SharePoint from within the sandbox. In addition to that, we do allow code in the sandbox to access external data but it must do so using the Business Data Connectivity object model. This is so that farm administrators can still have control over what non-SharePoint enterprise data the sandboxed code can have access to.

In the 2010 release, many SharePoint extensibility points were upgraded to enable sandboxed execution: web parts, event handlers, workflow actions, and InfoPath forms. But there are still some extensibility points that a non-sandboxed solution can deploy to the server for which there is not yet a sandboxed equivalent: ASMX web services, ASPX page-level code-behind, ASP.Net request handlers, and more.

I could go on for some time talking about the sandbox (I was one of the primary architects) but that’s not my intention with this post. There are already many other articles out there that go into lots of detail describing the sandbox runtime environment and how to create solutions designed for sandboxed execution. This one is a good starting point.

With this post, what I wanted to do was to demonstrate how you can, with some constraints, achieve sandboxed execution of a web service.

Web Service

When someone says “web service” what comes to mind? For many people, it’s a SOAP-based web service and an ASMX-based implementation that come to mind.

But that’s not what I think of when I think of a web service. To me, a web service is simply a service that does something remotely over the web. It is an http-based endpoint where I can send a request, have it do something, and get an answer back. There are so many options: the service could be SOAP-based; it could be REST-based; it could be a service that doesn’t conform to any industry standards; the data that is returned from the service could be XML, JSON, HTML, plain text; it could be encoded as ANSI, UTF-8, UCS-2; or it could be binary data like an image.

So, while it is certainly true that SharePoint 2010 does not support sandboxed execution of ASMX-based web services, that doesn’t mean that you can’t create sandboxed web services.

Web Parts

The approach that I’m taking here for a sandboxed web service is to leverage a sandboxed web part.

A web part is typically used to generate a fragment of a web page: a summary of recent changes in a document library, a sales forecast chart, some stock quotes, etc. With a web browser, users can add web parts to pages and drag them around within a variety of page layouts.

When a user is adding or editing a web part, there is generally no distinction made between full-trust and sandboxed web parts. There is a single web part selection and editing experience that hides the differences from the user. There are a few scenarios where there the differences are visible but these are not common so must users never know about the difference.

But when it comes to the underlying ASPX markup, they are actually quite different.

Sandboxed Web Part

In this example, you can see the markup where a full-trust web part is directly inserted into the page markup without using a web part zone.

A sandboxed web part can also be inserted into page markup, but the markup looks quite a bit different. The control that you are inserting through the page markup is actually an instance of the full-trust web part SPUserCodeWebPart and not the sandboxed web part itself. The SPUserCodeWebPart is as a wrapper for the sandboxed web part. As with any other page control, you configure the wrapper web part using properties. These properties allow you to specify which sandboxed web part should actually be rendered.

When the ASPX page is rendered, the SPUserCodeWebPart will activate an instance of the sandboxed web part over in the sandbox service and then ask the sandboxed web part to render its HTML output. When the wrapper web part receives the output back from the sandboxed web part, it inserts that output into the standard page response stream without any modification.

So, from what I’ve described so far, you should be able to see that if we were to create an ASPX page with a sandboxed web part then we would be able to send web requests to this page and get the sandboxed web part to do something on our behalf. Isn’t that the essence of a web service?

Removing Extraneous Output

Now, the web part is a control in an ASPX page. The output of the web part will coexist with:

• The static markup that is in the ASPX page
• Output from other controls that are in the page
• Static markup that is in the master page
• Output from controls that are in the master page.

If I’m using this web page as a web service then how is my client supposed to know what part of the response is from the web part and what is from the various other things in the page?

You could have your web part output some unique tokens at the beginning and end of its output. The client could then search for these book-end tokens, extract the output in the middle, and then use it. But that is really cumbersome. You really want is to be able to call this sandboxed web service using the XMLHttpRequest object from some javascript and easily process the results.

So, what if we take our sandboxed web part and put it into an ASPX page that doesn’t have anything else in it. Instead of using a “normal” SharePoint ASPX page, let’s use a custom ASPX page that has no other controls, no web part zones, and does not use a master page. What would the server return back to the caller? Well, if we were careful with the whitespace in the ASPX page, we can setup the page so that every single character in the response would be output from the sandboxed web part itself.

Any whitespace that is between directive tags will not be included in the response. So these can each be on their own lines. However, any text or whitespace that exists after the last directive will be included. So if you do not want your web service response to start out with a newline then you’ll need to start the markup for your sandboxed web part on the same line and immediately after the final %>. You will also need to watch out that you do not add any extra text or whitespace after the web part definition because that will also be included in every response immediately following the output of your web part.

Simple Example

Let’s look at a simple web part that returns plain text:

Now, why am I overriding the Render method? Aren’t all web parts supposed to override the RenderContents method? Well, think about what these two methods do. In a “normal” web part, the base class handles the Render method and its job is to spit out the chrome of the web part (title, frame, etc.) that goes around the body of the web part. After the Render method has generated the opening tags of the chrome, it will then call the RenderContents method in the child class to render the body of the web part. Once RenderContents is done, execution will return back to Render which will then generate the chrome’s closing tags. By overriding Render, we can ensure that no web part chrome is rendered and the only output that is generated comes from our child class.

Next, we will insert this plain text web part into an “empty” ASPX page as described above. This results in a page with no chrome and a web part with no chrome. If you navigate to the web page using a browser, you will get a simple plain text response. If you were to “view source”, you would see that every character in the response came from the web part’s Render method.

Now, you should be able to use this new ASPX page as a web service from some other ASPX page:

Clicking on the “Call” button a few times and, you’ll see.

Response Format

As I mentioned above, the full-trust wrapper web part simply takes the output string from the sandboxed web part and returns it to the client. This means that your web part can return data as XML, JSON, CSV, or any other text format that it wants.

However, you do not have complete control over the response. The web front end that is running the ASPX page is who owns the actual response stream that goes back to the client – not the sandboxed web part. The page also owns the content-type and encoding format of the response. This means that your sandboxed web part cannot alter these parameters of the response. Your web part will have to live with a text-only response stream. You can set the encoding format in the @Page directive (the default in SharePoint is UTF-8) but you can’t set the content type. The scenario where you want to return binary files (ex: images) remains a full-trust scenario only:

Sample Solution

In this sample solution, I’ve included the following:

Sandboxed web part – This web part returns SharePoint list items in JSON format. You can use the source code below and modify it to return XML or any other format that you want. The query string of the web part’s host page is used to send parameters to the web part. There are two input parameters: ListName is the name of the SharePoint list and ViewName is, optionally, the name of a view from that SharePoint list. The list that you specify must be in the same site as the ASPX page that is hosting the web part. Please Note: To discourage the use of this sample in production, the web part is limited to returning output for only the first ten items in the result set. Please get the source below and modify it for your specific scenario.

Host ASPX page – This is an “empty” ASPX page that hosts the sandboxed web part and acts as the endpoint of the web service. This page will be created at <YourSiteCollectionUrl>/AppPages/SandboxedWebService.aspx.

Sample “client app” page – This is an ASPX page that acts as a sample “client app.” It prompts you for the list/view names and then navigates to the web service allowing the browser to display the response. If you were building this solution for production use you would probably want to leave this page out of your solution. This page will be created at <YourSiteCollectionUrl>/TestClient/TestSandboxedWebService.aspx. Sample results are shown below:

Steps To Create Your Own Solution

For a solution like this, you should not start with the Visual Studio “Visual Web Part” solution template because it will take you down the ASCX path. Instead of an ASCX you want a code-based web part – but there isn’t a template for that. Instead, start out by creating a solution using the “Empty SharePoint Project” template. Set the trust level for this solution to be a sandboxed solution. In the empty solution, you should use “Add / New Item” and choose “Web Part” using whichever language you want to use. This will add a code-based web part to the solution.

By default, a .webpart file will also be added to your solution for this new web part. When developing a “normal” web part, this is likely a file that you want. It is needed for your web part to appear in the web part selection UI when a user chooses to add a web part to a page. However, there is only one page where we want this web part to be used and that is in the host ASPX page. We do not want users adding this web part to any other pages in the site . To prevent this, delete the .webpart file from the solution. The .webpart file does not need to be included in the solution in order for the web part to be used in a page. When you delete the .web part file in the Visual Studio solution, the corresponding changes to the feature/element manifests will be done for you automatically.

Next, you need to use “Add / New Item” and choose Module so that you can add the ASPX page that will host the sandboxed web part. A module lets you add any type of file to the SharePoint site. It comes with a default file named Sample.txt which you can rename to SandboxedWebRequest.aspx or whatever you desire. If you examine the Elements.xml file for this module, you will notice that, by default, the files for this module are automatically placed into a top-level folder of the site that has the same name as the module. This is advisable to keep the files from your solution separated from files in another solution. If common file names are used in the same directory by two different solutions then that will be a problem. One of the two solutions would be broken. Keeping things in solution-specific directories is advised.

Lastly, you’ll need to alter the markup in the ASPX page so that you have the minimalist markup described above. Note that in the markup for the SPUserCodeWebPart, the property AssemblyFullName contains the public key token of the assembly. If you sign your solution with a different key than I used (which I strongly suggest you do) and you cut-n-paste markup from my sample, then you will need to alter the public key token value to match whatever key you are using.

One other point, if you ever change the namespace or the class name of your web part, you will need to edit one of the files that drives the Visual Studio SharePoint build process. Unfortunately, there is no built-in UI for editing this file nor will you find the file in the Solution Explorer pane. You will need to either go to the file system or to the source browser (if you’ve got source control enabled). Look for the file SharePointProjectItem.spdata in the same directory as your web part source file and fix the SafeControl entry for your web part to have the right namespace and class name. Fortunately, this file leverages the build-time macro “$SharePoint.Project.AssemblyFullName$” to dynamically insert the strong name of the assembly. It will automatically get the public key token from your assembly so you do not have to futz with that part of this file if you change your key file.

Downloading The Sample

There are two downloads available from the MSDN Code Gallery for this sample:

SharePoint Solution – This is a working .WSP that you can upload to the Solution Gallery for your site collection and activate it. Once you do this, the sandboxed web part, the web service ASPX page, and the test client ASPX page will be added to the root site of your site collection. Again, to discourage the use of this sample in production, the web part is limited to returning output for only the first ten items in the result set. Please get the source below and modify it for your specific scenario.

Visual Studio Project – This is a .ZIP file containing the VS 2010 SharePoint solution that was used to generate the WSP above.

As with any of my other samples, you are free to use the solution in your site or take the source code and leverage it in some other solution that you are working on. Just keep in mind that this is a sample and, as such, has no warranty, no guarantee that it will work in all environments, and no promise that a future version of SharePoint won't break it.

Before we dig into the solution to the problem, I should first spend some time describing the problem.

When you create a lookup field, you need to choose a field from the “other” list to be used as the display value. When you are editing an item from “this” list, you will be shown a drop-down list box that allows you to choose an item from the “other” list. When each of the items from the other list are added to the drop-down list, some text value from the other item needs to be displayed here that uniquely identifies the item. Some fields work better than other fields.

For example, let’s say that you are adding a lookup field that is going to be used to select someone from a list of contacts. Using the Zip Code as the field that is displayed in the drop-down list isn’t likely to be very helpful. It would be extremely common for you to have multiple contacts that are in the same zip code. You would then see the same zip code listed more than once in the drop-down list and you would have no idea which entry represents which contact. Choose a contact would be a pretty frustrating experience. Using a field like Name would be a lot better. Granted, the Name field could also contain duplicates. But it would still be a whole lot better than a field like Zip Code which is likely to contain many duplicates.

The problem we have is when the “other” list is a document library. Which field should you choose for the lookup field to display? You could use the Title field but it doesn’t always get populated. Upload some text files into a document library and they will all wind up with an empty value for the Title field. Even if a Title value is specified for each document, this field allows duplicates which, as described above, is not good. So, is there anything about the documents in the document library that is normally going to be unique? Well, the file name. Duh! However, try creating a lookup field to a document library and you’ll notice that there is no file name field to choose. Doh!

The file name is exposed through the object model as a field named FileLeafRef. So why isn't this field available to use in lookup fields? I’m still trying to find someone that can explain that to me. If I figure it out I’ll update this post.

Ok, so now we can (finally) get back to the Knowledge Base site template that I was looking into and how it solved this problem.

One of the features that the Knowledge Base site template has is that articles written in the wiki library can each have one or more related articles. This was accomplished using a multi-value lookup field. Now, multi-valued lookup fields don’t exactly have the greatest UI experience. But they do allow the KB article author to accomplish the scenario without the KB site template developer having to invest in custom (read: expensive) list item selection UI.

But if we're using a multi-value lookup field to choose a document from a document library, what was the field that was used as the lookup field? Well, what the KB site template did was to add a text field to the document library called “Name” in conjunction with an event handler that would ensure that the value of this field always contained the actual name of the file. As wiki articles were created or modified, the event handler would always ensure that the Name field contained the right value.

Now, the way that the KB site template did this was specific to this site template and isn’t really reusable in other site templates. The concept is reusable but the specific implementation in that site template isn’t. So I created a sandboxed solution that contains a reusable implementation of this concept.

Using this solution is pretty easy:

• Upload the solution - Upload the solution to the Solution Gallery in your site collection and activate it.
• Activate the site feature - In each of the sites where you want to use this File Name field, activate the site feature named “BillGr's File Name Field”.
• Add the site column - In each document library where you want to use this field, add the site column named File Name to either the document library or the content types that are used in the document library.

Solution Implementation

In this solution, you’ll find three things:

• A site column that defines the File Name field.
• An assembly that contains a sandboxed event handler.
• A feature (which can be activated/deactivated) with two element manifests:
  • One that defines the File Name field.
  • One that wires-up the event handler to the ItemAdding and ItemUpdating events.

“File Name” Site Column Definition

One thing to note about the site column is that it has been defined with a very unique internal name (the Name attribute). This is to deal with field name collisions. What if a user has already defined a File Name column? What if they already have other data in that column? What if they already have business logic that is driven by the value of that field. My event handler can’t simply look for the first field named “File Name” that it finds and blindly overwrite its value with the current file name. This is one of the reasons why a field definition has both an internal name and a display name. As an aside, the other main reason for internal names is so that compiled code doesn’t need to be changed when the name of a field is localized into another language. But that’s a subject for a different post.

In this case, we’ve given our field an internal name that uses the solution name as a prefix. It is highly unlikely that another solution will define another field with this same internal name. Later, our event handler will use the same internal name when accessing the field value from the list item.

Event Handler Wire-Up

When a feature definition includes an event handler, usually the event handler has a ListTemplateId or ListUrl attribute. These attributes are normally used to narrow down the scope of the event handler to a specific list type. Since I haven’t included any of these attributes, my event handler will be globally registered. It will be called whenever any item in a list or document library is added or changed – anywhere in the site.

Event Handler Implementation

The event handler simply ensures that the value of the File Name field is correct given the current name of the document. The code relatively simple. Whenever we’ve been notified that an item has changed, we first have to confirm that the item is in a document library. If it is a “regular” list item, then it won't have a file so there isn’t any file name to stay in sync with. Now, please, no arguments here about Attachments <grin />. SharePoint treats the primary file stream of a document library item very differently from an attachment to a list item.

In addition to confirming that the item is in a document library, the event handler also needs to confirm that the item has a “File Name” field. If it doesn’t have the field then, again, there isn’t anything that we need to do.

Once we’ve ruled out these other scenarios, all that remains is to extract the file name from the URL and then ensure that the value of the “File Name” field contains this same file name.

Downloading The Sample

There are two downloads available from the MSDN Code Gallery for this sample:

• SharePoint Solution - This is a working .WSP that you can upload to the Solution Gallery for your site collection and activate it. Once you do this, the “BillGr's File Name Field” feature will be available to activate in all sites of your site collection. After activating the feature, you can add the site column named “File Name” to a document library to start getting updated file names. You'll find this site column in the “Custom Columns” group.
• Visual Studio Project - This is the VS 2010 SharePoint solution that was used to generate the solution above.

As with any of my other samples, you are free to use the solution in your site or take the source code and leverage it in some other solution that you are working on. Just keep in mind that this is a sample and, as such, has no warranty, no guarantee that it will work in all environments, and no promise that a future version of SharePoint won't break it.

My name is Bill Griffin. I’m a developer at Microsoft working on SharePoint. I’ve worked on a variety of things since I started at Microsoft back in 1993. I was one of the original members of the team that started work in 1998 on the project codenamed Tahoe - which eventually went on to ship as SharePoint Portal Server 2001. Over the years, I’ve held a variety of “senior-dev-guy” type roles on the SharePoint team. It’s been a fun ride and I’m still excited about the direction the SharePoint family of products are going and what the platform can do.

The features that are now available with the 2010 release are simply staggering. But one of the things that I’ve noticed over the years is that some of the ways that product team members know how to stretch, bend, and contort SharePoint and make it do really cool stuff sometimes doesn't makes it out of the hallways of the product team.

Those are the kinds of things that I’d like to post about here.

I’m not trying to produce comprehensive documentation or sample code regarding every feature in the entire SharePoint product family. I’ll leave that to others who are better at that than I am. I just wanted a place to share information about SharePoint and to be able to post samples that show novel ways of leveraging SharePoint as I run across them here in the product team.