The ODataUriParser is continuing its march towards completion. In fact in ODataLib version 5.5 (RTM drops on May 29th) it is almost complete.

In the last post I talked about how you could parse $filter and $orderby. In 5.5 you can also parse $select, $expand and the path part of an OData url too. Which means all that remains it putting all these bits together into a method that will parse the whole Url with a single step.

That said with 5.5 you can do this yourself, the algorithm is essentially:

1. Parse the Uri path into an ODataPath (see below)
2. Use the Type and IEdmEntitySet of the ODataPath to:
1. Parse $filter clause – Use ODataPathParser.ParseFilter(string, IEdmType, IEdmEntitySet) – (see this post)
2. Parse $orderby clause – Use ODataPathParser.ParseOrderBy(string, IEdmType, IEdmEntitySet) – (see this post)
3. Parse $select and $expand clauses – (see below)
4. Parse $inlinecount clause – Use ODataPathParser.ParseInlineCount(string)
5. Parse $top and $skip clauses – Trivial since these are just whole numbers

If you do this you can build a structure that holds the whole URI converted into various ASTs. In 5.6 we will add a new Parse(..) method that will do this for you and return an appropriate data structure to hold the ODataPath and various query clauses.

In the meantime lets take a look what we’ve added in 5.5:

Parsing the Path

This API is pretty simple, you need an ODataUriParser (which requires an IEdmModel and a base Uri) then you call ParsePath(..), like this:

// Get a model, clearly you could do this in many ways, in this example we borrow one of the sample OData models
IEdmModel model = EdmxReader.Parse(XmlTextReader.Create(“http://services.odata.org/V3/OData/OData.svc/$metadata”));

// Construct a parser for that model
ODataUriParser parser = new ODataUriParser(model, new Uri(“http://server/service”));

// Parse a relative path
ODataPath path = parser.ParsePath(new Uri("http://services.odata.org/V3/OData/OData.svc/Products/ODataDemo.FeaturedProduct(1)/Advertisement"));

If you do this you get an ODataPath that hold an enumeration of segments.
In this case the segments will be (from first to last):

There are of course other segments OData paths you will encounter other segments like: PropertySegment, BatchSegment, MetadataSegment, ValueSegment, CountSegment, NavigationPropertyLinkSegment, OperationSegment and so forth.

Doesn’t WebAPI have an ODataPath too?

Those of you using WebAPI will be aware that it has it’s own code for parsing the path part of an OData URI. This is basically just bad timing, if we’d finished the ODataLib URI parser sooner Web API would have used it to parse paths as well. Indeed going forward Web API is likely to refactor their ODataPath code so that it uses the new ODataLib path parsing capabilities.

Parsing $select and $expand

The $select and $expand clauses are related, for example $select effects what is expanded. So we have a single method for parsing both simultaneously, this allows us to create an AST that captures the semantics of both query options together.

For example if someone issued a query like this:

GET http://server/service/Products?$select=ID,Name,Category&$expand=Category

You would convert this into an AST by making the following calls:

// Construct a parser for that model
ODataUriParser parser = new ODataUriParser(model, new Uri(“http://server/service”));

// Parse the $select and $expand clauses
SelectExpandClause selectAndExpand = parser.ParseSelectAndExpand("ID,Name,Category", "Category", type, set);

The SelectExpandClause is a data structure that looks like this:

Summary:

The new version of the ODataUriParser that ships in ODataLib version 5.5 is finished. 5.5 adds support for parsing Paths, $select and $expand, which means the ODataUriParser now handles all of OData. All that remains is to add a more convenient way of parsing a full Url in one step, and we will ship something that does that in version 5.6. In the meantime you can still parse an OData Uri completely if you use the algorithm outlined above.

If you have any questions, concerns or suggestion, as always we are keen to hear from you.

-Alex

In this post I will cover what is new since the Alpha, if you need a more complete view of what we is currently supported I recommend you read about the August Release and the Alpha release as well.

Simplified OData support

• Registers an OData wild card route. Which will have a prefix if you specified a prefix when you called EnableOData(..). For example this:
• HttpConfiguration.EnableOData(model, “api”) will position your OData service under the ~/api url prefix.
• Registers the various ODataMediaTypeFormatters.
• Note today this will stomp on application/json for your whole service. By RTM we aim to make this much more selective, so you get the OData version of application/json only if your request addresses an OData resource.
• Stashes the IEdmModel on the configuration
• Stashes the DefaultODataPathHandler on the configuration.
• Registers the ODataControllerSelector and ODataActionSelectors and configures them to use default OData routing conventions. These selectors only do OData routing when the request is recognized as an OData request, otherwise they delegate to the previously registered selectors.

First class OData routing

In the alpha release to get everything (OData Actions, OData casts, OData navigations etc) working together I updated the sample to do custom routing. This routing was based on a component that understood OData Paths and dispatched to actions based on custom routing conventions. At the time we knew this was a merely a stop gap.

That code has now been refactored, improved and moved into the core code base.

The code consists of:

• A class called ODataPath that represents the path part of an OData Url. An ODataPath is made up of a list of ODataPathSegments, these semantically represent the OData operations you compose to address a resource. For example something like this: ~/Customers(10)/Orders, semantically has 3 instructions that are encoded as ODataPathSegments in an ODataPath:
• Start in the Customers EntitySet
• Then find a single customer with the key 10
• Then Navigate to the related Orders.
• An interface called IODataPathHandler that has two methods:
• ODataPath Parse(string odataPath) –> to take a Url and parse it into an ODataPath
• string Link(ODataPath) –> to take an ODataPath and generate a Url.
• A default implementation of IODataPathHandler called DefaultODataPathHandler, that implements these methods using the OData V3’s built-in conventions.
• A way to register and retrieve a IODataPathHandler on your HttpConfiguration. Two things worth calling out here:
• Normally you don’t have to worry about this, because calling HttpConfiguration.EnableOData(…) registers the DefaultODataPathHandler for you.
• This design means that you don’t like the way OData puts keys in parenthesis (i.e.~/Customers(1)) and would prefer to put keys in their own segments (i.e. ~/Customer/1) then all you need to do is override the DefaultODataPathHandler so that it recognizes this syntax. I’ve tried this personally and doing targeted overloads like this is pretty easy.
• All OData link generation conventions (for example for EditLinks or links to invoke actions) now build an ODataPath to represent the link and then ask the registered IODataPathHandler to convert that to a url using the IODataPathHandler.Link(..) method.
• An ODataControllerSelector and an ODataActionSelector that route OData requests based on a configurable set of IODataRoutingConventions. Out the box EnableOData(..) will register these routing conventions:
• EntitySetRoutingConvention –> for routing request to manipulate and query an EntitySet, for example:
• GET ~/Customers
• GET ~/Customers/Namespace.VIP
• POST ~/Customers
• EntityRoutingConvention –> for routing request to retrieve or manipulate an Entity, for example:
• GET ~/Customer(1)
• GET ~/Customers(1)/Namespace.VIP
• PUT ~/Customer(1)
• PATCH ~/Customers(1)
• DELETE ~/Customers(1)
• NavigationRoutingConvention –> for routing requests to retrieve related items, for example:
• GET ~/Customers(1)/Orders
• GET ~/Customers(1)/Namespace.VIP/Benefits
• MetadataRoutingConvention –> for routing request to retrieve $metadata or the service document
• GET ~/$metadata
• GET ~
• LinksRoutingConvention –> for routing requests to manipulate relationship, for example:
• DELETE ~/Customers(1)/$links/Orders(1)
• POST ~/Customers(1)/$links/Orders
• PUT ~/Customers(1)/$links/RelationshipManager
• ActionRoutingConvention –> for routing requests to invoke an OData Action
• POST ~/Customers(1)/ConvertToVIP
• UnmappedRequestRoutingConvention –> for routing requests that match no other convention to a fall back method, for example:
• GET ~/Customers(1)/RelationshipManager/Customers
• POST ~/Customers(1)/Orders
• A new ODataPathRouteConstraint class that implements IHttpRouteConstraint an makes sure the ‘wildcard route’ that captures every request against your OData service only matches if the Url is in fact an OData url.
• An EntitySetController<TEntity,TKey> class that provides a convenient starting point for creating controllers for your OData EntitySets. This class provides stub methods that follow the default IODataRoutingConventions will route requests too. Your job is simply to handle the request.

For a complete example of how to use all this new goodness check out the sample OData service which you can find at http://aspnet.codeplex.com in the source tree under:  /Samples/Net4/CS/WebApi/ODataService.

Query validation

One of the most exciting new features is the ability to specify validation rules to be applied to a query. Essentially this allows you to constrain what query options you allow against your [Queryable]. Under the hood this is all implemented via the virtual ODataQueryOptions.Validate(ODataValidationSettings) method, which you can use if you need to.

That said the 90-99% scenario is simply to specify additional parameters to the [Queryable] attribute that the control what is allowed in a query, and then before the query is processed, Web API converts those settings into an ODataValidationSettings object and calls ODataQueryOptions.Validate(..). If anything not supported is found in the AST a validation error results, and your backend Queryable never gets called.

Here are some examples:

[Queryable(AllowedQueryParameters = ODataQueryParameters.Skip| ODataQueryParameters.Top)]

This means only Skip and Top are allowed, i.e. a $filter or $orderby would result in a validation error. By default every thing is allowed, until you mention a particular setting, then only the things you list are supported.

Another example is:

[Queryable(AllowedLogicalOperators=ODataLogicalOperators.Equal)]

This says the $filter only supports the Equal operator, so for example this:

~/Customer?$filter=Name eq ‘Bob’

will pass validation but this:

~/Customers?$filter=Name ne ‘Bob’

will fail.

Checkout the new [Queryable] attribute to find out more.

Delta<T> improvements

This now supports patching classes that derive from the T too. This is useful if you want both these requests:

PATCH ~/Customers(1)

PATCH ~/Customers(1)/Namespace.VIP

to be dispatched to the same method, for example:

public virtual HttpResponseMessage Patch([FromODataUri] TKey key, Delta<Customer> patch)
{
   …
}

Now because Delta<Customer> can also hold changes made to a VIP (i.e. a class the derives from Customer) you can route to the same method, and deal with a whole inheritance hierarchy in one action.

Another thing to notice is the new [FromODataUri] attribute, this tells Web API the key parameter is in the URL encoded as an OData Uri Literal.

$filter improvements

We now handle DateTime or DateTimeOffset literals or properties for the first time, and we allow you to specify casts in your filter too, for example:

~/Customers?$filter=Name eq ‘Bob’ or Namespace.VIP/RelationshipManager/Name eq ‘Bob’

At this point the only thing that we don’t support (and won’t support for RTM either) is spatial types and custom OData functions inside the filter. These will come later when support for them is added to ODataLib’s ODataUriParser.

Partial support for OData’s new JSON format

This release also includes a little support for the new OData v3 json format, and of course by RTM time we will have full support. This OData V3 json format is more efficient than our older V1/V2 json format (which incidentally the OData team has started calling ‘JSON verbose’, to make sure you know we don’t like it any more :)

The new JSON format has sub types, which allow clients to specify how much metadata they want, ranging from FullMetadata, through MinimalMetadata to NoMetadata. The RC supports the new OData JSON format only for write scenarios (i.e. responding to requests), it doesn’t currently handle read scenarios at all (i.e. request payloads that are in the new JSON format). It supports the most common payload kinds, feeds, entries, errors and service documents, and by RTM we will flesh this out to include other payloads like properties, links etc.

In the RC we don’t support NoMetadata at all, and we treat MinimalMetadata as if it is FullMetadata. We do this because MinimalMetadata means only send metadata and links that the client can’t deduce them by convention, and in the RC we don’t have anyway to tell the formatter that you are indeed following conventions. This forces us to always emit links. By RTM we will add a way to say you are following convention, and that will allow us to support MinimalMetadata properly.

Read What is JSON Light and Enabling JSON Light from WCF DS to learn more about the new OData JSON format.

Summary

As you can see we’ve been really busy making Web API a first class stack for creating and exposing OData services, ranging from supporting just the OData query syntax, through supporting the majority of the OData protocol, all the way up to supporting extra things not in OData but enabled by Web API, like for example new formats.

As always please let us know what you think.

-Alex

For a while now we’ve been shipping an early pre-release version of an ODataUriParser in ODataLib Contrib. The idea is to have code that converts a Uri used to interact with an OData Service into a Semantically bound Abstract Syntax Tree (AST) which represents the request in terms of the OData concepts used.

The ODataUriParser is currently an alpha at best, with many known functionality gaps and bugs. However the ASP.NET Web API OData library has a dependency on the parser and is RTMing soon, so we need to bring the quality up quickly.

Our plan is to move Uri Parser out of ODataLib Contrib into a fully supported ODataLib release.

Currently ASP.NET Web API only needs support for parsing $filter and $orderby: because today Web API doesn’t support $select and $expand, has it’s own parser for the PATH part of the Uri and all the other OData query options which are very simple to parse. This scoped set of requirements and tight time constraints means the ODataUriParser support we ship in ODataLib will initially be a subset, albeit a better tested and more complete subset, of the ODataUriParser currently in ODataLib Contrib.

So what will the API look like?

Parsing $filter

To parse a $filter you need at least two things: an IEdmModel, and an IEdmType to use as your collection element type. Optionally you can provide an IEdmEntitySet too.

For example imagine this request:

GET /Service/Customers?$filter=Name eq ‘ACME’

To parse the $filter query option, you would need an IEdmModel for the service hosted at ~/Service, and you need to provide the element type of the collection being filtered, in this case Customer.

To create the model, you can use EdmLib code directly:

var model= new EdmModel();

var customer = new EdmEntityType("TestModel", "Customer");
bool isNullable = false;
var idProperty = customer.AddStructuralProperty("Id", EdmCoreModel.Instance.GetInt32(isNullable);
customer.AddKeys(idProperty);
customer.AddStructuralProperty("Name", EdmCoreModel.Instance.GetString(isNullable));
edmModel.AddElement(customer);

var container= new EdmEntityContainer("TestModel", "DefaultContainer");
container.AddEntitySet(“Customers”, customer);
model.AddElement(container);

This code, builds a model with a single EntityType called customer (with Id and Name properties) an exposes customer instances via an EntitySet called Customers in the default container. Working with EdmLib directly like this, is however quite low-level, another simpler option is to use the ODataConventionModelBuilder from the Web API OData package, which provide a nice higher level API for building models. If you go the WebAPI route, be sure to checkout these blog posts:

• OData Support in ASP.NET Web API (August 2012)
• OData in Web API – Microsoft ASP.NET Web API OData 0.2.0-alpha release (November 2012)

In the above example working out the type of the collection is pretty simple, but of course OData paths (i.e. the bit before the ?) can be quite complicated. To help in the long run this functionality will be provided by the Uri Parser, but for now you have to work this out for the OData Uri Parser. That said, you are not alone, the WebAPI team had the same problem and has create a class called the  DefaultODataPathParser that will work out the type (and EntitySet if appropriate) of any OData path.

Once you have the IEdmModel and IEdmType you need, you can parse the $filter like this:

FilterNode filter = ODataUriParser.ParseFilter(“Name eq ‘ACME’”, model, customer);

If you start exploring this filter which is an AST, you will notice it looks like this:

You could easily visit this to produce some sort alternate query syntax, perhaps SQL or CLR expressions. In fact this is what the ASP.NET Web API does, to support the [Queryable] attribute, it first parses the $filter and then it converts the AST into a LINQ expression. You don’t need to do this though, you could for example convert from this AST directly to SQL, which is very useful if you don’t have a LINQ provider, because creating a LINQ provider is a significant undertaking.

Parsing $OrderBy

To parse a $orderby you again need an IEdmModel and an IEdmType for the element type of the collection you want to order. For example:

To handle this:

GET /Service/Customers?$orderby=Name desc,Id

Which orders first by Name descending and then by Id ascending, you would make this call:

OrderByNode orderby = ODataUriParser.ParseOrderBy(“Name desc, Id”, model, customer);

Again if you visit you will see something like this:

If you are familiar with ASTs this will seem pretty simple.

The Node type hierarchy

The root of the type hierarchy is QueryNode. Beyond that the node hierarchy has been organized to prioritize two key pieces of information, namely:

1. Whether the node represents a Collection or a Single value
2. Whether the node represents an Entity or another type.

The first level down from the root of the type hierarchy indicates whether you are dealing with a Single value (SingleValueNode) or a Collection (CollectionNode). Then the second level down is used to distinguish between Entities and other types, so for example there is a node called SingleEntityNode that derives from SingleValueNode and a EntityCollectionNode that derives from CollectionNode.

Then the most derived node types distinguish between the various ways of getting to a Collection or Single instance of an Entity or Nonentity. For example the various ways you can get a single entity, one possible way is by following a navigation property with target multiplicity of 0..1 or 1 (SingleNavigationNode), and so on, each of these ways are represented as nodes that derive from SingleEntityNode.

This approach yields a hierarchy that (when simplified) looks a little like this:

Amongst the leaf nodes, a few deserve a special mention:

• BinaryOperatorNode which represents a Binary operator like for example:  Name eq ‘Bob’ 
• UnaryOperatorNode which represents a Unary operator like for example: not True 
• AnyNode which represents OData any filter expression, for example: collection/any(a: a/Age gt 100)
• AllNode which represents an OData all filter expression, for example: collection/all(a: a/Age gt 10)

From a type system perspective these all represent a boolean value, so then all derive from the natural place in the type hierarchy SingleValueNode.

To help recognize which kind of node you are dealing with at any time, all nodes have a Kind property which in the RC return one of the constants defined on the static QueryNodeKind class. For example if you given a QueryNode and you want to write a visitor, you should visit the appropriate derived nodes, first by checking the QueryNode.Kind and then casting as appropriate. For RTM we’ll convert Kind to an Enum.

All nodes are held in the Expressions held by classes like FilterClause and OrderByClause, In the future new classes will be introduced to support ODataPath, $select, $expand, etc. Then finally we’ll add a class to represent the full url once we have full coverage of OData urls.

Conclusion

The ODataUriParser is on the road to RTM quality now, but it is not happening in one big step, we will start with $filter and $orderby and overtime cover the rest of OData including paths, $expand and $select.

If you are interested in the latest code I invite you to try out the latest version of ODataLib.

-Alex

Server Driven Paging:

Our code has supported client initiated paging using $skip and $top for some time now. However there are situations where the server wants to initiate paging. For example if a naïve (or potentially malicious) client makes a request like this:

GET ~/Products

and you have a lot of products, you are supposed to return all the products (potentially thousands, millions, billions) to the client. This uses a lot of computing resources, and this single request ties up all those resources. This is unfortunate because your client:

• Might simply be malicious
• Might be naïve, perhaps it only needed 20 results?
• Might lockup waiting for all the products to come over the wire.
• etc.

Thankfully OData has a way to initiate what we call server driven paging, this allows the server to return just a ‘page’ of results + a next link, which tells the client how to retrieve the next page of data. This means naïve clients only get the first page of data and servers have the opportunity to throttle requests from potentially malicious clients because to get all the data multiple requests are required.

This is now really easy to turn on in WebAPI using the Queryable attribute, like this:

[Queryable(ResultLimit=100)]
public IQueryable<Product> Get()
{
   …
}

This code, tells WebAPI to return the first 100 matching results, and then add an OData next-link to the results that when followed will re-enter the same method and continue retrieving the next 100 matching results. This process continues until either the client stops following next-links or there is no more data.

If the strategy [Queryable] uses for Server Driven Paging is not appropriate for your data source, you can also drop down and use ODataQueryOptions and ODataResult<T> directly.

Inheritance:

The OData protocol supports entity type inheritance, so one entity type can derive from another, and often you’ll want to setup inheritance in your service model. To support OData inheritance we have:

• Improved the ModelBuilder – you can explicitly define inheritance relationships or you can let the ODataConventionModelBuilder infer them for you automatically.
• Improved our formatters so we can serialize and deserialize derived types.
• Improved our link generation to include needed casts.
• and we need to improve our controller action selection so needed casts are routed correctly. 

Model Builder API

You can explicitly define inheritance relationships, with either the ODataModelBuilder or the ODataConventionModelBuilder, like this:

// define the Car type that inherits from Vehicle
modelBuilder
.Entity<Car>()
.DerivesFrom<Vehicle>()
.Property(c => c.SeatingCapacity); 

// define the Motorcycle type
modelBuilder
.Entity<Motorcycle>()
.DerivesFrom<Vehicle>()
.Property(m => m.CanDoAWheelie);

With inheritance it occasionally makes sense to mark entity types as abstract, which you can do like this:

modelBuilder
.Entity<Vehicle>()
.Abstract()
.HasKey(v => v.ID)
.Property(v => v.WheelCount); 

Here we are telling the model builder that Vehicle is an abstract entity type.

When working with derived types you can explicitly define properties and relationship on derived types just as before using EntityTypeConfiguration<TDerivedType>.Property(..), EntityTypeConfiguration<TDerivedType>.HasRequired(…) etc.

Note: In OData every entity type must have a key, either declared or inherited, whether it is abstract or not.

ODataConventionModelBuilder and inheritance

The ODataConventionModelBuilder, which is generally recommended over the ODataModelBuilder, will automatically infer inheritance hierarchies in the absence of explicit configuration. Then once the hierarchy is inferred, it will also infer properties and navigation properties too. This allows you to write less code, focusing on where you deviate from our conventions.  

For example this code:

ODataModelBuilder modelBuilder = new ODataConventionModelBuilder();
modelBuilder.EntitySet<Vehicle>(“Vehicle”);

Will look for classes derived from Vehicle and go ahead and create corresponding entity types.

Sometimes you don’t want to have entity types for every .NET type, this is easy to achieve you instruct the model builder to ignore types like this:

builder.IgnoreTypes(typeof(Sportbike));

With this code in place the implicit model discovery will not add an entity type for Sportbike, even though it derives from Vehicle (in this case indirectly i.e. Sportbike –> Motorcycle –> Vehicle).

Known inheritance issues

In this alpha our support for Inheritance is not quite complete. You can create a service with inheritance in it but there are a number of issues we plan to resolve by RTM:

• Delta<T>doesn’t currently support derived types. This means issuing PATCH requests against instances of a derived type is not currently working.
• Type filtering in the path is not currently supported. i.e. ~/Vehicles/NS.Motorcycles?$filter=…
• Type casts in $filter is not currently supported. i.e. ~/Vehicles?$filter=NS.Motorcyles/Manufacturer/Name eq ‘Ducati’
• Type casts in $orderby is not currently supported. i.e. ~/Vehicles?$filter=Name, NS.Motorcycle/Manufacturer/Name

OData Actions:

The other major addition since the august preview is support for OData Actions. Quoting the OData blog:

“Actions … provide a way to inject behaviors into an otherwise data centric model without confusing the data aspects of the model, while still staying true to the resource oriented underpinnings of OData."

Adding OData actions support to the WebAPI involves 4 things:

1. Defining OData Actions in the model builder.
2. Advertising bindable and available actions in representations of the entity sent to clients.
3. Deserializing parameters values when people attempt to invoke an Action.
4. Routing requests to invoke OData Actions to an appropriate controller action.

Model Builder API

Firstly we added a new class called ActionConfiguration. You can construct this directly if necessary, but generally you use factory methods that simplify configuring the most common kinds of OData Actions, namely those that bind to an Entity or a collection of Entities. For example:

ActionConfiguration pullWheelie = builder.Entity<Motorcycle>().Action(“PullWheelie”);
pullWheelie.Parameter<int>(“ForSeconds”);
pullWheelie.Returns<bool>();

defines an Action called ‘PullWheelie’ that binds to a Motorcycle, and that takes an integer parameter “ForSeconds” indicating how long to hold the wheelie, and returns true/false indicating whether the wheelie was successful.

You can also define an action that binds to a Collection of entities like this:

ActionConfiguration pullWheelie = builder.Entity<Motorcycle>().Collection.Action(“PullWheelie”);

Calling this action would result in a collection of Motorcycles all attempting to ‘Pull a Wheelie’ at the same time :)

There is currently no code in the ODataConventionModelBuilder to infer OData Actions, so actions have to be explicitly added for now. That might change as we formalize our conventions more, but if that happens it won’t be until after the first RTM release.

Controlling Action links and availability

When serializing an entity with Actions the OData serializer calls the delegate you pass to ActionConfiguration.HasActionLink(…) for each action. This delegate is responsible for returning a Uri to be embedded in the response sent to the client. The Uri when present tells clients how to invoke the OData Action bound to the current entity. Basically this is hypermedia.

If you are using the ODataConventionModelBuilder, by default the HasActionLink is automatically configured to generate links in the form: ~/entityset(key)[/cast]/action, or for example:

~/Vehicles(1)/Drive

or to access an action bound to a derived type like Motorcyles:

~/Vehicles(1)/Namespace.Motorcycle/PullWheelie

OData also allows you define actions that are only occasionally bindable. For example you might not be able to ‘Stop’ a Virtual Machine if it has already been stopped. This makes ‘Stop’ a transient action. Use the TransientAction() method, which like Action hangs off EntityTypeConfiguration<T>, to define your transient actions.

Finally to make your action truly transient you need to pass a delegate to HasActionLink that returns null when the action is in fact not available.

Handing requests to invoke ODataActions

The OData specification says that OData Actions must be invoked using a POST request, and that parameters to the action (excluding the binding parameter) are passed in the body of post in JSON format. This example shows an implementation of the PullWheelie action that binds to a Motorcycle:

[HttpPost]
public bool PullWheelieOnMotorcycle(int boundId, ODataActionParameters parameters)
{
   // retrieve the binding parameter, in this case a motorcycle, using the boundId.
   // i.e. POST ~/Vehicles(boundId)/Namespace.Motorcycle/PullWheelie
   Motorcycle motorcycle = _dbContext.Vehicles.OfType<Motorcycle>().SingleOrDefault(m => m.Id == boundId);

   // extract the ForSeconds parameter
   int numberOfSeconds = (int) parameters[“ForSeconds”];

   // left as an exercise to the reader.
   DoWheelie(motorcycle, numberOfSeconds);

   return true;
}

As you can see there is a special class here called ODataActionParameters, this is configured to tell the ODataMediaTypeFormatter to read the POST body as an OData Action invocation payload. The ODataActionParameters class is essentially a dictionary from which you can retrieve the parameters used to invoke the action. In this case you can see we are extracting the ‘ForSeconds’ parameter. Finally because the PullWheelie action was configured to return Bool when we defined it, we simply return Bool and the ODataMediaTypeFormatter takes care of the rest.

The only remaining change is setting up routing to handle Actions, Inheritance, NavigationProperties and all the other OData conventions. Unfortunately this problem is a little too involved for standard WebAPI routing. Which means integrating all these new features is pretty hard.

We realized this and started fleshing something called System.Web.Http.OData.Futures to help out…

Integrating everything using System.Web.Http.OData.Futures

While the core of all these new features are complete, getting them to work together is tricky. That makes checking out the ODataSampleService (part of aspnet.codeplex.com) even more important.

The sample service itself is starting to look a lot simpler: you don’t need to worry about setting up complicated routes, registering OData formatters etc. In fact all you need to do is call EnableOData(…) on your configuration, passing in your model:

// Create your configuration (in this case a selfhost one).
HttpSelfHostConfiguration configuration = new HttpSelfHostConfiguration(_baseAddress);
configuration.Formatters.Clear();

// Enable OData
configuration.EnableOData(GetEdmModel());

// Create server
server = new HttpSelfHostServer(configuration);

// Start listening
server.OpenAsync().Wait();
Console.WriteLine("Listening on " + _baseAddress);

As you can see this is pretty simple.

For a description of how GetEdmModel() works checkout my earlier post.

As you can imagine EnableOData(…) is doing quite a bit of magic, it:

• Registers the ODataMediaTypeFormatter
• Registers a wild card route, for matching all incoming OData requests
• Registers OData routes for generating links in responses (these will probably disappear by RTM).
• Registers custom Controller and Actions selectors that parse the incoming request path and dispatch all well understood OData requests by convention. The Action selector dispatches deeper OData requests (i.e. ~/People(1)/BestFriend/BestFriend) to a ‘catch all method’ called HandleUnmappedRequest(…) which you can override if you want.

All of this is implemented in System.Web.Http.OData.Futures, which includes:

• OData Route information.
• OData specific Controller and Actions selectors – These classes help avoid routing conflicts. They are necessary because WebAPI’s built-in routing is not sophisticated enough to handle OData’s context sensitive routing needs.
• ODataPathParser and ODataPathSegment – These classes help our custom selectors establish context and route to Controller actions based on conventions.
• EntitySetController<T> – This class implements the conventions used by our custom selectors, and provides a convenient base class for your controllers when supporting OData.

System.Web.Http.OData.Futures is currently only at sample quality, but it is basically required to creating OData services today, so we plan on merging it into System.Web.Http.OData for the RTM release.

OData WebAPI Action Routing Conventions:

The OData ActionSelector is designed to work best with the EntitySetController<TEntity,TKey> and it relies on a series of routing conventions to dispatch OData requests to Controller Actions. You don’t actually need to use the EntitySetController class, so long as you follow the conventions that the OData ActionSelector uses.

The conventions currently defined in Futures and used by the sample are:

These conventions are not complete, in fact by RTM we expect to add a few more, in particular to handle:

We are also experimenting with the idea that anything that doesn’t match one of these conventions will get routed to:

If we did this you would be able to override the default implementation of this and potentially handle OData requests at arbitrary depths :)

Summary

As you can see we’ve made a lot of progress, and our OData support is getting more complete and compelling all the time. We still have a number of things to do before RTM, including bringing the ideas from System.Web.Http.OData.Futures into System.Web.Http.OData, finalizing conventions, enabling JSON light, working on performance and fixing bugs etc. That said I hope you’ll agree that things are taking shape nicely?

As always we are keen to hear what you think, even more so if you've kicked the tires a little!

-Alex

The rest of this post will outline what is currently supported, what limitations currently exist and which limitations are hopefully just temporary.

Current Support

Support for different ElementTypes

In the preview the [Queryable] attribute works with any IQueryable<> or IEnumerable<> data source (Entity Framework or otherwise), for which a model has been configured or can be inferred automatically.

Today this means that the element type (i.e. the T in IQueryable<T>) must be viewed as an EDM entity. This implies a few constraints:

• All properties you wish to expose must be exposed as CLR properties on your class.
• A key property (or properties) must be available
• The type of all properties must be either:
• a clr type that is mapped to an EDM primitive, i.e. System.String == Edm.String
• Or clr type that is mapped to another type in your model, be that a ComplexType or an EntityType

NOTE: using IEnumerable<> is recommended only for small amounts of data, because the options are only applied after everything has been pulled into memory.

Null Propagation

This feature takes a little explaining, so please bear with me. Imagine you have an action that looks like this:

[Queryable]
public IQueryable<Product> Get()
{
    …
}

Now imagine someone issues this request:

GET ~/Products?$filter=startswith(Category/Name,’A’)

You might think the [Queryable] attribute will translate the request to something like this:

Get().Where(p => p.Category.Name.StartsWith(“A"));

But that might be very bad…
If your Get() method body looks like this:

return _db.Products; // i.e. Entity Framework.

It will work just fine. But if your Get() method looks like this:

return products.AsQueryable();

It means the LINQ provider being used is LINQ to Objects. L2O evaluates the where predicate in memory simply by calling the predicate. Which could easily null ref if either p.Category or p.Category.Name are null.

The [Queryable] attribute handles this automatically by injecting null guards into the code for certain IQueryable Providers. If you dig into the code for ODataQueryOptions you’ll see this code:

…
string queryProviderAssemblyName = query.Provider.GetType().Assembly.GetName().Name;
switch (queryProviderAssemblyName)
{
    case EntityFrameworkQueryProviderAssemblyName:
        handleNullPropagation = false;
        break;
    case Linq2SqlQueryProviderAssemblyName:
        handleNullPropagation = false;
        break;
    case Linq2ObjectsQueryProviderAssemblyName:
        handleNullPropagation = true;
        break;
    default:
    handleNullPropagation = true;
    break;
}
return ApplyTo(query, handleNullPropagation);

As you can see for Entity Framework and LINQ to SQL we don’t inject null guards (because SQL takes care of null guards/propagation automatically), but for L2O and all other query providers we inject null guards and propagate nulls.

If you don’t like this behavior you can override it by dropping down and calling ODataQueryOptions.Filter.ApplyTo(..) directly.

Supported Query Options

In the preview the [Queryable] attribute supports only 4 of OData’s 8 built-in query options, namely $filter, $orderby, $skip and $top.

What about the 4 other query options? i.e. $select, $expand, $inlinecount and $skiptoken. Today you need to use ODataQueryOptions rather than [Queryable], hopefully that will change overtime.

Dropping down to ODataQueryOptions

The first thing to understand is that this code:

[Queryable]
public IQueryable<Product> Get()
{
    return _db.Products;
}

Is roughly equivalent to:

public IEnumerable<Product> Get(ODataQueryOptions options)
{
    // TODO: we should add an override of ApplyTo that avoid all these casts!
    return options.ApplyTo(_db.Products as IQueryable) as IEnumerable<Product>;
}

Which in turn is roughly equivalent to:

public IEnumerable<Product> Get(ODataQueryOptions options)
{

    IQueryable results = _db.Products;
    if (options.Filter != null)
        results = options.Filter.ApplyTo(results);
    if (options.OrderBy != null) // this is a slight over-simplification see this.
        results = options.OrderBy.ApplyTo(results);
    if (options.Skip != null)
        results = options.Skip.ApplyTo(results);
    if (options.Top != null)
        results = options.Top.ApplyTo(results);

    return results;
}

This means you can easily pick and choose which options to support. For example if your service doesn’t support $orderby you can assert that ODataQueryOptions.OrderBy is null.

ODataQueryOptions.RawValues

Once you’ve dropped down to the  ODataQueryOptions you also get access to the RawValues property which gives you the raw string values of all 8 ODataQueryOptions… So in theory you can handle more query options.

ODataQueryOptions.Filter.QueryNode

The ApplyTo method assumes you have an IQueryable, but what if you backend has no IQueryable implementation?

Creating one from scratch is very hard, mainly because LINQ allows so much more than OData allows, and essentially obfuscates the intent of the query.
To avoid this complexity we provide ODataQueryOptions.Filter.QueryNode which is an AST that gives you a parsed metadata bound tree representing the $filter. The AST of course it tuned to allow only what OData supports, making it much simpler than a LINQ expression.

For example this test fragment illustrates the API:

var filter = new FilterQueryOption("Name eq 'MSFT'", context); 
var node = filter.QueryNode; 
Assert.Equal(QueryNodeKind.BinaryOperator, node.Expression.Kind); 
var binaryNode = node.Expression as BinaryOperatorQueryNode; 
Assert.Equal(BinaryOperatorKind.Equal, binaryNode.OperatorKind); 
Assert.Equal(QueryNodeKind.Constant, binaryNode.Right.Kind); 
Assert.Equal("MSFT", ((ConstantQueryNode)binaryNode.Right).Value); 
Assert.Equal(QueryNodeKind.PropertyAccess, binaryNode.Left.Kind); 
var propertyAccessNode = binaryNode.Left as PropertyAccessQueryNode; 
Assert.Equal("Name", propertyAccessNode.Property.Name);

If you are interested in an example that converts one of these ASTs into another language take a look at the FilterBinder class. This class is used under the hood by ODataQueryOptions to convert the Filter AST into a LINQ Expression of the form Expression<Func<T,bool>>.

You could do something very similar to convert directly to SQL or whatever query language you need. Let me assure you doing this is MUCH easier than implementing IQueryable!

ODataQueryOptions.OrderBy.QueryNode

Likewise you can interrogate the ODataQueryOptions.OrderBy.Query for an AST representing the $orderby query option.

Possible Roadmap?

These are just ideas at this stage, really we want to hear what you want, that said, here is what we’ve been thinking about:

Support for $select and $expand

We hope to add support for both of these both as QueryNodes (like Filter and OrderBy), and natively by the [Queryable] attribute.

But first we need to work through some issues:

• The OData Uri Parser (part of ODataContrib) currently doesn’t support $select / $expand, and we need that first.
• Both $expand and $select essentially change the shape of the response. For example you are still returning IQueryable<T> from your action but:
• Each T might have properties that are not loaded. How would the formatter know which properties are not loaded?
• Each T might have relationships loaded, but simply touching an unloaded relationship might cause lazyloading, so the formatters can’t simply hit a relationship during serialization as this would perform terribly, they need to know what to try to format.
• There is no guarantee that you can ‘expand’ an IEnumerable or for that matter an IQueryable, so we would need a way to tell [Queryable] which options it is free to try to handle automatically.

Support for $inlinecount and $skiptoken

Again we hope to add support to [Queryable] for both of these.

That said today you can implement both of these by returning ODataResult<> from your action today.
Implementing $inlinecount is pretty simple:

public ODataResult<Product> Get(ODataQueryOptions options)

{

    var results = (options.ApplyTo(_db.Products) as IQueryable<Product>);

    var count = results.Count;

    var limitedResults = results.Take(100).ToArray();

    return new ODataResult<Product>(results,null,count);

}

However implementing server driven paging (i.e. $skiptoken) is more involved and easy to get wrong. 
I’ll blog about how to do Server Driven Pages pretty soon.

Support for more Element Types.

We want to support both Complex Types (Complex Type are just like entities, except they don’t have a key and have no relationships) and primitive element types. For example both:

public IQueryable<string> Get(); – maps to say GET ~/Tags

and

public IQueryable<Address> Get(parentId); – maps to say GET ~/Person(6)/Addresses

where no key property has been configured or can be inferred for Address.

You might be asking yourself how do you query a collection of primitives using OData? Well in OData you use the $it implicit iteration variable like this:

GET ~/Tags?$filter=startswith($it,’A’)

Which gets all the Tags that start with ‘A’.

Virtual Properties and Open Types

Essentially virtual properties are things you want to expose as properties via your service that have no corresponding clr property. A good example might be where you to use methods to get and set a property value. This one is a little further out, but it is clearly useful.

Conclusion

As you can see [Queryable] is a work in progress that is layered above ODataQueryOptions, we are planning to improve both over time, and we have a number of ideas. But as always we’d love to hear what you think!

UPDATE 1 @10:00 am on 16th August (PST):
If you’ve tried using the preview nuget package and had problems, rest assured we are working on the issue (which is a dependency version issue). Essentially the preview and one of its dependencies have conflicting dependencies on specific versions of packages. The ETA for a fix that resolves this nuget issue is later today. 
Also if you’ve made a comment and it isn’t showing up, don’t worry it will. I’m currently having technical problems approving comments.

Earlier versions ASP.NET Web API included basic support for the OData, allowing you to use the OData Query Options $filter, $orderby, $top and $skip to shape the results of controller actions annotated with the [Queryable] attribute. This was very useful and worked across formats. That said true support for the OData format was rather limited.  In fact we barely scratched the surface of OData, for example there was no support for creates, updates, deletes, $metadata and code generation etc.

To address this we’ve create a preview of a new nuget package (and codeplex project) for building OData services that:

• Continues to support the [Queryable] attribute, but also allows you to drop down to an Abstract Syntax Tree (or AST) representing $filter & $orderby.
• Adds ways to infer a model by convention or explicitly customize a model that will be familiar to anyone who’s used Entity Framework Code First.
• Adds support for service documents and $metadata so you can generate clients (in .NET, Windows Phone, Metro etc) for your Web API.
• Adds support for creating, updating, partially updating and deleting entities.
• Adds support for querying and manipulating relationships between entities.
• Adds the ability to create relationship links that wire up to your routes.
• Adds support for complex types.
• Adds support for Any/All in $filter.
• Adds the ability to control null propagation if needed (for example to avoid null refs working about LINQ to Objects).
• Refactors everything to build upon the same foundation as WCF Data Services, namely ODataLib.

In fact this is an early preview of a new OData Server stack built to take advantage of Web APIs inherent flexibility and power which compliments WCF Data Services.

This preview ships with a companion OData sample service built using Web API. The sample includes three controllers, that each expose an OData EntitySet with varying capabilities. One is rudimentary supporting just query and create, the other two are more complete, supporting Query, Create, Update, Patch, Delete and Relationships. The first complete example does everything by hand, the second derives from a sample base controller called EntitySetController that takes care of a lot of the plumbing for you and allows you to focus on the business logic.

The rest of this blog post will introduce you to the components that make up this preview and how to stitch them together to create an OData service, using the code from this sample from the ASP.NET Web Stack Sample repository if you want to follow along.

[Queryable] aka supporting OData Query

If you want to support OData Query options, without necessarily supporting the OData Formats, all you need to do it put the [Queryable] attribute on an action that returns either IQueryable<> or IEnumerable<>, like this:

[Queryable]
public IQueryable<Supplier> GetSuppliers()
{
    return _db.Suppliers;
}

Here _db is an EntityFramework DBContext. If this action is routed to say ~/Suppliers then any OData Query options applied to that URI will be applied by an Action Filter before the result is sent to the client.

For example this: ~/Suppliers?$filter=Name eq ‘Microsoft’

Will pass the result of GetSuppliers().Where(s => s.Name == “Microsoft”) to the formatter.

This all works as it did in earlier previews, although there are a couple of, hopefully temporary, caveats:

• The element type can’t be primitive (for example IQueryable<string>).
• Somehow the [Queryable] attribute must find a key property. This happens automatically if your element type has an ID property, if not you might need to manually configure the model (see setting up your model).

Doing more OData

If you want to support more of OData, for example, the official OData formats or allow for more than just reads, you need to configure a few things.

The ODataService.Sample shows all of this working end to end, and the remainder of this post will talk you through what that code is doing.

Setting up your model

The first thing you need is a model. The way you do this is very similar to the Entity Framework Code First approach, but with a few OData specific tweaks:

• The ability to configure how EditLinks, SelfLinks and Ids are generated.
• The ability to configure how links to related entities are generated.
• Support for multiple entity sets with the same type.

If you use the ODataConventionModelBuilder most of this is done automatically for you, all you need to do is tell the builder what sets you want, for example this code:

ODataModelBuilder modelBuilder = new ODataConventionModelBuilder();
modelBuilder.EntitySet<Product>("Products");
modelBuilder.EntitySet<ProductFamily>("ProductFamilies");
modelBuilder.EntitySet<Supplier>("Suppliers");
IEdmModel model = modelBuilder.GetEdmModel();

Builds a model with 3 EntitySets (Products, ProductFamilies and Suppliers) where the EntityTypes of those sets are inferred from the CLR types (Product, ProductFamily and Supplier) automatically, and where EditLinks, SelfLinks and IdLinks are all configured to use the default OData routes.

You can also take full control of the model by using the ODataModelBuilder class, here you explicitly add EntityTypes, Properties, Keys, NavigationProperties and how to route Links by hand. For example this code:

var products = modelBuilder.EntitySet<Product>("Products");
products.HasEditLink(entityContext => entityContext.UrlHelper.Link( 
    ODataRouteNames.GetById, 
    new { controller = "Products", id = entityContext.EntityInstance.ID } 
));
var product = products.EntityType;
product.HasKey(p => p.ID);
product.Property(p => p.Name);
product.Property(p => p.ReleaseDate);
product.Property(p => p.SupportedUntil);

Explicitly adds a EntitySet called Products to the model, configures the EditLink (and unless overridden the SelfLink and IdLink) generation so that if uses the ODataRouteNames.GetById route, and then as you can see the code needed to configure the Key and Properties is very similar to the Code First.

For a more complete example take a look at the GetExplicitEdmModel() method in the sample it builds the exact same model as the ODataConventionModelBuilding by hand.

Setting up the formatters, routes and built-in controllers

To use the OData formats you first need to register an ODataMediaTypeFormatter, which will need the model you built previously:

// Create the OData formatter and give it the model
ODataMediaTypeFormatter odataFormatter = new ODataMediaTypeFormatter(model);

// Register the OData formatter
configuration.Formatters.Insert(0, odataFormatter);

Next you need to setup some routes to handle common OData requests, below are the routes required for a Read/Write OData model built using the OData Routing conventions that also supports client side code-generation (vital if you want a WCF DS client application to talk to your service).

// Metadata routes to support $metadata and code generation in the WCF Data Service client.
configuration.Routes.MapHttpRoute(
    ODataRouteNames.Metadata,
    "$metadata",
    new { Controller = "ODataMetadata", Action = "GetMetadata" }
);
configuration.Routes.MapHttpRoute(
    ODataRouteNames.ServiceDocument,
    "",
    new { Controller = "ODataMetadata", Action = "GetServiceDocument" }
);

// Relationship routes (notice the parameters is {type}Id not id, this avoids colliding with GetById(id)).
// This code handles requests like ~/ProductFamilies(1)/Products
configuration.Routes.MapHttpRoute(ODataRouteNames.PropertyNavigation, "{controller}({parentId})/{navigationProperty}");

// Route for manipulating links, the code allows people to create and delete relationships between entities
configuration.Routes.MapHttpRoute(ODataRouteNames.Link, "{controller}({id})/$links/{navigationProperty}");

// Routes for urls both producing and handling urls like ~/Product(1), ~/Products() and ~/Products
configuration.Routes.MapHttpRoute(ODataRouteNames.GetById, "{controller}({id})");
configuration.Routes.MapHttpRoute(ODataRouteNames.DefaultWithParentheses, "{controller}()");
configuration.Routes.MapHttpRoute(ODataRouteNames.Default, "{controller}");

One thing to note is the way that the ODataRouteNames.PropertyNavigation route attempts to handle requests to urls like ~/ProductFamilies(1)/Products and ~/Products(1)/Family etc. Essentially a single route for all Navigations. For this to work without requiring a single Action for all navigation properties, we register a custom action selector that will build the Action name using the {navigationProperty} parameter of the PropertyNavigation route:

// Register an Action selector that can include template parameters in the name
configuration.Services.Replace(typeof(IHttpActionSelector), new ODataActionSelector());

This custom action selector will dispatch a request to GET ~/ProductFamilies(1)/Products, to an action called the GetProducts(int parentId) on the ProductFamilies controller.

At this point our previous GetSupplier action can return OData formats. However it will still produce links that won’t work when dereferenced. To fix this we need to start creating our controllers.

Adding Support for OData requests

In our model we added 3 entitysets: Products, ProductFamilies and Suppliers. So first we create 3 controllers, called ProductController, ProductFamiliesController and SuppliersController respectively.

Queries

By convention your controllers should have a method called Getxxx() that returns IQueryable<T>. Here the T is the CLR type backing your EntitySet, so for example on the ProductsController which is for the Products entityset the T should be Product, and the action should look something like this:

[Queryable]
public IQueryable<Product> GetProducts()
{
    return _db.Products;
}

As you can see this is the same as before, but now it is participating in a ‘compliant’ OData service. Inside this method you can do all sorts of things, you can add additional filters based on the identity of the caller, you do auditing, you can do logging, you can aggregate data from multiple sources.

There is even a way to drop down a layer so you get to the see the OData query that has been received. Doing this allows you to process the query yourself in whatever way is appropriate for your data sources. For example you might not have an IQueryable you can use. See ODataQueryOptions for more on this.

At this point because you’ve registered a model and you’ve setup routes for $metadata and the OData Service Document, you should be able to use Visual Studio’s Add Service Reference to generate a set of client proxy classes that you could use like this:

foreach(var product in ctx.Products.Where(p => p.Name.StartsWith(“MS”))
{
     Console.WriteLine(product.Name);
}

WCF DS client will then translate this into a GET request like this:

~/Products$filter=startswith(Name,’MS’)

And everything should just work.

Get by Key

To retrieve an individual item using it’s key in OData you send a GET request to a url like this: ~/Products(1). The ODataRouteNames.GetById route handles requests like this:

public HttpResponseMessage GetById(int id)
{
    Supplier supplier = _db.Suppliers.SingleOrDefault(s => s.ID == id); 
    if (supplier == null) 
    { 
         return Request.CreateResponse(HttpStatusCode.NotFound); 
    } 
    else 
    { 
         return Request.CreateResponse(HttpStatusCode.OK, supplier); 
    }
}

As you can see the code simply attempts to retrieve an item by ID, and then return it with a 200 OK response or return 404 Not Found.

If you have WCF DS proxy classes on the client a request like this:

ctx.Products.Where(p => p.ID == 5).FirstOrDefault();

Should now hit this action.

Inserts (POST requests)

To create a new Entity in OData you POST a new Entity directly to the url that represents the EntitySet, this means that insert requests just like queries end up matching the ODataRouteNames.Default route, however Web API’s action selector picks actions prefixed with Post for POST requests to distinguish from GET requests for Queries:

public HttpResponseMessage PostProduct(Product product)
{
    product.Family = null;

    Product addedProduct = _db.Products.Add(product); 
    _db.SaveChanges(); 

    var response = Request.CreateResponse(HttpStatusCode.Created, addedProduct); 
    response.Headers.Location = new Uri(
       Url.Link(ODataRouteNames.GetById, new { Controller = "Products", Id = addedProduct.ID })
    ); 
    return response;
}

Here we simply add the product to our Entity Framework context and then call SaveChanges(). The only interesting bit way we set the location header using the ODataRouteNames.GetById route (i.e. the default OData route for Self/Edit links).

Updates (PUT requests)

To replace an Entity in OData you PUT the updated version of the entity to the uri specified in the EditLink of the entity. This will match the GetById route, but because it is a PUT request it will look for an Action name prefixed with Put.

This example shows how to do this:

public HttpResponseMessage Put(int id, Product update)
{
    if (!_db.Products.Any(p => p.ID == id))
    {
        throw ODataErrors.EntityNotFound(Request);
    }
    update.ID = id; // ignore the ID in the entity use the ID in the URL.
    _db.Products.Attach(update);
    _db.Entry(update).State = System.Data.EntityState.Modified;
    _db.SaveChanges();
    return Request.CreateResponse(HttpStatusCode.NoContent);
}

The code is pretty simple, first we verify the entity the client is trying to update actually exists, then we ignore the ID in the entity and instead use the id extracted from the uri of the request, this stops clients updating one entity using the editlink of another entity.

Once all that validation is out of the way we attach the updated entity and tell EF it has been modified before calling save changes and finally returning a No Content response.

As you can see this leverages a helper class called ODataErrors, its job is simply to raise OData compliant errors. The sample uses the EntityNotFound method quite frequently, and it follows the general pattern for failing in an OData compliant way so lets take a peak at its implementation:

public static HttpResponseException EntityNotFound(this HttpRequestMessage request)
{ 
    return new HttpResponseException( 
        request.CreateResponse( 
            HttpStatusCode.NotFound, 
            new ODataError 
            { 
                Message = "The entity was not found.", 
                MessageLanguage = "en-US", 
                ErrorCode = "Entity Not Found." 
            } 
        ) 
    );
}

We create a HttpResponseException with a nested Not Found response, the body of which contains an ODataError. By putting an ODataError in the body we get the ODataMediaTypeFormatter to format the body as a valid OData error message as per the content-type requested by the client.

NOTE: Over time we will try to teach the ODataMediaTypeFormatter to handle HttpError as well by using a default translation from HttpError to ODataError.

Partial Updates (PATCH requests)

PUT request have replace semantics which makes updates all or nothing, meaning you have to send all the properties even if only a subset have changed.This is where PATCH comes in, PATCH allows clients to send just the modified properties on the wire, essentially allowing for partial updates.

This example shows an implementation of Patch for Products:

public HttpResponseMessage PatchProduct(int id, Delta<Product> product)
{
    Product dbProduct = _db.Products.SingleOrDefault(p => p.ID == id);
    if (dbProduct == null)
    {
        throw new HttpResponseException(HttpStatusCode.NotFound);
    }

    product.Patch(dbProduct);
    _db.SaveChanges();

    return Request.CreateResponse(HttpStatusCode.NoContent);
}

Notice that the method receives a Delta<Product> rather than a Product.

We do this because, if we use a Product directly and set the properties that came on the wire as part of the patch request, we would not be able to tell why a property has a default value. It could be because the request set the property to its default value or it could be because the property hasn’t been set yet. This could lead to mistakenly resetting properties the client doesn’t want reset. Can anyone say ‘data corruption’?

The Delta<Product> class is a dynamic class that acts as a lightweight proxy for a Product. It allows you to set any Product property, but it also remembers which properties you have set. It uses this knowledge when you call .Patch(..) to copy across only properties that have been set.

Given this we simply retrieve the product from the database and then we call Patch to apply the changes requested to the entity in the database. Once that is done we call SaveChanges to push the changes to the database.

Deletes (DELETE requests)

To delete an entity you send a DELETE request to it’s editlink, so this is similar to GetById, Put and Patch, except this time Web API will look for an action prefixed with Delete, something like this:

public HttpResponseMessage DeleteProduct(int id)
{
    Product toDelete = _db.Products.FirstOrDefault(p => p.ID == id);
    if (toDelete == null)
    {
        throw ODataErrors.EntityNotFound(Request);
    }
    _db.Products.Remove(toDelete);
    _db.SaveChanges();
    return Request.CreateResponse(HttpStatusCode.Accepted);
}

As you can see if the entity is found it is simply deleted and we return an Acccepted response.

Following Navigations

To address related entities in OData you typically start with the Uri of a particular entity and then append the name of the navigationProperty. For example to retrieve the Family for Product 15 you do a GET request here: ~/Products(15)/Family.

Implementing this is pretty simple:

public ProductFamily GetFamily(int parentId)
{
     return _db.Products.Where(p => p.ID == parentId).Select(p => p.Family).SingleOrDefault();
}

This matches the ODataRouteNames.PropertyNavigation which you’ll notices uses {parentId} rather than {id}, if it was id, this method would also be a match for the ODataRouteNames.GetById route, which would cause problems in the action selector infrastructure. By using parentId we sidestep this issue.

Creating and Deleting links

The OData protocol allows you to get and modify links between entities as first class resources. In the preview however we only support modifications.

To create links you POST or PUT a uri (specified in the request body) to a url something like this: ~/Products(1)/$links/Family.

The uri in the body points to the resource you wish to assign to the relationship. So in this case it would be the uri of a ProductFamily you wish to assign to Product.Family (i.e. setting Product 1’s Family).

Whether you POST or PUT depends on the cardinality of the relationship, if it is a Collection you POST, if it is a single item (as Family is) you PUT. Both are mapped through the ODataRouteNames.Link route, which is set up to handle creating links for any relationship. Unlike following Navigations it feels okay to use a single method because the return type always the same, i.e. a No Content response:

public HttpResponseMessage PutLink(int id, string navigationProperty, [FromBody] Uri link)
{
    Product product = _db.Products.SingleOrDefault(p => p.ID == id);

    switch (navigationProperty)
    {
        case "Family":
            // The utility method uses routing (ODataRoutes.GetById should match) to get the value of {id} parameter
            // which is the id of the ProductFamily.
            int relatedId = Configuration.GetKeyValue<int>(link);
            ProductFamily family = _db.ProductFamilies.SingleOrDefault(f => f.ID == relatedId);
            product.Family = family;
            break;

        default:
            throw ODataErrors.CreatingLinkNotSupported(Request, navigationProperty); 
    }
    _db.SaveChanges();

    return Request.CreateResponse(HttpStatusCode.NoContent);
}

The {navigationProperty} parameter from the route is used to decide what relationship to create, also notice the [FromBody] attribute on the Uri link parameter it is vital otherwise the link will always be null.

If you want to remove a relationship between entities you send a DELETE request to the same URL, the code for that looks like this:

public HttpResponseMessage DeleteLink(int id, string navigationProperty, [FromBody] Uri link)
{
    Product product = _db.Products.SingleOrDefault(p => p.ID == id);
    switch (navigationProperty)
    {
        case "Family":
            product.Family = null;
            break;

        default:
            throw ODataErrors.DeletingLinkNotSupported(Request, navigationProperty);
    }
    _db.SaveChanges();
    return Request.CreateResponse(HttpStatusCode.NoContent);
}

Conclusion

If you implement all these methods for each of your OData EntitySets you should have a compliant OData service. Of course this code is only at preview quality so is likely to have bugs, that said I hope you’ll agree this provides a good foundation for creating OData services.

We are currently thinking about adding support for: inheritance, OData actions & functions, etags and JSON Light etc. And of course we want to hear your thoughts so we can incorporate your feedback!

Next up

This blog post doesn’t cover everything you can do with the Preview, you can use:

• ODataQueryOptions rather than [Queryable] to take full control of handling the query.
• ODataResult<> to implement OData features like Server Driven Paging and $inlinecount.
• EntitySetController<,> to simplify creating fully compliant OData entitysets.

I’ll be blogging more about these soon.

Enjoy!

I’ll admit I’m taking some liberties here because these two models are actually ‘merged’ into a single metadata document ($metadata) that describes them both, and you can share types between these two models… however this feels insignificant because the key benefits remain.

Why would you want to move from a CRUD style application to a CQRS style one?

Let’s look at a simple scenario, imagine you have Products that look like this:

public class Product
{
   public int ID {get;set;}
   public string Name {get;set;}
   public Decimal Cost {get;set;}
   public Decimal Price {get;set;}
}

And imagine you want to Discount Marmite (a product in your catalog) by 15%. Today using the CRUD style, the default in OData before Actions, there is only one option: you PUT a new version of the Marmite resource with the new Price to the URL that represents Marmite, i.e. something like this:

POST ~/Products(15) HTTP/1.1
Content-Type: application/json

{
   // abbreviated for readability
   “ID”: 15,
   “Name”: “Marmite”,
   “Cost”: 3.50,
   “Price”: 4.25    // ($5 – 15%) 
}

Notice to support this you have to allow PUT for Products. And this has some real issues:

• People can now make changes that we don’t necessarily want to allow, i.e. modifying the Name & Cost or changing the Price too much.
  • Basically “Update Product” is NOT the same as “Discount Product”.
• When a change comes through we don’t actually know it is a Discount. It just looks like an attempt to update a Product.
• If you need information that is not part of Product to perform a Discount (perhaps a justification) there is no where to put that information.

More generally the CRUD model is painful because:

• If you want to update lots of resources simultaneously, imagine for example that you want to discount every product in a particular category, you first have to retrieve every product in that category, and then you have to do a PUT for each of them. This of course introduces a lot of unnecessary latency and introduces consistency challenges (it is hard to maintain a transaction boundary across requests & it the longer the ‘transaction’ lasts the more likely a concurrency check will fail).
• If you want to update something you have to allow it to be read somewhere.

Back to our scenario, it would be much better to disable PUT completely and create a Discount action, and advertise it’s availability in the Marmite resource (to keep your system as Hypermedia driven as possible):

{
  “__metadata”: {
     // abbreviated for simplicity
     “Actions”: {
          “#MyActions.Discount”: [{ “title”: “Discount Marmite”, “target”: “Products(15)/Discount”}]
     }
  }, 
  “ID”: 15,
  “Name”: “Marmite”,
  “Cost”: 3.50,
  “Price”: 5.00
}

The name of the Action (i.e. #MyActions.Discount) is an ‘anchor’ into the metadata document that can be found at ~/$metadata that says you need to provide a percentage.

POST ~/Products(15)/Discount HTTP/1.1
Content-Type: application/json

{
    “percentage”: 15
}

This is much better. Notice this doesn’t allow me to modify the Cost or the Name, and indeed can easily be validated to make sure the percentage is within an acceptable range, and it is semantically much clearer what is happening.

In fact by moving from a CRUD style architecture to one inspired by CQRS but based on actions you can:

• Give targeted update capabilities that:
  • Allow only certain parts of the ‘Read’ model to be modified.
  • Allow things that are not even in the ‘Read’ model to be modified or provided if needed.
• Selectively give users permissions to only the ‘Actions’ or Commands they need.
• Log every Action and replay them at a later date to rebuild your data (i.e. Event Sourcing).
• Capture what is requested (i.e. Discount the product) and respond immediately before the change has actually been made, safe in the knowledge you will eventually process the request and achieve “Eventual Consistency”.
• Capture more information about what is happening (i.e. User X discounted Marmite by 15% is much better than User X updated Marmite).
• Create Actions that manipulate a lot of entities simultaneously (i.e. POST ~/Categories(‘Yeast Spreads’)/Products/Discount…)

Of course simply separating the read/write models in OData doesn’t give you all of these advantages immediately, but at least it creates a foundation that can scale to support things like Event Sourcing or Eventual Consistency later if required.

Some of you may be thinking you can achieve many of these goals by having a more granular model that makes things like “Discount” a resource, and that would be true. However for most people using OData that way of thinking is foreign and more importantly the EDM foundations of OData get in the way a little too. So for me Actions seems like the right approach in OData.

I love this.
But what do you think?
-Alex

Unfortunately judgment isn't easy.

Now when I was younger - say 10 years ago - I was sure about everything - not least my latest harebrained API design. I chalk that up to inexperience and not knowing better.

But now after 15 years of making design mistakes I have a lot more experience. So much so that I can literally see problems with almost every design. A big part of this is experience morphing into instinct.

Instinct is very useful because it helps weed out absurd ideas quickly. But instinct has limits. It is not much use if you are feeling equally uneasy about 3 possible designs.

You need to realize your instincts are based on all your experiences, and no one project encompasses all your experiences. Which means you can easily can fooled into thinking something matters when it really doesn't.

For example if you feel uneasy about one of the possible solutions because it looks inefficient. It might be your 'performance is important instinct' kicking in. If you are working on a website expected to get millions of hits a day, that is a valid concern. But if this is a departmental app with 10 users who each hit the site no more than 10 times a day, then probably not so much.

I know this performance example is contrived, but believe me often it won't be so obvious. You need a reliable way to discard irrelevant concerns and focus on the real issues.

The best way to do this is to have a set of guiding principles for every project you work on. Things like:

• Simplicity is a feature
• Predictability & Uniformity
• Embrace existing standards
• Payload Driven (HATEOS)

Creating a list like this gives you something more concrete than instincts to test when you get that uneasy feeling.

If you have an uneasy feeling you simple ask yourself, Does this violate any of my projects guiding principles?

I generally find that uneasy feelings will vanish or become showstoppers when you do this.
Enough so that you'll be much more comfortable making a choice...

... perhaps even the right choice!

Articulating your thought process has a number of significant benefits:

• Once you know your approach you can critique and try to improve it.
• Once you know what you'd like to do, in a collaborative setting you can ask others for permission to give it a go. In my experience phrases like "I'd like to explore how this is similar to LINQ ..." work well if you want others to brainstorm with you.
• Once you understand what you do perhaps you can teach or mentor.
• Or perhaps others can teach you.

Clearly these are compelling.

So I thought I'd give this a go by trying to document an idealized version of the thought process for Any/All in OData...

The Problem

OData had no way to allow people to query entities based on properties of a related collection, and a lot of people, myself included, wanted to change this...

Step 1: Look for inspiration

I always look for things that I can borrow ideas from. Pattern matching, if you will, with something proven. Here you are limited only by your experience and imagination, there is an almost limitless supply of inspiration. The broader your experience the more source material you have and the higher the probability you will be able to see a useful pattern.

If you need it – and you shouldn’t - this is just another reason to keep learning new languages, paradigms and frameworks!

Sometimes there is something obvious you can pattern match with, other times you really have to stretch...

In this case the problem is really easy. Our source material is LINQ. LINQ already allows you to write queries like this:

from movie in Movies
where movie.Actors.Any(a => a.Name == 'Zack')
select movie; 

Which filters movies by the actors (a related collection) in the movie, and that is exactly what we need in OData.

Step 2: Translate from source material to problem domain

My next step is generally to take the ideas from the source material and try to translate them to my problem space. As you do this you'll start to notice the differences between source and problem, some insignificant, some troublesome.

This is where judgment comes in, you need to know when to go back to the drawing board. For me I know I'm on thin ice if I start saying things like 'well if I ignore X and Y, and imagine that Z is like this' . 

But don't give up too early either. Often the biggest wins are gained by comparing things that seem quite different until you look a little deeper and find a way to ignore the differences that don't really matter. For example Relational Databases and the web seem very different, until you focus on ForeignKeys vs Hyperlinks, pull vs push seems different until you read about IQueryable vs IObservable.

Notice not giving up here is all about having tolerance for ambiguity, the higher your tolerance, the further you can stretch an idea. Which is vital if you want to be really creative.  

In the case of Any/All it turns out that the inspiration and problem domain are very similar, and the differences are insignificant.

So how do you do the translation?

In OData predicate filters are expressed via $filter, so we need to convert this LINQ predicate:

(movie) => movie.Actors.Any(actor => actor.Name == 'Zack')  

into something we can put in an OData $filter.

Let's attack this systematically from left to right. In LINQ you need to name the lambda variable in the predicate, i.e. movie, but in OData there is no need to name the thing you are filtering, it is implicit, for example this: 

from movie in Movies
where movie.Name == "Donnie Darko"
select movie

is expressed like this in OData:

~/Movies/?$filter=Name eq 'Donnie Darko'

Notice there is no variable name, we access the Name of the movie implicitly. 

So we can skip the ‘movie’.

Next in LINQ the Any method is a built-in extension method called directly off the collection using '.Any'. In OData '/' is used in place of '.', and built-in methods are all lowercase, so that points are something like this: 

~/Movies/?$filter=Actors/any(????)

As mentioned previously in OData we don't name variables, everything is implicit. Which means we can ignore the actor variable. That leaves only the filter, which we can convert using existing LINQ to OData conversion rules, to yield something like this: 

~/Movies/?$filter=Actors/any(Name eq 'Zack')

Step 3: Did you lose something important?

There is a good chance you lost something important in this transformation, so my next step is generally to assess what information has been lost in translation. Paying particular attention to things that are important enough that your source material had specific constructs to capture them.

As you notice these differences you either need to convince yourself the differences don’t matter, or you need to add something new to your solution to bring it back. 

In our case you'll remember that in LINQ the actor being tested in the Any method had a name (i.e. 'actor'), yet in our current OData design it doesn't.

Is this important? 

Yes it is! Unlike the root filter, where there is only one variable in scope (i.e. the movie), inside an Any/All there are potentially two variables in scope (i.e. the movie and the actor). And if neither are named we won't be able to distinguish between them!

For example this query, which finds any movies with the same name as any actors who star in the movie, is unambiguous in LINQ: 

from movie in Movies
where movie.Actors.Any(actor => actor.Name == movie.Name)
select movie;

But our proposed equivalent is clearly nonsensical:

~/Movies/?$filter=Actors/any(Name eq Name) 

It seems we need a way to refer to both the inner (actor) and outer variables (movie) explicitly.

Now we can't change the way existing OData queries work - without breaking clients and servers already deployed - which means we can't explicitly name the outer variable, we can however introduce a way to refer to it implicitly. This should be a reserved name so it can't collide with any properties. OData already uses the $ prefix for reserved names (i.e. $filter, $skip etc) so we could try something like $it . This results in something like this: 

~/Movies/?$filter=Actors/any(Name eq $it/Name)

And now the query is unambiguous again. 

But unfortunately we aren't done yet. We need to make sure nesting works calls works too, for example this:

from movie in Movies
where movie.Actors.Any(actor => actor.Name == movie.Name && actor.Awards.Any(award => award.Name == 'Oscar'))
select movie;

If we translate this, with the current design we get this: 

~/Movies/?$filter=Actors/any(Name eq $it/Name AND Awards/any(Name eq 'Oscar'))

But now it is unclear whether Name eq 'Oscar' refers to the actor or the award. Perhaps we need to be able to name the actor and award variables too. Here we are not restricted by the past, Any/All is new, so it can include a way to explicitly name the variable. Again we look at LINQ for inspiration: 

award => award.Name == 'Oscar'

Cleary we need something like '=>' that is URI friendly and compatible with the current OData design. It turns out ':' is a good candidate, because it works well in querystrings, and isn’t currently used by OData, and even better there is a precedent in Python lambda’s (notice the pattern matching again). So the final proposal is something like this: 

~/Movies/?$filter=Actors/any(actor: actor/Name eq $it/Name)

Or for the nested scenario: 

~/Movies/?$filter=Actors/any(actor: actor/Name eq $it/Name AND actor/Awards/any(award: award/Name eq 'Oscar'))

And again nothing is ambiguous.

Step 4: Rinse and Repeat

In this case the design feels good, so we are done.

But clearly that won't always be the case. However you should at least know whether the design is looking promising or not. If not it is back to the drawing board.

If it does look promising, I would essentially repeat steps 1-3 again using some other inspiration to inform further tweaking of the design, hopefully to something that feels complete.

Conclusion

While writing this up I definitely teased out a number of things that had previously been unconscious, and I sure this will help me going forward, hopefully you too?

Next time I do this I'll explore something that involves more creativity... and I'll try to tease out more of the unconscious tools I use.

What do you think about all this? Do you have any suggested improvements? 

-Alex

Including my Best Practices: Creating an OData Service using WCF Data Services session.

If you watch that video and want to follow try it out - you can download all the source code I uploaded previously.

Enjoy.