WCF Data Services Team Blog

New and improved EULA!

dpblogs — Wed, 13 Nov 2013 14:12:00 GMT

TL;DR: You can now (legally) use our .NET OData client and ODataLib on Android and iOS.

Backstory

For a while now we have been working with our legal team to improve the terms you agree to when you use one of our libraries (WCF Data Services, our OData client, or ODataLib). A year and a half ago, we announced that our EULA would include a redistribution clause. With the release of WCF Data Services 5.6.0, we introduced portable libraries for two primary reasons:

Portable libraries reduce the amount of duplicate code and #ifdefs in our code base.
Portable libraries increase our reach through third-party tooling like Xamarin (more on that later).

It took some work to get there, and we had to make some sacrifices along the way, but we are now focused exclusively on portable libraries for client-side code. Unfortunately, our EULA still contained a clause that prevented the redistributable code from being legally used on a platform other than Windows.

OData and Xamarin: Extending developer reach to many platforms

We are really excited about Microsoft’s new collaboration with Xamarin. As Soma says, this collaboration will allow .NET developers to broaden the reach of their applications and skills. This has long been the mantra of OData – a standardized ecosystem of services and consumers that enables consumers on any platform to easily consume services developed on any platform. This collaboration will make it much easier to write a shared code base that allows consumption of OData on Windows, Android or iOS.

EULA change

To fully enable this scenario, we needed to update our EULA. We, along with several other teams at Microsoft, are rolling out a new EULA today that has relaxed the distribution requirements. Most importantly, we removed the clause that prevented redistributable code from being used on Android and iOS. The new EULA is effective immediately for all of our NuGet packages. This means that (even though we already released 5.6.0) you can create a Xamarin project today, take a new dependency on our OData client, and legally run that application on any platform you wish.

Thanks

As always, we really appreciate your feedback. It frequently takes us some time to react, but the credit for this change is due entirely to customer feedback. We hear you. Keep it coming. Thanks,
The OData Team

odatacomponenteula.zip

New version of OData Validator

dpblogs — Sun, 27 Oct 2013 20:09:54 GMT

The OData team has been working on updating the OData Validator tool to support the new JSON format validation. We are pleased to announce that the tool now supports validating your V3 service for all three formats – ATOM, old JSON format (aka JSON Verbose) and the new JSON format. We are also working on adding OData V4 service validation support. We will continue adding more validation rules over the next few months.

Please check out the tool here : https://services.odata.org/validation/ and provide us feedback.

About the tool

OData Validator is an OData protocol validation tool. We have gone through the OData spec and created a set of rules to validate against a given OData payload. You can point the tool to your service and choose what you want to validate. The tool will run the right set of rules against the returned payload and tell you which ones passed and which ones failed. The tool will also link you to the relevant spec section so you can open the spec to see what it says. The tool supports validating various OData payloads like service document, metadata document, feed, entity and error payloads.

Using WCF Data Services 5.6.0 with Entity Framework 6+

dpblogs — Wed, 02 Oct 2013 15:05:01 GMT

And now for some exciting news: you can finally use WCF Data Services with Entity Framework 6+! Today we are uploading a new NuGet package called WCF Data Services Entity Framework Provider. This NuGet package bridges the gap between WCF Data Services 5.6.0 and Entity Framework 6+. We were able to build this provider as an out-of-band provider (that is, a provider that ships apart from the core WCF DS stack) because of the public provider work we did recently.

Upgrading an existing OData service to EF 6

If you are upgrading an existing OData service to Entity Framework 6 or greater, you will need to do a couple of things:

Install the WCF Data Services Entity Framework Provider NuGet package. Since this package has a dependency on WCF Data Services 5.6.0 and Entity Framework 6 or greater, some of the other NuGet packages in your project may be upgraded as well.
Replace the base type of your DataService. For EF 5 or below, your data service should inherit from DataService where T is a DbContext or ObjectContext. For EF 6 or greater, your data service should inherit from EntityFrameworkDataService where T is a DbContext. See What’s the difference between DataService and EntityFrameworkDataService below for more details.

Creating a new OData service with EF 6

If you are creating a new OData service and would like to use Entity Framework 6 or greater, you will need to follow similar steps:

Create your new project. I typically use an ASP.NET Empty Web Application for this, but you can use whatever you want. Note that if you do use the empty template, you may need to create an App_Data folder for Entity Framework to work properly with LocalDB.
Install the WCF Data Services Entity Framework Provider NuGet package. Since this package has a dependency on WCF Data Services 5.6.0 and Entity Framework 6 or greater, some of the other NuGet packages in your project may be upgraded as well.
Add a new WCF Data Service. It’s best if you ensure that your tooling is up-to-date as we occasionally fix bugs in the item template. Our latest tooling installer was released with WCF DS 5.6.0. It can be downloaded here.
Replace the base type of the DataService that was generated by the item template. For EF 6 or greater, your data service should inherit from EntityFrameworkDataService where T is a DbContext. See What’s the difference between DataService and EntityFrameworkDataService below for more details.

What’s the difference between DataService and EntityFrameworkDataService?

Historically the WCF DS stack required all WCF DS-based OData services to inherit from DataService. Internally, the data service would determine whether the service should use the in-box EF provider, the in-box Reflection provider, or a custom provider. When we added support for EF 6, we utilized the new public provider functionality to allow the provider to ship separately. This will allow us, for instance, to use WCF DS 5.6.0 with either EF 5, 6, or some future version. However, the new public provider functionality comes with a little bit of code you need to write. Since that code should be the same for every default EF 6 WCF DS provider, we went ahead and included a class that does this for you. EntityFrameworkDataService inherits from DataService and implements all the code you would need to implement otherwise. By shipping this additional class, we literally made the upgrade process as simple as changing the base type of your service.

Feedback please

We are heads down on getting our stacks updated to support OData v4, so we’ve had very limited resources to focus on testing this provider. We have a few automated tests and have tried a number of ad-hoc tests. That said, our coverage could be better so… we’re going to rely on you, our dear customer, to provide feedback of whether or not this provider works in your situation. If we don’t hear anything back, we’ll go ahead and release the provider in a week or so.

Thanks,
The OData Team

WCF Data Services 5.6.0 Release

dpblogs — Mon, 26 Aug 2013 21:59:35 GMT

Recently we released updated NuGet packages for WCF Data Services 5.6.0. You will need the updated tooling (released today) to use the portable libraries feature mentioned below with code gen.

What is in the release:

Visual Studio 2013 Support

The WCF DS 5.6.0 tooling installer has support for Visual Studio 2013. If you are using Visual Studio 2013 and would like to consume OData services, you can use this tooling installer to get Add Service Reference support for OData. Should you need to use one of our prior runtimes, you can still do so using the normal NuGet package management commands (you will need to uninstall the installed WCF DS NuGet packages and install the older WCF DS NuGet packages).

Portable Libraries

All of our client-side libraries now have portable library support. This means that you can now use the new JSON format in Windows Phone and Windows Store apps. The core libraries have portable library support for .NET 4.0, Silverlight 5, Windows Phone 8 and Windows Store apps. The WCF DS client has portable library support for .NET 4.5, Silverlight 5, Windows Phone 8 and Windows Store apps. Please note that this version of the client does not have tombstoning, so if you need that feature for Windows Phone apps you will need to continue using the Windows Phone-specific tooling.

URI Parser Integration

The URI parser is now integrated into the WCF Data Services server bits, which means that the URI parser is capable of parsing any URL supported in WCF DS. We have also added support for parsing functions in the URI Parser.

Public Provider Improvements - Reverted

In the 5.5.0 release we started working on making our providers public. In this release we hoped to make it possible to override the behavior of included providers with respect to properties that don’t have native support in OData v3, for instance enum and spatial properties. Unfortunately we ran into some non-trivial bugs with $select and $orderby and needed to cut the feature for this release.

Public Transport Layer

In the 5.4.0 release we added the concept of a request and response pipeline to WCF Data Service Client. In this release we have made it possible for developers to directly handle the request and response streams themselves. This was built on top of ODataLib's IODataRequestMessage and IODataResponseMessage framework that specifies how requests and responses are sent and recieved. With this addition developers are able to tweak the request and response streams or even completely replace the HTTP layer if they so desire. We are working on a blog post and sample documenting how to use this functionality.

Breaking Changes

In this release we took a couple of breaking changes. As these bugs are tremendously unlikely to affect anyone, we opted not to increment the major version number but we wanted everyone to be aware of what they were:

Developers using the reading/writing pipeline must write to Entry rather than Entity on the WritingEntryArgs
Developers should no longer expect to be able to modify the navigation property source in OnNavigationLinkStarting and OnNavigationLinkEnding
Developers making use of the DisablePrimitiveTypeConversion knob may see a minor change in their JSON payloads; the knob previously only worked for the ATOM format

Bug Fixes

Fixes a performance issue with models that have lots of navigation properties
Fixes a performance issue with the new JSON format when creating or deleting items
Fixes a bug where DisablePrimitiveTypeConversion would cause property type annotations to be ignored in the new JSON format
Fixes a bug where LoadProperty does not remove elements from a collection after deleting a link
Fixes an issue where the URI Parser would not properly bind an action to a collection of entities
Improves some error messages

Known Issues

The NuGet runtime in Visual Studio needs to be 2.0+ for Add Service Reference to work properly. If you are having issues with Add Service Reference in Visual Studio 2012, please ensure that NuGet is up-to-date.

Using the new client hooks in WCF Data Services Client

dpblogs — Fri, 26 Jul 2013 11:49:02 GMT

What are the Request and Response Pipeline configurations in WCF Data Services Client?In WCF Data Services 5.4 we added a new pattern to allow developers to hook into the client request and response pipelines. In the server, we have long had the concept of a processing pipeline. Developers can use the processing pipeline event to tweak how the server processes requests and responses. This concept has now been added to the client (though not as an event). The feature is exposed through the `Configurations` property on the `DataServiceContext`. On `Configurations` there are two properties, called `ResponsePipeline` and `RequestPipeline`. The `ResponsePipeline` contains configuration callbacks that influence reading to OData and materializing the results to CLR objects. The `RequestPipeline` contains configuration callbacks that influence the writing of CLR objects to the wire. Developers can then build on top of the new public API and compose higher level functionality. The explanation might be a bit abstract so let’s move look at a real world example. The code below will document how to remove a property that is unnecessary on the client or that causes materialization issues. Previously this was difficult to do, and impossible if the payload was returning the newer JSON format, but this scenario is now trivially possible with the new hooks. Below is a code snippet to remove a specific property: This code is using the `OnEntryRead` response configuration method to remove the property. Behind the scenes what is happening is the `Microsoft.Data.ODataReader` calls `reader.Read()`. As it reads though the items, depending on the `ODataItem` type a call will be made to the all configuration callbacks of that type that are registered. A couple notes about this code:
Since `ODataEntry.Properties` is an `IEnumerable` and not an `ICollection`, we need to replace the entire `IEnumerable` instead of just calling `ODataEntries.Properties.Remove()`.

`ResolveType` is used here to use the `TypeName` and get the `EntityType`, typically for a code generated `DataServiceContext` this method is automatically hooked up but if you are using a `DataServiceContext` directly then delegate code will need to be written.
What if this scenario has to occur for other properties on the same type or properties on a different type? Let’s make some changes to make this code a bit more reusable. Extension method for removing a property from an `ODataEntry`: Extension method for removing a property from the `ODataEntry` on the selected type: And now finally the code that the developer would write to invoke the method above and set the configuration up: The original code is now broken down and is more reusable. Developers can use the `RemoveProperties` extension above to remove any property from a type that is in the `ODataEntry` payload. These extension methods can also be chained together. The example above shows how to use `OnEntryEnded`, but there are a number of other callbacks that can be used. Here is a complete list of configuration callbacks on the response pipeline: All of the configuration callbacks above with the exception of `OnEntityMaterialized` and `OnMessageReaderSettingsCreate` are called when the `ODataReader` is reading through the feed or entry. The `OnMessageReaderSettingsCreate` callback is called just prior to when the `ODataMessageWriter` is created and before any of the other callbacks are called. The `OnEntityMaterialized` is called after a new entity has been converted from the given `ODataEntry`. The callback allows developers to apply any fix-ups to an entity after it was converted. Now let’s move on to a sample where we use a configuration on the `RequestPipeline` to skip writing a property to the wire. Below is an example of an extension method that can remove the specified properties before it is written out: As you can see we are following the same pattern as the extension method we wrote to `RemoveProperties` for the `ResponsePipeline`. In comparison to this extension method this function doesn’t require the type resolving func, so it’s a bit simpler. The type information is specified on the `OnEntryEnding` args in the `Entity` property. Again this example only touches on `ODataEntryEnding`. Below is the complete list of configuration callbacks that can be used: With the exception of `OnMessageWriterSettingsCreated`, the other configuration callbacks are called when the `ODataWriter` is writing information to the wire. In conclusion, the request and response pipelines offer ways to configure the how payloads are read and written to the wire. Let us know any other questions you might have to leverage this feature.
Chris Robinson – OData Team

WCF Data Services 5.6.0 Alpha

dpblogs — Fri, 28 Jun 2013 16:53:09 GMT

Today we are releasing updated NuGet packages and tooling for WCF Data Services 5.6.0. This is an alpha release and as such we have both features to finish as well as quality to fine-tune before we release the final version.

You will need the updated tooling to use the portable libraries feature mentioned below. The tooling installer is available from the download center.

What is in the release:

Visual Studio 2013 Support

The WCF DS 5.6.0 tooling installer has support for Visual Studio 2013. If you are using the Visual Studio 2013 Preview and would like to consume OData services, you can use this tooling installer to get Add Service Reference support for OData. Should you need to use one of our prior runtimes, you can still do so using the normal NuGet package management commands (you will need to uninstall the installed WCF DS NuGet packages and install the older WCF DS NuGet packages).

Portable Libraries

URI Parser Integration

The URI parser is now integrated into the WCF Data Services server bits, which means that the URI parser is capable of parsing any URL supported in WCF DS. We are currently still working on parsing functions, with those areas of the code base expected to be finalized by RTW.

Public Provider Improvements

In the 5.5.0 release we started working on making our providers public. In this release we have made it possible to override the behavior of included providers with respect to properties that don’t have native support in OData v3. Specifically, you can now create a public provider that inherits from the Entity Framework provider and override a method to make enum and spatial properties work better with WCF Data Services. We have also done some internal refactoring such that we can ship our internal providers in separate NuGet packages. We hope to be able to ship an EF6 provider soon.

Known Issues

With any alpha, there will be known issues. Here are a few things you might run into:

We ran into an issue with a build of Visual Studio that didn’t have the NuGet Package Manager installed. If you’re having problems with Add Service Reference, please verify that you have a version of the NuGet Package Manager and that it is up-to-date.
We ran into an issue with build errors referencing resource assemblies on Windows Store apps. A second build will make these errors go away.

We want feedback!

This is a very early alpha (we think the final release will happen around the start of August), but we really need your feedback now, especially in regards to the portable library support. Does it work as expected? Can you target what you want to target? Please leave your comments below or e-mail me at mastaffo@microsoft.com. Thank you!

WCF Data Services 5.5.0 Release

dpblogs — Thu, 30 May 2013 08:32:37 GMT

WCF Data Services 5.5.0 has officially been released! The 5.5.0 release will be another NuGet-only release as we did not make any updates to the Visual Studio tooling. The last tooling update was version 5.3.0. Services created using this version of tooling should update the runtime binaries to 5.5.0 with NuGet.

What is in the release:

This release has two primary features: 1) significant enhancements to the URI parser and 2) public data source providers. In addition to the primary features, there are two secondary enhancements and over 40 bug fixes included with this release.

URI Parser

In the 5.2.0 release ODataLib provided a way to parse $filter and $orderby expressions into a metadata-bound abstract syntax tree (AST). In the 5.5.0 release we have updated the URI parser with support for most OData URIs. The newly introduced support for parsing $select and $expand is particularly notable. With the 5.5.0 release the URI Parser is mostly done. Future releases will focus on higher-order functions to further improve the developer experience.

Public Data Source Providers

In this release we have made the Entity Framework and Reflection data source providers public. This gives more control to service writers. There is more work planned in the future but the work we’ve completed allows some advanced scenarios which were not possible earlier. For example, a service writer can now make use of the Entity Framework query-caching feature by intercepting the request and parameterizing the LINQ query before handing it off to Entity Framework. (Note that parameterizing a LINQ query is not the same as parameterizing a SQL query; EF always does the latter and therefore there is no security implications to failing to parameterize a LINQ to Entities query, the only impact is performance related.)

While the potential unlocked with this release is limited, this is the first move in a direction which will unlock many previously unachievable scenarios with the built in providers.

Performance Improvements

We constantly strive to improve performance and reliability with every release. In this release, we have increased performance by double digit percentages for service authors that want to respond with JSON but are not able to (or don't want to) provide a data model to ODataLib.

Improved Developer Experience

In this release we have caught up with some missing IntelliSense guidance and we are uploading symbols and source to SymbolSource.org. We will upload symbols for many of our past releases as well.

Bug Fixes

In addition to the features above, we have included fixes for the following notable bugs:

Fixes an issue where the reflection provider would not work properly if the generic parameter provided to DataService was an interface
Fixes an issue where some system headers were not being set when a client called BuildingRequest
Fixes an issue where setting InstanceContextMode to Single on DataService would result in cache result being returned for subsequent requests
Fixes an issue where ODataLib would sometimes allow null to be written whether or not the expected type was nullable
Fixes a regression in 5.4 where ODataLib started writing unnecessary type information in certain instances
Fixes an issue where the WCF DS client would sometimes not dispose the response if the response had no content
Improves a number of errors and error messages

WCF Data Services 5.5.0 Prerelease

dpblogs — Wed, 15 May 2013 17:30:14 GMT

It’s that time again: yesterday we uploaded an RC for the upcoming 5.5.0 release. The 5.5.0 release will be another NuGet-only release.

What is in the release:

This release has two primary features: 1) significant enhancements to the URI parser and 2) public data source providers.

URI Parser

Public Data Source Providers

While the potential unlocked with this release is limited, this is the first move in a direction which will unlock many previously unachievable scenarios with the built in providers.

WCF Data Services 5.4.0 Release

dpblogs — Tue, 02 Apr 2013 15:16:15 GMT

Today we are releasing version 5.4.0 of WCF Data Services. As mentioned in the prerelease post, this release will be NuGet packages only. That means that we are not releasing an updated executable to the download center. If you create a new WCF Data Service or add a reference to an OData service, you should follow the standard procedure for making sure your NuGet packages are up-to-date. (Note that this is standard usage of NuGet, but it may be new to some WCF Data Services developers.)

Samples

If you haven’t noticed, we’ve been releasing a lot more frequently than we used to. As we adopted this rapid cadence, our documentation has fallen somewhat behind and we recognize that makes it hard for you to try out the new features. We do intend to release some samples demonstrating how to use the features below but we need a few more days to pull those samples together and did not want to delay the release. Once we get some samples together we will update this blog post (or perhaps add another blog post if we need more commentary than a gist can convey).

What is in the release:

Client deserialization/serialization hooks

We have a number of investments planned in the “request pipeline” area. In 5.4.0 we have a very big set of hooks for reaching into and modifying data as it is being read from or written to the wire format. These hooks provide extensibility points that enable a number of different scenarios such as modifying wire types, property names, and more.

Instance annotations on atom payloads

As promised in the 5.3.0 release notes, we now support instance annotations on Atom payloads. Instance annotations are an extensibility feature in OData feeds that allow OData requests and responses to be marked up with annotations that target feeds, single entities (entries), properties, etc. We do still have some more work to do in this area, such as the ability to annotate properties.

Client consumption of instance annotations

Also in this release, we have added APIs to the client to enable the reading of instance annotations on the wire. These APIs make use of the new deserialization/serialization pipelines on the client (see above). This API surface includes the ability to indicate which instance annotations the client cares about via the Prefer header. This will streamline the responses from OData services that honor the odata.include-annotations preference.

Simplified transition between Atom and JSON formats

In this release we have bundled a few less-noticeable features that should simplify the transition between the Atom and (the new) JSON format. (See also the bug fixes below on type resolver fixes.)

Bug fixes

In addition to the features above, we have included fixes for the following notable bugs:

Fixes an issue where reading a collection of complex values would fail if the new JSON format was used and a type resolver was not provided
Fixes an issue where ODataLib was not escaping literal values in IDs and edit links
Fixes an issue where requesting the service document with application/json;odata=nometadata would fail
Fixes an issue where using the new JSON format without a type resolver would create issues with derived types
(Usability bug) Makes it easier to track the current item in ODataLib in many situations
Fixes an issue where the LINQ provider on the client would produce $filter instead of a key expression for derived types with composite keys
(Usability bug) Fixes an issue where the inability to set EntityState and ETag values forced people to detach and attach entities for some operations
Fixes an issue where some headers required a case-sensitive match on the WCF DS client
Fixes an issue where 304 responses were sending back more headers than appropriate per the HTTP spec
Fixes an issue where a request for the new JSON format could result in an error that used the Atom format
Fixes an issue where it was possible to write an annotation value that was invalid according to the term
Fixes an issue where PATCH requests for OData v1/v2 payloads would return a 500 error rather than 405

We want your feedback

We always appreciate your comments on the blog posts, forums, Twitterverse and e-mail (mastaffo@microsoft.com). We do take your feedback seriously and prioritize accordingly. We are still early in the planning stages for 5.5.0 and 6.0.0, so feedback now will help us shape those releases.

WCF Data Services 5.4.0 Prerelease

dpblogs — Tue, 26 Mar 2013 10:53:57 GMT

Recently we uploaded an RC for our upcoming 5.4.0 release. This release will be NuGet packages only.

What is in the release:

Client deserialization/serialization hooks

Instance annotations on atom payloads

Client consumption of instance annotations

Simplified transition between Atom and JSON formats

In this release we have bundled a few less-noticeable features that should simplify the transition between the Atom and (the new) JSON format. (See also the bug fixes below on type resolver fixes.)

Bug fixes

In addition to the features above, we have included fixes for the following notable bugs:

Fixes an issue where reading a collection of complex values would fail if the new JSON format was used and a type resolver was not provided
Fixes an issue where ODataLib was not escaping literal values in IDs and edit links
Fixes an issue where requesting the service document with application/json;odata=nometadata would fail
Fixes an issue where using the new JSON format without a type resolver would create issues with derived types
(Usability bug) Makes it easier to track the current item in ODataLib in many situations
Fixes an issue where the LINQ provider on the client would produce $filter instead of a key expression for derived types with composite keys
(Usability bug) Fixes an issue where the inability to set EntityState and ETag values forced people to detach and attach entities for some operations
Fixes an issue where some headers required a case-sensitive match on the WCF DS client
Fixes an issue where 304 responses were sending back more headers than appropriate per the HTTP spec
Fixes an issue where a request for the new JSON format could result in an error that used the Atom format
Fixes an issue where it was possible to write an annotation value that was invalid according to the term
Fixes an issue where PATCH requests for OData v1/v2 payloads would return a 500 error rather than 405

What to expect over the next six months:

We will blog about this in more detail soon, but we have multiple releases planned that have some level of overlap. We should be publishing a 5.5.0 alpha soon (with additional URI parser functionality for Web API’s OData stack) and in a couple of months you should see a very early alpha of 6.0.0. We’re not ready to say much about 6.0.0 yet other than the fact that it will support OData v4 and will probably have some breaking changes, so we want to get it out there as soon as possible because…

We want your feedback

We always appreciate your comments on the blog posts, forums, Twitterverse and e-mail (mastaffo@microsoft.com). We do take your feedback seriously and prioritize accordingly. We would encourage you strongly to start pulling down these early bits, testing with your existing services, and telling us where things break, where we’ve gone too far, and where we haven’t gone far enough.