The Azure platform currently does not directly support highly available, geographically distributed services. Customers can deploy services to multiple data centers, but the platform does nothing to help them integrate the separate deployments, or to achieve high availability with the resulting composite.

Geographically distributed high availability is essentially continuous disaster recovery. Each tenant’s data is replicated to multiple physical service instances. Load is distributed dynamically across those service instances to provide an optimal user experience. The two primary challenges are:

• integrating physical service instances to create a single virtual service, and
• minimizing data inconsistencies as client requests are routed among the different physical service instances.

Creating a Virtual Service

To create a single virtual service from a collection of physical service instances, clients of the service must use logical URIs that can be bound to different physical URIs. Using a global load balancer (GLB), a variety of policies can then be used to select a physical service instance to service a given client request. For example, many GLBs support the following types of routing:

• ratio, which routes some fraction of client requests to each physical service instance,
• geographical, which routes client requests to physical service instances in a given geography,
• proximity, which routes client requests to the nearest physical service instances, and
• round-robin, which routes client requests to physical service instances in a round-robin fashion.

Geographical routing is usually based on the requested domain name, but some services may require routing decisions to be based on other components of the request, such as path segments, query string parameters, or payload contents. In these cases, the service may use proximity routing to route requests to the nearest physical instance, which then proxies them to appropriate locations using a custom routing strategy.

Minimizing Data Inconsistencies

If the service data changes frequently, clients may see data inconsistencies, such as recently inserted data disappearing and recently deleted data reappearing, when their requests are routed to different service instances. One way to mitigate but not completely eliminate the inconsistencies is to affinitize clients to specific service instances, and to change affinities infrequently. For some data, this level of consistency may be acceptable. For other data, however, the service may have to use synchronous replication, which was described briefly in the previous post in this series, or to block writes for a client until all outstanding changes have propagated before routing their requests to a different service instance.

Disaster Recovery

Disaster recovery typically involves a pair of processes called fail over and fail back. Both involve the use of redundant service components. A fail over moves service load from one set of components to another. In the case of an actual outage, a fail over is performed because the original components have failed. However, planned fail overs may be performed for testing, upgrade, or other purposes. Because most issues within a single data center are addressed by the high availability features of SQL Azure and Windows Azure, fail over will generally be used to move load between data centers. A fail back restores the original service load distribution.

For stateful services, these processes target both compute and storage components. For stateless services, only compute components are targeted. Compute fail overs reroute client requests to different compute components, usually by changing DNS CNAME records to bind a given logical URI to different physical URIs. Storage fail overs cause service requests to be served from different storage components, typically by changing connection strings used internally by the service.

Both processes require redundant service components to be available and ready to receive the service load being moved. Satisfying this requirement can be challenging. Provisioning redundant components in advance may incur additional cost, for example, but provisioning them on demand during a fail over may incur unacceptable latency. There is also the question of how to make sure the redundant components are correctly deployed and configured, so that the service continues to behave correctly following the move.

Deployment Units

One way to simplify the task of providing redundant service components is to use deployment units. A deployment unit is a set of resources provisioned in advance and configured to support the execution of an instance of a given service. Conceptually, it's like a template for a service instance. For example, if a given service had two web roles, three worker roles and three databases before loading production data, then a deployment unit for that service might consist of five hosted services with deployed exectuables, and one SQL Azure logical sever containing three databases configured with appropriate schema and initial data.

Configuring a scale unit involves a variety of tasks, such as provisioning certificates, configuring DNS, setting configuration parameters in application and library configuration files, building executables from the configured sources, setting configuration parameters in deployment configuration files, building deployable packages from the executables and deployment settings, deploying the packages to the hosted services, and running SQL scripts to configure the databases.

The process should be automated, since the number of configuration parameters may be too large to reliably set by hand, and some parts of it may have to run in Azure, since some of the configuration parameters, such as SQL Azure connection strings and other credentials may have to be sourced from secure locations rather than developer machines, or by operations staff, rather than developers. Of course, the ability to rapidly provision configured deployment units provides significant value beyond disaster recovery. Deployment units can be used for development, test, staging, private releases, a/b testing and upgrade scenarios.

The key to success with deployment units is systematic organization. When provisioning a deployment unit is a merely matter of running a tool, it doesn't take long before the number of hosted services and databases gets out of hand, and it can quickly become challenging to track which deployment units have been provisioned, which resources each one is using, and what purpose each one is serving. A key step toward service maturity is therefore building a deployment unit management system, with systematic naming of deployment units and the resources they contain. Typically, the DNS names of the hosted services will reflect the organizational structure, identifying the name of the component, the name of the deployment unit, and the name of the data center in which it resides, for example, making it easy to identify components programmatically for fail over and fail back operations.

Compute Redundancy

With Windows Azure, a cost effective compute redundancy strategy is to operate redundant service instances in two or more data centers, and to fail over by moving load from one instance to another. Because the platform varies the number of worker roles running as the load varies, this approach can be used to add capacity without incurring additional cost because the additional worker roles will not run until the load actually moves. This is called an active/active configuration, as opposed to an active/passive configuration, where the service instance in one data center does not carry any load until a fail over occurs.

Storage Redundancy

Storage fail overs are more complex than compute fail overs because of the challenge of maintaining consistency among copies of the data when the data is changing. Because of the time required to copy data between locations, there is a possibility of data loss if a location fails or is isolated from other locations while holding data that has not yet been replicated. On the other hand, forcing synchronous replication to multiple locations when the data is written can result in poor performance due to added latency. In the extreme, when too few locations are available, forcing synchronous replication causes writes to block until additional locations become available.

This trade-off, known as the CAP theorem[1], is the motivation behind the many technologies for managing geographically distributed data used on the Internet. The two major categories of distributed data technology are as follows:

• Fully consistent stores, such as Google’s MegaStore[2], use synchronous replication based on distributed algorithms like PAXOS, to write to a quorum of replicas. Synchronous replication avoids inconsistency, but is generally too slow to be used for high throughput, low latency applications. Also, to prevent loss of service and resynchronization issues, a quorum of three or more locations is typically required. Currently, on the Azure platform, there are only two data centers in every region, making it hard to build quorums with reasonable performance.
• Eventually consistent stores, such as Amazon’s Dynamo[3], use asynchronous replication to copy data to replicas after it is written to a primary location, and provide mechanisms for dealing with data loss and inconsistency. Without an eventually consistent store, service developers must deal with inconsistencies in business logic or use conflict resolution mechanisms to detect and resolve them.

Asynchronous Replication

In a multi-master architecture, writes can occur in multiple locations. Continuous conflict resolution is required to resolve inconsistencies. In a single-master architecture, writes can occur in one location. However, conflict resolution is still required if the master is allowed to change, as it must in the case of a fail over, and must be performed after the master changes to build a consistent view of service state. An alternative to conflict resolution is to flatten and rebuild from scratch any replicas holding a later copy of the data than the new master.

On the Azure platform, one of the best bets for achieving asynchronous replication is the Sync Framework, which handles the complexities of virtual clocks, knowledge vectors and difference computation, while providing enormous flexibility through the use of confict resolution polcies, application level conflict resolution, and a provider architecture for data and metadata sources. The SQL Azure Data Sync Service, now in CTP, hosts the Sync Framework on Windows Azure to provide synchronization as a service. Using the service, instead of rolling your own solution with the Sync Framework means living with some limitations, but also offloads the work of building, maintaining and operating an asynchronous replication service. Check this blog for more on these two technologis and the trade off between them in upcoming posts.

In the extreme, asynchronous replication becomes backup and restore, which offers a simple form of storage redundancy, if customers are willing to accept an RPO defined by the window between backups. If a service cannot block writes for extended periods of time, then data may change while a backup is taken, and the contents of different storage components captured by the backup may reflect different views of the state of the service. In other words, backups may also contain inconsistencies, and recovery processing is therefore generally required following a restore.

Some services use data mastered by other services, and update their copies when data changes at the source. In these cases, recovery processing involves replaying changes from the source to ensure that the copies are up to date. All updates must be idempotent, so that the data to be replayed can overlap with the data already stored in the copies to ensure that no updates are missed.

Cross Data Center Operation

While compute and storage fail overs generally occur in tandem, so that the entire service fails over from one data center to another, there are situations where it may make sense to fail over one but not the other. Since storage fail over may cause data loss, for example, it may make sense to fail over only the compute components, allowing them to access the primary storage facilities, assuming they’re still accessible.

Following a compute only fail over, compute and storage components may be located in two different data centers. Storage access calls will therefore traverse the backbone network between data centers, instead of staying within a single data center, causing performance degradation. One the Azure platform, round trip times between data centers within a region are about 6 times higher than they are within a data center. Between regions, the performance penalty grows to a factor of 30. Services will also incur data egress costs when running across data centers. Cross data center operation really only makes sense for short lived outages, and/or for services that only move small amounts of data.

Windows Azure

Windows Azure uses a combination of resource management, elasticity, load balancing, and partitioning to enable high availability within a single data center. The service developer must do some additional work to benefit from these features.

Resource Management

All services hosted by Windows Azure are collections of web, worker and/or virtual machine roles. One or more instances of a given role can run concurrently. The number of instances is determined by configuration. Windows Azure uses Fabric Controllers (FCs) to monitor and manage role instances. FCs detect and respond to both software and hardware failure automatically.

• Every role instance runs in its own VM and communicates with its FC through a guest agent (GA). The GA collects resource and node metrics, including VM usage, status, logs, resource usage, exceptions, and failure conditions. The FC queries the GA at configurable intervals, and reboots the VM if the GA fails to respond.
• In the event of hardware failure, the FC responsible for the failed node moves all affected role instances to a new hardware node and reconfigures the network to route traffic there. FCs use the same mechanisms to ensure the continuous availability of the services they provide.

Elasticity

The FC dynamically adjusts the number of worker role instances, up to the limit defined by the service through configuration, according to system load.

Load Balancing

All inbound traffic to a web role passes through a stateless load balancer, which distributes client requests among the role instances. Individual role instances do not have public IP addresses, and are not directly addressable from the Internet. Web roles are stateless, so that any client request can be routed to any role instance. A StatusCheck event is raised every 15 seconds.

Partitioning

FCs use two types of partitions: update domains and fault domains.

• An update domain is used to upgrade a service’s role instances in groups. For an in-place upgrade, the FC brings down all the instances in one upgrade domain, upgrades them, and then restarts them before moving to the next upgrade domain. This approach ensures that in the event of an upgrade failure, some instances will still be available to service requests.
• A fault domain represents potential points of hardware or network failure. For any role with more than one instance, the FC ensures that the instances are distributed across multiple fault domains, in order to prevent isolated hardware failures from disrupting service. All exposure to VM and cluster failure in Windows Azure is governed by fault domains.

According to the Windows Azure SLA^[1], Microsoft guarantees that when two or more web role instances are deployed to different fault and upgrade domains, they will have external connectivity at least 99.95% of the time. There is no way to control the number of fault domains, but Windows Azure allocates them and distributes role instances across them automatically. At least the first two instances of every role are placed in different fault and upgrade domains in order to ensure that any role with at least two instances will satisfy the SLA.

Implementation

The service developer must do some additional work to benefit from these features.

• To benefit from resource management, developers should ensure that all service roles are stateless, so that they can go down at any time without creating inconsistencies in the transient or persistent state of the service.
• To achieve elasticity, developers should configure each of their worker roles with the maximum number of instances sufficient to handle the largest expected load.
• To optimize load balancing, developers should use the StatusCheck event when a role instance reaches capacity to indicate that it is busy and that it should be temporarily removed from the load-balancer rotation.
• To achieve effective partitioning, developers should configure at least two instances of every role, and at least two upgrade domains for every service.

The requirement to keep roles stateless deserves further comment. It implies, for example, that all related rows in a SQL Azure database should be changed in a single transaction, if possible. For example, instead of inserting a parent in one transaction, and then its children in another, the code should insert both the parent and the children in the same transaction, so that if it goes down after writing just one of the row sets, the data will be left in a consistent state.

Of course, it is not always possible to make all changes in a single transaction. Special care must be taken to ensure that role failures do not cause problems when they interrupt long running operations that span two or more updates to the persistent state of the service.

For example, in a service that partitions data across multiple stores, if a worker role goes down while relocating a shard, the relocation of the shard may not complete, or may be repeated from its inception by a different worker role, potentially causing orphaned data or data corruption. To prevent problems, long running operations must be idempotent (i.e., repeatable without side effect) and/or incrementally restartable (i.e., able to continue from the most recent point of failure).

• To be idempotent, a long running operation should have the same effect no matter how many times it is executed, even when it is interrupted during execution.
• To be incrementally restartable, a long running operation should consist of a sequence of smaller atomic operations, and it should record its progress in durable storage, so that each subsequent invocation picks up where its predecessor stopped.

Finally, all long running operations should be invoked repeatedly until they succeed. For example, a provisioning operation might be placed in an Azure queue, and removed from the queue by a worker role only when it succeeds. Garbage collection may be needed to clean up data created by interrupted operations.

Common long running operations that create special challenges include provisioning, deprovisioning, rolling upgrade, data replication, restoring backups and garbage collection.

SQL Azure

SQL Azure uses a combination of replication and resource management to provide high availability within a single data center. Services benefit from these features just by using SQL Azure. No additional work is required by the service developer.

Replication

SQL Azure exposes logical rather than physical servers. A logical server is assigned to a single tenant, and may span multiple physical servers. Databases in the same logical server may therefore reside in different SQL Server instances.

Every database has three replicas: one primary and two secondaries. All reads and writes go to the primary, and all writes are replicated asynchronously to the secondaries. Also, every transaction commit requires a quorum, where the primary and at least one of the secondaries must confirm that the log records are written before the transaction can be considered committed. Most production data centers have hundreds of SQL Server instances, so it is unlikely that any two databases with primary replicas on the same machine will have secondary replicas that also share a machine.

Resource Management

Like Windows Azure, SQL Azure uses a fabric to manage resources. However, instead of a fabric controller, it uses a ring topology to detect failures. Every replica in a cluster has two neighbors, and is responsible for detecting when they go down. When a replica goes down, its neighbors trigger a Reconfiguration Agent (RA) to recreate it on another machine. Engine throttling is provided to ensure that a logical server does not use too many resources on a machine, or exceed the machine’s physical limits.

Failure Detection

Failures are expected during normal operation due to the characteristics of the cloud environment.

They can be detected by monitoring the service. Reactive monitoring observes that the service is not meeting Service Level Objectives (SLOs), while proactive monitoring observes conditions which indicate that the service may fail, such as rapidly increasing response times or memory consumption.

Failure detection is always relative to some observer. For example, in a partial network outage, the service may be available to one observer but not another. If the network outage blocks all external access to the service, it will appear to be unavailable to all observers, even if it is still operating correctly. A service therefore cannot reliably detect its own failure.

Failure Response

Failure response involves two related but different objectives: high availability and disaster recovery. Both boil down to satisfying SLOs: high availability is about maintaining acceptable latencies, while disaster recovery is about recovering from outages in an acceptable amount of time with no more than an acceptable level of data loss.

High Availability

Availability is generally measured in terms of responsiveness. A service is considered responsive if it responds to requests within a specified amount of time. Availability is measured as the amount of time the service is responsive in some time window divided by the length of the window, expressed as a percentage. Most services typically offer at least 99.9% availability, which amounts to 43.2 minutes of downtime or less per month.

Availability SLOs may not offer an accurate indication of the actual availability of a service in practice, because of how responsiveness is defined. For example, a common way to define availability is to say that a service is considered responsive if at least one request succeeds in every 5 minute interval. By this definition, the service can be unresponsive for 4 minutes and 59 seconds in every 5 minute interval, providing an actual availability of 0.3%, while satisfying a stated SLO of 99.9% availability.

Disaster Recovery

Disaster recovery is governed by SLOs that become relevant when the service becomes unavailable.

• A Recovery Time Objective (RTO) is the amount of time that can elapse before normal operation is restored following a loss of availability. It typically includes the time required to detect the failure, and the time required to either restore the affected components of the service, or to fail over to new components if the affected components cannot be restored fast enough.
• A Recovery Point Objective (RPO) is a point in time to which the service must recover data committed by the user prior to the failure. It should be less than or equal to what users consider an "acceptable loss" in the event of a disaster.

Windows Azure and SQL Azure currently provide high availability within a data center, but they don’t provide any protection when data centers go down or offline. Customers can deploy services to multiple data centers to work around this limitation, but the platform doesn’t help them integrate the separate deployments to form highly available, geographically distributed services. This is the first of several blog posts on this topic taken from a paper that I recently wrote with Dima Sonkin and Erik Wahlstrom.

The posts will provide an introduction to business continuity, describe Azure platform features enabling business continuity, and describe how to achieve business continuity with the existing features.

Hello, World!

Before we get to harder problems, let’s make sure we have the basics right.

If you’re doing Silverlight client development, then you probably already know that any service accessed by a Silverlight client must either be hosted from the same Internet zone, domain and scheme as the client, or have a cross domain security policy file in place. What you may not know is that the AppFabric Team recently released a version of ACS to AppFabric Labs with a cross domain security policy file, allowing access from any domain over either http or https to any path below the base URI of any STS endpoint. You should be able to obtain and inspect the file using your web browser.

You probably also know that you need to specify client HTTP handling for REST endpoints like ACS. What you may not know is that for cross domain access to data using WCF Data Services (formerly ADO.NET Data Services), you’ll either need to move to Silverlight 4, or use ADO.NET Data Services for Silverlight 3 Update CTP3 (thanks to Mike Flasko and Pablo Castro for that pointer).

Red Herrings

Now, assuming you’re using the right networking stack, and hitting the AppFabric Labs release of ACS, instead of production, you may see Not Found (404) WebExceptions raised when you try to request a token.

These exceptions are often misleading. If you read the response body, you may find that the problem is reported as an XML formatted “request unauthorized” error in some cases, or as an HTML formatted “HTTP method not allowed” in other cases.

The “request unauthorized” error is expected behavior from ACS when the credentials needed to authenticate with the service are missing. The only surprise here is that it is reported by the Silverlight as Not Found (404), instead of the expected Unauthorized (401).

The “HTTP method not allowed” error is more perplexing. When you see it, you may also notice that the web request identified in the exception is not the one you used to do the POST to request the token, but rather a GET with different headers targeting the same URI.

The Case of the Missing Slash

The “HTTP method not allowed” error can be hard to track down, since it occurs inside the HTTPS tunnel, where it can’t be directly observed. Aaron Oneal provided the critical clue: when a POST results in a redirect, with auto-redirect enabled, Silverlight strips the request headers and issues a GET to the redirect address, not a POST. User code in the client sees one operation, while the server sees two.

The clue was enough for Cyrus Harvesf to understand the problem. When the endpoint address used to access ACS doesn’t contain a trailing slash, ACS redirects the caller to the same address with a trailing slash. When the Silverlight networking stack responds with a GET to the redirect address, the GET operation fails because it is not supported by the endpoint.

Why didn’t we notice this problem before?

Because the .NET framework silently replays POSTs in response to redirects, so omitting the trailing slash didn’t block any scenarios until now, with the enablement of Silverlight access in AppFabric Labs. Even more insidious is that in the case of POSTs to ACS without the trailing slash, the redirect and silent replay take place inside the HTTPS tunnel, where no one can really see what’s going on, even with any of the popular network monitoring tools (e.g., NetMon, WireShark, Fiddler, etc.).

Which behavior is correct?

Marco Matos notes that the Silverlight behavior is designed to prevent cross site attacks in which request headers on a POST could be redirected to unknown domains. Clearly, this is preferable to the .NET framework behavior, which puts the client at risk. However, the HTTP specification says that in the case of a 301 or 303 response code, the user agent should not redirect the request without explicit permission from the user, unless the request is either a GET or HEAD, for precisely this reason. So, according to the specification, neither behavior is correct.

Down the Garden Path

Now, here’s the kicker. Most of us copy and paste generated URIs to make sure nothing is lost in translation. However, the AppFabric portal displays the addresses for the endpoints it provisions WITHOUT the trailing slashes required by ACS, thereby ensuring that most of us have it wrong. Quick… go add slashes to all your STS endpoint addresses.

How should ACS correct the problem?

It could fail outright when the trailing slash is missing, instead of redirecting the client. However, changing the behavior now would break existing clients, if they are using URIs without the trailing slashes.

A better approach would be to preserve the existing behavior in the next release, and to document the problem, with a warning that the behavior may change in later releases. This would give clients time to prepare for the change, by making sure their URIs have trailing slashes.

Catch 22

So, now we’re ready to rock and roll, right? Not quite.

There’s an important requirement in OAuth WRAP that can’t be satisfied using Silverlight 3 and earlier versions. The specification recommends that that clients place the SWT in the authorization header:

Clients always present an Access Token to access a Protected Resource.  Use of the Authorization header is RECOMMENDED, since HTTP implementations are aware that Authorization headers have special security properties and may require special treatment in caches and logs.  Protected Resources SHOULD take precautions to insure that Access Tokens are not inadvertently logged or captured.  In addition to the methods presented here, the Protected Resource MAY allow the Client to present the Access Token using any scheme agreed on by the Client and Protected Resource.

Unfortunately, the Authorization header can’t be set in versions prior to Silverlight 4 because it’s on the list of restricted headers, and it can only be set in Silverlight 4 when the cross domain security policy file contains an appropriate entry (e.g., an allow-from element with an http-request-headers attribute whose value contains the name of the Authorization header).

The best solution is to move to Silverlight 4. However, for clients that can’t make the move, an alternate header should be a workable solution, provided that the service supports it, and configures the cross domain security policy file to allow it, as described above.

For example, the OData Service checks both Authorization and X-Authorization. If it finds tokens in both locations, they must be identical. Also, the ACS management service checks Authorization and x-ms-Authorization. If it finds tokens in both locations, it returns an error to the client.

[Updated - October 5, 2010 - Revised text and URIs to refer to ACS production, since ACS Labs is now running ACS v2.]

Looking for a way to expose data in SQL Azure to web clients using simple REST based APIs? If so, life just got easier.

Over the last few weeks, I've been working with Mike Pizzo and David Robinson, building a portal and Silverlight clients for a new OData Service for SQL Azure. Together, the OData Portal and Service let you publish SQL Azure databases as OData services using simple configuration. (For more information about SQL Azure, OData or AppFabric Access Control, please see the glossary at the end of this post.)

Of course, you can still use WCF Data Services (formerly ADO.NET Data Services) to build a custom data service. But, if you just want to expose the data, without any business logic, then the OData Portal and Service provide a no code solution with web friendly security based on AppFabric Access Control (ACS).

Try it out using your web browser at this public data service endpoint.

Access Modes

The OData Service has two access modes: anonymous and authenticated. In both modes, it executes OData queries by impersonating a designated database user. You use the OData Portal to designate database users for both modes.

• Anonymous mode requires no credentials, as the name implies.
• Authenticated mode, by contrast, requires clients to pass credentials with their queries. The OData Service uses Simple Web Token (SWT), a lightweight, web friendly security token format. You can get a SWT to pass to it by querying an ACS based Security Token Service (STS) using OAuth WRAP, a lightweight, web friendly authorization protocol. Before going into more detail about how to get a SWT and pass it to the OData service, let's look at the OData Portal.

Using the Portal

To use the OData Portal, visit SQL Azure Labs, and navigate to the center tab in the top navigation bar, labeled "OData Service for SQL Azure". If you're not already signed in with a valid Windows Live ID, you'll be taken to a sign in page. When you've signed in, connect to a SQL Azure server by providing the server name (e.g., hqd7p8y6cy.database.windows.net), and the name and password for a login with access to the master catalog, and then clicking Connect.

When a connection is made, OData Service configuration data is displayed for the databases on the selected server. Use the drop down list to switch between the databases on the server.

To enable OData access to the currently selected database, check the box labeled "Enable OData". When OData access is enabled, database user mapping information is displayed at the bottom of the form.

• Use the drop down list labeled "Anonymous Access User" to select an anonymous access user. If an anonymous access user is selected, then all queries against the database presented without credentials will execute by impersonating that user. You can access the database as the anonymous user by clicking on the link provided at the bottom of the page. If no anonymous access user is selected, then the OData Service will not allow anonymous access to the database.
• Click the link labeled "Add User" to add a user for authenticated access. In the pop up panel, select the user from the drop down list. Leave the issuer name empty for simple authentication, or provide the name of a trusted Security Token Service (STS) for federated authentication. For example, to federate with another ACS based STS, provide the base URI for the STS endpoint displayed by the Windows Azure AppFabric Portal for the STS, as shown in the first line below.

• Click the "OK" button to complete the configuration process and dismiss the pop up panel. When one or more authenticated access users are added, the OData Service will impersonate them when appropriate credentials are presented. You can designate as many authenticated access users as you like. The OData Service will decide which one to impersonate for each query by inspecting the credentials presented with the query.

Building Clients

For authenticated access, clients pass SWTs to the OData Service. To get a SWT, just query a Security Token Service (STS) trusted by the OData Service. Currently, the only STS trusted by the OData Service is an ACS based STS located at https://odataacs.accesscontrol.windows.net/WRAPv0.9/. (Note that we are using production ACS, not AppFabric Labs, which is now running ACS v2.)

The query is a token request, as defined by the OAuth WRAP specification: an HTTP POST to the STS endpoint containing a URL encoded form with the following elements:

• The name of the protected resource to be accessed (the wrap_scope parameter). In the case of the OData Service, this is the base URI for the data source to be queried.
• The name of the issuer to be used for authentication (the wrap_name parameter). In the case of the OData Service, this is the issuer name displayed on the portal. The default issuer name is the base URI for the data source to be queried with the database user name appended as a path segment (e.g., https://odata.sqlazurelabs.com/OData.svc/v0.1/tatbuixa8p/Pubs/pubsuser). To enable federation, the name of an STS can be provided, instead of the default issuer name, as described above. Note, however that if you plan to use federation, the alternate issuer name must be specified on the OData Portal before you can use it to request a SWT.
• The secret key for the specified issuer name (the wrap_password parameter). In the case of the OData Service, this is the secret key for the database user displayed on the OData Portal.
• One or more claims, expressed as form parameters. The OData Service requires a single claim (authorized=true).

Here is a typical token request body (note - the real secret key is not shown for security purposes):

wrap_scope=https%3a%2f%2fodata.sqlazurelabs.com%2fOData.svc%2fv0.1%2fhqd7p8y6cy%2fAdventureWorks&wrap_name=https%3a%2f%2fodata.sqlazurelabs.com%2fOData.svc%2fv0.1%2fhqd7p8y6cy%2fAdventureWorks%2fodata_guest&wrap_password=9vR4uV7GvBXUcYhcBtbuPWJDwUPKu0UCd%2b%2bRJRC7Egw%3d&authorized=true

If the token request is valid, then the STS will return a SWT in the body of the response as an URL encoded form. Here is a typical SWT for the OData Service (note - this token has expired, and therefore cannot be used):

user=odata_guest&Issuer=https%3a%2f%2fodataacs.accesscontrol.windows.net%2f&Audience=https%3a%2f%2fodata.sqlazurelabs.com%2fOData.svc%2fv0.1%2fhqd7p8y6cy%2fAdventureWorks&ExpiresOn=1271458956&HMACSHA256=St1j434tXjyXbSJMPoAY6gsFMTzc3xsLum1UgvXokpw%3d

SWTs issued by the STS for the OData Service expire 3 minutes after they are issued. Your client should therefore either request a new SWT for each query, or reuse SWTs to reduce network usage, checking the expiration timestamp as necessary, and requesting a new SWT whenever the current one expires.

To pass a SWT to the OData Service, place it in either the Authorization or X-Authorization header, encoding it as follows:

WRAP access_token="user=odata_guest&Issuer=https%3a%2f%2fodataacs.accesscontrol.windows.net%2f&Audience=https%3a%2f%2fodata.sqlazurelabs.com%2fOData.svc%2fv0.1%2fhqd7p8y6cy%2fAdventureWorks&ExpiresOn=1271458956&HMACSHA256=St1j434tXjyXbSJMPoAY6gsFMTzc3xsLum1UgvXokpw%3d"

Which header should you use? The standard Authorization header is recommended by the OAuth WRAP specification, and is therefore preferred. However, it cannot be set by user code in Silverlight 3 and earlier. Silverlight 4 does let user code set it, but only if the clientaccesspolicy.xml file on the target service contains an allows-from entry with an appropriate http-request-headers attribute. The OData service checks both headers for SWTs. If SWTs are found in both locations, they must match.

For more information about how to request and pass SWTs, see the AppFabric SDK, especially the Programming Guide and Samples. Maciej Skierkowski also has some excellent examples on his blog, including examples for PHP, Java and Python. For more information about identity federation, see my blog post on using ACS with Codename "Dallas", and the FabrikamJets example on Code Gallery.

What Now?

That's all there is to it. It's easy to get started, so visit the OData Portal, enter your SQL Azure server information, and try it out. If you don't have a SQL Azure server, you can provision one easily on the SQL Azure Portal.

Glossary

SQL Azure

SQL Azure is a highly available, scalable, multi-tenant, cloud-based relational database service hosted in Windows Azure and based on SQL Server. Users can provision SQL Azure virtual servers using the SQL Azure Portal access them over the Internet.

SQL Azure Labs

SQL Azure Labs is a site that the SQL Azure team is using to publish incubations and early preview bits for products and enhancements in order to gather feedback from the community. Usage of the services in the Labs environment is not billed. Note, however, that usage of production services accessed through the services in the Labs environment will be billed, if applicable. For example, usage of the OData Service is not billed, but usage of an underlying SQL Azure server accessed through the OData Service will be billed, if applicable. Please note that there is no support or SLA associated with the Labs environment, but in return you can preview the future of SQL Azure while helping to shape it. Though similar to a Community Technology Preview, SQL Azure Labs technologies may occasionally be even farther away from commercial availability.

OData

The Open Data Protocol (OData) is an emerging standard for querying and updating data from a wide variety of sources over the Internet using REST based protocols. It extends several existing standards, including Atom (RFC 4287), an XML format for describing structured data as collections of entries consisting of typed name-value pairs, and AtomPub (RFC 5023), a protocol for getting, adding, updating and removing entries. OData builds on the foundation they provide, adding a familiar entity relationship based data model for describing collections, a query language, formatting facilities, and other features required to enable simple and efficient access to structured data over the Internet.

As Chris Sells points out, OData is analogous to earlier data access standards like JDBC, ODBC and OLEDB. Rather than target a platform, however, it targets Web applications, regardless of implementation technology. OData is supported by many Microsoft products, including Windows Azure Storage, SQL Server 2008 R2, Excel 2010, SharePoint 2010, and WCF Data Services (formerly ADO.NET Data Services), which provides client libraries for .NET, Silverlight, AJAX, PHP and Java. It is also supported by IBM's WebSphere eXtreme Scale REST data service and the Open Government Data Initiative. OData is published under the Open Specification Promise, so that anyone can build servers, clients or tools without royalties or restrictions. For more information see, the OData FAQ on MSDN.

AppFabric Access Control

AppFabric Access Control (ACS) is a REST based service that hosts Security Token Services (STSs) provisioned with the Windows Azure AppFabric Portal. ACS based STSs support claims based authentication and authorization using multiple credential types, including SAML and X.509 certificates, and federate with Active Directory Federation Services and other standards based identity providers.

ACSBasedSTSIssuerName.PNG