Using network loopback in side-loaded Windows Store apps

  • Article

[ This article is for Windows 8.x and Windows Phone 8.x developers writing Windows Runtime apps. If you’re developing for Windows 10, see the latest documentation ]

This paper provides information about the use of network loopback for side-loaded Windows Store apps in Windows 8.1 Update.

Introduction

Windows 8 introduced a new approach for app confidence. Consumers must be confident that apps from the Windows Store will perform, co-exist well with other apps on the system and use only the data the user allows. To achieve this confidence, Windows isolates Windows Store apps from each other. Windows Store apps can't communicate with each other except via system-provided contracts and extensions. Likewise, Windows isolates Windows Store apps from apps running on the desktop.

Unfortunately, app isolation blocks many common app architectures used in commercial, line-of-business style apps. In particular, this isolation blocks the use of network loopback for inter-process communication.

With Windows 8.1 Update, we now support the use of network loopback for side-loaded Windows Store apps. This white paper will describe how to develop and deploy such a solution. This white paper has several associated samples:

App architecture

At its heart, a network loopback based solution is like a traditional connected app. A traditional connected app connects to a remote service that exposes data via XML or JSON. In a network loopback based solution, the client app still connects to a service that exposes data via XML or JSON. The only difference is that the service runs on the same device as the connected client app.

Likewise, the development process for a network loopback based solution will be like that of a connected app. The developer builds the back end of the solution using whatever service technology they desire. For managed developers, this is often WCF or ASP.NET. However, a developer could choose to use other technologies such as node.js or PHP instead.

Regardless of the technology used, the service gets installed on the client device. Like a server in a data center, this service listens on a network port for incoming requests. For security purposes, an admin can configure the service to only respond to local machine requests.

The Windows Store app connects to the service via a "localhost" address. Localhost is a special network machine name that resolves to the local machine. Once connected, the Windows Store apps sends requests to and receives responses from the service as if it were on a remote server.

Managed service technologies

Managed developers often use either Windows Communication Foundation (WCF) or ASP.NET Web API to build services running on remote servers. Both WCF and ASP.NET Web API support hosting inside a local service running on a client device. Either technology is a good choice for network loopback scenarios.

Windows Communication Foundation

Windows Communication Foundation(WCF) is a unified programming model for building service-oriented applications. While WCF services are usually accessed across the network, we can also access them via the network loopback. Local services and console apps can host WCF services for loopback scenarios.

WCF service

To use WCF in a loopback scenario, you start by defining and implementing a service contract. Note, these are exactly the same steps you would follow for using WCF in a remote service scenario. The primary difference between loopback and remote scenarios is how the WCF service is hosted. WCF is typically hosted by IIS in remote scenarios. For loopback scenarios, it is more common for WCF services to be hosted in a managed Windows service instead.

The NetworkLoopbackWcf sample includes the following service contract for retrieving information from a Northwind database instance.

[DataContract]
public class Product
{
    [DataMember]
    public int ID { get; set; }
    [DataMember]
    public string Name { get; set; }
    [DataMember]
    public string QuantityPerUnit { get; set; }
    [DataMember]
    public Nullable<decimal> UnitPrice { get; set; }
    [DataMember]
    public Nullable<short> UnitsInStock { get; set; }
    [DataMember]
    public Nullable<short> UnitsOnOrder { get; set; }
    [DataMember]
    public Nullable<short> ReorderLevel { get; set; }
    [DataMember]
    public bool Discontinued { get; set; }
}

[ServiceContract]
public interface INorthwindService
{
    [OperationContract]
    Task<IEnumerable<string>> GetCategories();
 
    [OperationContract]
    Task<IEnumerable<Product>> GetProductsByCategory(string category);
}

Once the service contract is defined, the next step is to implement the contract. Here is the service implementation for the INorthwindService defined above.

class NorthwindService : INorthwindService
{
    public async Task<IEnumerable<string>> GetCategories()
    {
        using (var db = new NorthwindDB.NorthwindEntities())
        {
            return await db.Categories
                .Select(cat => cat.CategoryName)
                .ToListAsync();
        }
    }
 
    public async Task<IEnumerable<Product>> GetProductsByCategory(
                                                string category)
    {
        using (var db = new NorthwindDB.NorthwindEntities())
        {
            var cat = await db.Categories
                .Where(c => c.CategoryName == category)
                .FirstAsync();
 
            return await db.Products
                .Where(prod => prod.CategoryID == cat.CategoryID)
                .Select(p => new Product
                    {
                        ID = p.ProductID,
                        Name = p.ProductName,
                        Discontinued = p.Discontinued,
                        QuantityPerUnit = p.QuantityPerUnit,
                        ReorderLevel = p.ReorderLevel,
                        UnitPrice = p.UnitPrice,
                        UnitsInStock = p.UnitsInStock,
                        UnitsOnOrder = p.UnitsOnOrder,
                    })
                .ToListAsync();
        }
    }
}

This code uses Entity Framework to query the Northwind Database and transform the results into types that WCF can send across the network.

Once the WCF service has been implemented, it must be hosted inside a process on the local device. For production scenarios, it is recommended that the WCF service be hosted inside a managed Windows service. However, during development it is often more convienent to host the WCF service inside a simple console app. Console apps can easily be started, stopped and recompiled more easily than managed Window services.

The NetworkLoopbackWcf sample hosts the NorthwindService in a console app. This is the main entry point of the WcfServiceHost project within the sample solution:

static void Main(string[] args)
{
    var baseAddress = new Uri("net.tcp://localhost:8888/");
 
    // Create the ServiceHost.
    using (var host = new ServiceHost(typeof(NorthwindService), 
                                      baseAddress))
    {
        //configure service metadata behavior
        var smb = host.Description.Behaviors
             .Find<ServiceMetadataBehavior>()
             ?? new ServiceMetadataBehavior();
        smb.MetadataExporter.PolicyVersion = PolicyVersion.Policy15;
        host.Description.Behaviors.Add(smb);
 
#if DEBUG
        //add metadata exchange endpoint in debug builds only
        host.AddServiceEndpoint(
            ServiceMetadataBehavior.MexContractName,
            MetadataExchangeBindings.CreateMexTcpBinding(),
            "mex");
#endif
 
        //add service endpoint
        host.AddServiceEndpoint(
            typeof(INorthwindService),
            new NetTcpBinding(SecurityMode.None),
            string.Empty);
 
        // Open the ServiceHost to start listening for messages. 
        host.Open();
        Console.WriteLine("The service is ready at {0}", 
                          host.BaseAddresses.First());
 
        Console.WriteLine("Press <Enter> to stop the services.");
        Console.ReadLine();
 
        // Close the ServiceHost.
        host.Close();
    }
}

This is fairly straight forward WCF hosting code that is covered in depth in the WCF documentation. One thing to note is the inclusion of a metadata exchange endpoint in debug builds of the service. We will use that endpoint in the next section to automatically generate the client side code for interacting with this WCF service. Since the code generation only occurs during development, the metadata endpoint is not needed in production and is removed via the #if DEBUG directive.

WCF client

Once the service is implemented, the next step is to implement the WCF client. Because the service exposes a metadata endpoint, we can use the Visual Studio Add Service Reference feature to generate the client side code for interacting with the service.

In order to use the Add Service Reference feature, the WCF service needs to be running. In Visual Studio, you can right click the command line service host process and select Set as StartUp project and then select Debug / Start Without Debugging from the main menu.

Once the service host process is running, you can right click on the References node of the client project in the Solution Explorer and select Add Service Reference…. This will bring up the Add Service Reference dialog.

Enter the address for the local service (in the sample app, the address is net.tcp://localhost:888) and press the Go button. Visual Studio will automatically connect to the metadata endpoint of the service and pull down the information needed to generate the client side code. Enter a reasonable namespace name (it defaults to "ServiceReference1") and press OK.

Once the client side code has been generated, you can create an instance of the client object and invoke methods on it directly. Note, all of the methods on the generated client will be async methods, so use C# 5.0 async and await keywords to simplify your async code. Here is an example event handler from the sample app.

private async void GetCategories_Click(object sender, 
                                       RoutedEventArgs e)
{
    NorthwindServiceClient nwindClient = new NorthwindServiceClient();
    var categories = await nwindClient.GetCategoriesAsync();
    CategoriesListBox.ItemsSource = categories;
}

ASP.NET

ASP.NET is a framework for building web sites, apps and services. It includes the Web API framework for building web services. It also includes the SignalR library that enables bi-directional communication between server and client. Both Web API and SignalR can be hosted in console apps and local services for loopback scenarios.

Web API controllers

To use ASP.NET Web API in a loopback scenario, you start by defining and implementing a Web API controller. Note, these are exactly the same steps you would follow for using Web API in a remote service scenario. The primary difference between loopback and remote scenarios is how the Web API components are hosted. Like WCF, Web API is typically hosted by IIS in remote scenarios. For loopback scenarios, it is more common for Web API components to be hosted in a managed Windows service instead.

The NetworkLoopbackRest sample includes the following Web API controllers for retrieving information from a Northwind database instance.

public class CategoriesController : ApiController
{
    public IEnumerable<string> Get()
    {
        using (var db = new NorthwindEntities())
        {
            var cats = db.Categories
                .Select(c => c.CategoryName.Replace('/', '-‘))
                .Select(s => WebUtility.UrlEncode(s))
                .ToList();
 
            return cats;
        }
    }
}

public class ProductsController : ApiController
{
    public IEnumerable<Product> Get()
    {
        using (var db = new NorthwindEntities())
        {
            var p = db.Products
                .Include("Category")
                .Include("Supplier")
                .ToList();

            return p;
        }
    }

    public IEnumerable<Product> Get(string id)
    {
        var cat_name = WebUtility.UrlDecode(id.Replace('-', '/'));

        using (var db = new NorthwindEntities())
        {
            var products = db.Products
                .Include("Category")
                .Include("Supplier")
                .Where(p => string.Compare(p.Category.CategoryName,
                                           cat_name, true) == 0)
                .ToList();

            return products;
        }
    }
}

As you can see, we have a separate Web API controller for each data entity we want to expose. Like the WCF sample, we are using Entity Framework for data access.

Additionally, the data needs to be encoded in a way that allows for transfer via standard web protocols such as URIs, JSON and XML. For Categories, that are simple strings, the sample does the encoding directly in the Get function. For Products, the sample includes a type (ProductJsonConverter) to handle the JSON encoding. Data encoding is a standard part of Web API and is not covered in this whitepaper.

SignalR Connections and Hubs

To use ASP.NET SignalR in a loopback scenario, you start by defining and implementing a SignalR Connection and/or Hub. Note, these are exactly the same steps you would follow for using SignalR in a remote service scenario.

The NetworkLoopbackRest sample includes the following SignalR connection for handling messages received from the client. In this trivial example, the received message is simply reversed and re-broadcast to the client.

public class MyConnection : PersistentConnection
{
    protected override Task OnReceived(IRequest request, 
                                       string connectionId, 
                                       string data)
    {
        // Broadcast data to all clients
        Console.WriteLine("{0} {1}", connectionId, data);
        string it = new string(data.Reverse().ToArray());
 
        return Connection.Send(connectionId, it);
    }
}

Additionally, the NetworkLoopbackRest sample includes the following code to read messages from the commandline and then broadcast them to the client app.

var ctx = GlobalHost.ConnectionManager
            .GetConnectionContext<MyConnection>();

var line = Console.ReadLine();
while (!string.IsNullOrWhiteSpace(line))
{
    ctx.Connection.Broadcast(line);
    line = Console.ReadLine();
}

Hosting Web API and SignalR components

Web API and SignalR component must be hosted in a process in order to be accessable to app clients. ASP.NET has a standard hosting interface (Open Web Interface for .NET or OWIN) and a set of hosting components (project Katana) that developers can use in hosting these components in loopback scenarios.

The host process can be a console app or a local service. Either way, the Katana hosting components looks for a class named Startup that describes what services are to be hosted and where. Here is the startup class from the NetworkLoopbackRest sample:

public class Startup
{
    public void Configuration(IAppBuilder app)
    {
        var config = new HttpConfiguration();
        config.Routes.MapHttpRoute(
            name: "DefaultApi",
            routeTemplate: "api/{controller}/{id}",
            defaults: new { id = RouteParameter.Optional }
        );

        //Add a custom JSON converter for Product 
        //to eliminate circuluar references 
        config.Formatters.JsonFormatter.SerializerSettings
          .Converters.Add(new ProductJsonConverter());
 
        app.UseWebApi(config);
        app.MapSignalR<MyConnection>("/myconnection");
    }
}

Details regarding self hosting for Web API and SignalR are included in the official documentation for those products. However, please note the registration of ProductJsonConverter described earlier.

Once the Startup class is defined, it is passed as a type parameter to the WebApp.Start method when the apps start. In the case of the NetworkLoopbackRest sample, it is in the Main method of the console host app. This code registers both the Web API and SignalR components to listen on localhost port 9000. Additionally, note the code we saw earlier for reading messages off the command line and sending them to the client app.

static void Main(string[] args)
{
    string baseAddress = "https://localhost:9000/";
 
    using (Microsoft.Owin.Hosting.WebApp.Start<Startup>(baseAddress))
    {
       Console.WriteLine("REST service is ready at {0}", baseAddress);
       Console.WriteLine("Enter message to send, empty line to exit");
 
       var ctx = GlobalHost.ConnectionManager
                   .GetConnectionContext<MyConnection>();

        var line = Console.ReadLine();
        while (!string.IsNullOrWhiteSpace(line))
        {
             ctx.Connection.Broadcast(line);
             line = Console.ReadLine();
        }
    }
}

Web API client

Once the Web API and SignalR components as well as the process to host them, the next step is to build the client app.

For Web API client code, there isn't a strongly typed client-side class like there was for WCF clients. However, because Web API controllers return code in a self-defining format like JSON or XML, it's fairly straight forward to parse the string returned from the controller into an object that can be consumed in the client code.

Here is the code from networkLoopbackRest to retrieve the list of categories. The data is returned as an array of strings. The code below uses LINQ to extract the string from the JsonArray so it can be data bound directly in the XAML.

private async void CategoriesListBox_SelectionChanged(object sender, 
                     SelectionChangedEventArgs e)
{
    if (CategoriesListBox.SelectedItem == null)
    {
        ProductsListBox.ItemsSource = null;
        return;
    }
 
    var url = new Uri("https://localhost:9000/api/products/" + 
       CategoriesListBox.SelectedItem.ToString());
    var client = new HttpClient();
    var json = await client.GetStringAsync(url);
 
    var products = JsonArray.Parse(json)
       .Select(jv => jv.GetObject());
    ProductsListBox.ItemsSource = products
       .Select(jo => jo.GetNamedString("name"));
}

SignalR client

SignalR provides client libraries for both JavaScript and .NET. The NetworkLoopbackRest sample uses .NET, but could also have been built in HTML and JavaScript.

The client library provides a simple to use object model for connecting to SignalR endpoints. Here is the sample code for connecting to the SignalR connection shown previously:

protected override async void OnNavigatedTo(NavigationEventArgs e)
{
    connection = new Connection("https://localhost:9000/myconnection");
    connection.Received += connection_Received;
    await connection.Start();
}

Once the connection has been created and started, client code can send messages to the server via SignalR via the Connection.Send method. Any messages from the server will trigger the Connection.Received event. Here is code from the sample demonstrating both:

private async void Go_Click(object sender, RoutedEventArgs e)
{
    var msg = MessageTextBox.Text;
    await connection.Send(msg);
}

async void connection_Received(string obj)
{
    await Dispatcher.RunIdleAsync(args =>
    {
        rootPage.NotifyUser("Message Received: " + obj,
                            NotifyType.StatusMessage);
    });
}

Configuring the firewall

Windows Firewall blocks inbound network connections by default. This includes inbound connections that originate on the device. The Windows Firewall must be configured to the service to communicate thru the firewall.

CheckNetIsolation utility

Windows Firewall also blocks loopback connections for all Windows Store apps by default. A device admin can enable loopback for a Windows Store app using the CheckNetIsolation.exe tool. This tool is available on the command line in any Windows 8.1 installation.

To see a list of all Windows Store apps currently allowed to use loopback connections, run checknetisolation loopbackexempt -s.

Note  f you do this from a machine that is used for Windows Store app development, you may see entries for apps currently under development on the machine. Visual Studio automatically enables loopback access though the firewall for any apps that are in development.

 

To add a Windows Store app to the list of apps that are exempt from the loopback firewall, run checknetisolation loopbackexempt -a -n=<package family name> from an elevated command prompt. The package family name for a Windows Store app is available from Visual Studio via the Package.appxmanifest editor on the packaging tab.

Firewall management API

Windows includes a set of firewall management APIs that can be used to disable the loopback firewall programmatically. Demonstration of this is beyond the scope of this white paper, but the Windows 8 Loopback Exemption Manager project on CodePlex (Windows 8 Loopback Exemption Manager) demonstrates how to use the APIs to disable the firewall for specific Windows Store apps. In particular, there are two functions needed to disable the firewall:

  • NetworkIsolationEnumAppContainers is used to retrieve an array of properties about installed Windows Store apps. In particular, the properties include the Package Family Name (shown above) and the app container's security identifier (or SID).
  • NetworkIsolationSetAppContainerConfig is used to add specific Windows Store apps to list of apps allowed to traverse the loopback firewall. This function needs the app container's SID, which can be found using the NetworkIsolationEnumAppContainers function described above.

Resources