• Article

July 2014

Volume 29 Number 7

Azure Web Sites : Hybrid Connectivity: Connecting Azure Web Sites to LOB Apps Using PortBridge

Tejaswi Redkar

Contrary to popular belief, not everything is going to the cloud, at least not yet. Just as all cars are not yet hybrid and most appliances are not yet energy efficient, a good amount of software will still be running on-premises for years to come. Deep investments are difficult to abandon, and most companies require a significant business justification for modernizing an existing app. Do you have solar panels in your house? In spite of long-term energy savings and tax benefits, people are reluctant to invest in them due to upfront costs. It’s not that different for organizations when it comes to moving to the cloud.

Fortunately, however, hybrid connectivity allows you to modernize your applications by tapping into the power of the cloud, while still plugging into existing software services that run on-premises in your datacenter. In this article, I’ll show you how to build a Web site that leverages the Azure Service Bus for connecting to software services running in non-Azure datacenters. Instead of using the Service Bus API directly, I’ll use a popular utility called PortBridge to connect Azure Web Sites with on-premises services.

Hybrid Connectivity in Azure

Azure includes a number of services for building hybrid applications. For hybrid connectivity, the two most commonly used are Azure Virtual Network and Azure Service Bus. Azure Virtual Network lets you extend your on-premises network into Azure, thus carving out a private hybrid network between Azure and the datacenter. The extension between Azure and your datacenter happens at the network layer, rather than the application layer in Service Bus. Because this article is about application connectivity using Service Bus, I won’t be covering Azure Virtual Network here. For more information, visit bit.ly/QAODgX.

The Service Bus Relay service lets you connect two applications residing behind a firewall. The applications can be residing in Azure or your datacenters or both. Service Bus Relay gives you the ability to register your application’s Windows Communication Foundation (WCF) endpoints into the Service Bus registry in Azure datacenters. Method invocations can then be securely exchanged between the client and the server within the Service Bus infrastructure and then communicated to the respective applications running in your datacenter. This is ideal, for example, for a scenario in which a utility company needs access to the HVAC or power meter for collecting data and sending important events to these devices from a central server running in Azure. Figure 1 illustrates this example, where the utility company has a head-end running in Azure and devices in buildings.

Service Bus Relay Scenario at a Utility Company
Figure 1 Service Bus Relay Scenario at a Utility Company

The control gateway is a device running in buildings that’s responsible for controlling electrical devices. Each control gateway has a WCF endpoint registered with the Service Bus registry with a globally unique identifier. Whenever the head-end service, running in one of the Azure Compute Services, wants to communicate with a particular control gateway, it opens a connection to the globally unique endpoint in Service Bus and starts sending or retrieving messages to or from the control gateway. Typically, the control gateway resides behind the building’s firewall. But what if you have a non-WCF service (such as a SQL Server database) running on-premises that you want to connect from an Azure Web Site?

Introducing PortBridge

PortBridge is a utility built on top of the Service Bus API that allows you to communicate between any TCP-based service endpoints running on-premises and on Azure. The core concept of PortBridge and the prototype were developed by Clemens Vasters (bit.ly/SI93GM). I’ve modified it a little for this article and compiled it with the recent version of the Service Bus SDK. When you build applications for Service Bus Relay, you’re forced to build a WCF interface for all the relevant endpoints you want to expose in the cloud. This can get inconvenient and tedious when you have a large number of endpoints or generic platform endpoints, like databases and search engines. Adding another WCF abstraction on top of these services doesn’t make sense. Instead, you can use PortBridge to expose non-WCF TCP endpoints for Service Bus Relay connectivity. Front-end applications like Azure Web Sites can then take advantage of connecting to any TCP data source residing in your datacenter behind a firewall. Conceptually, PortBridge creates a generic WCF interface as shown in Figure 2.

Figure 2 The Generic PortBridge Windows Communication Foundation Interface

namespace Microsoft.Samples.ServiceBus.Connections
{
  using System;
  using System.ServiceModel;
  [ServiceContract(Namespace="n:",
    Name="idx", CallbackContract=typeof(IDataExchange),
    SessionMode=SessionMode.Required)]
  public interface IDataExchange
  {
    [OperationContract(Action="c", 
      IsOneWay = true, IsInitiating=true)]
    void Connect(string i);
    [OperationContract(Action = "w", IsOneWay = true)]
    void Write(TransferBuffer d);
    [OperationContract(Action = "d", 
      IsOneWay = true, IsTerminating = true)]
    void Disconnect();
  }
  public interface IDataExchangeChannel : 
    IDataExchange, IClientChannel { }
}

With just three generic methods, PortBridge acts as a proxy between the client and server endpoints and forwards all TCP calls to the designated server over a Service Bus Relay. The biggest advantage of using PortBridge is you don’t have to build a WCF interface for every on-premises endpoint you want to enable for hybrid connectivity.

PortBridge is composed of two components:

  1. An Agent that runs closer to the client application, or in a virtual machine (VM) in the Azure Infrastructure as a Service (IaaS).
  2. A Windows Service (or a console application) that runs on-premises and acts as a proxy for service endpoints running in the same environment.

Some common use cases for PortBridge are:

  • Connecting to any TCP-based data store on-premises
  • Connecting to third-party Web services that can’t be migrated to Azure
  • Remote desktop connections

In this article, you’ll learn how to deploy and configure PortBridge to enable communication between Azure Web Sites and a SQL Server database running on your local machine.

A Hybrid Solution

Azure Web Sites typically forms the Web front end or Web services of your application tier. But in many situations, you can’t migrate some of the data sources that power the Web site to Azure. In such cases, PortBridge is an ideal solution for connecting Azure Web Sites to databases running on-premises. Azure Web Sites communicates with the PortBridge Agent, which in turn communicates with the PortBridge service via Service Bus. The PortBridge service then forwards the call to the destination database. Figure 3 illustrates the architecture of the hybrid Web site I’ll build in this article and shows a typical message exchange in a PortBridge (or Service Bus Relay) environment:

PortBridge Sample Architecture
Figure 3 PortBridge Sample Architecture

  1. The on-premises PortBridge console application (or Windows service) registers the database’s TCP endpoint (1433 for SQL Server) within the Service Bus naming registry with a unique URI. Service Bus Relay opens an outbound bi-directional connection with the console application.
  2. When an HTTP or HTTPS request comes in to Azure Web Sites, the Web site opens the database connection as usual, but with the IP address of the PortBridge Agent VM instead.
  3. The PortBridge Agent VM acts as the database proxy in the cloud and has two communication interfaces—for receiving TCP requests and for connecting to the PortBridge console’s Service Bus Relay endpoint.
  4. The PortBridge Agent routes the database request to Service Bus Relay service. The beauty of the solution is that for Azure Web Sites, it’s business as usual. The API to the database hasn’t changed, only the IP address has.
  5. Service Bus Relay routes the call to the PortBridge application running on your localhost (or datacenter).
  6. Finally, the call reaches the database, and the data is retrieved using the same connection.

Note that there are a number of components involved in communicating over PortBridge, and you do need an extra VM that might not be needed if using Service Bus directly. As I mentioned earlier, the advantage here is the generalization of exposing any TCP endpoints—you can reuse the same PortBridge Agent VM to connect with multiple data sources running in different locations. Now that I’ve shown you the system architecture of the solution, I’ll start constructing it.

Creating a RESTful Web Service

In order to keep this article simple, I’ll design a RESTful ASP.NET Web API service called Countries that retrieves a list of all countries from a SQL Server database table. In the development process, I always recommend building and testing locally before deploying to the cloud. Figure 4 shows the code for the Countries Web service.

Figure 4 Countries Web Service

public class CountriesController : ApiController
{
  private string DatabaseConnectionString { get; set; }
  public CountriesController()
  {
    try
    {
      DatabaseConnectionString =
        ConfigurationManager.
        ConnectionStrings["DatabaseConnectionString"].
        ConnectionString;
      }catch(Exception ex)
      {
        Trace.TraceError(ex.Message);
      }
  }
  // GET: api/Countries
  public IEnumerable<Country> Get()
  {
    return CountryService.GetCountries(DatabaseConnectionString);
  }
  // GET: api/Countries/AD
  public Country Get(string id)
  {
    return CountryService.GetCountry(DatabaseConnectionString, id);
  }
}

The Web service has only two methods—Get and Get(string id). The Get method retrieves all the countries from the database table and the Get(id) method retrieves a Country object by the country code. Figure 5 illustrates the Country database table in the local SQL Server database.

Countries Database Table
Figure 5 Countries Database Table

Figure 6 shows the CountryService class that retrieves the data from the database and returns Country objects.

After the database and the Web service are ready, you can quickly test the Web service locally by pressing F5 to run it from Visual Studio. If everything goes well, you have a Web service that works locally. Now you want to move it to the cloud, but you can’t migrate the database to the cloud because it’s shared with other applications that will still be running on-premises. PortBridge fits perfectly in this scenario, with minimal or no code changes to your Web service.

Figure 6 The CountryService Class

public class CountryService
{
  public static IEnumerable<Country> GetCountries(string connectionString)
  {
    IList<Country> countries = new List<Country>();
    using (IDataReader reader = 
      SqlHelper.ExecuteReader(connectionString,
      System.Data.CommandType.Text, "SELECT CountryId, CountryName,
      CountryCode FROM Country"))
    {
      while(reader.Read())
      {
        Country c = new Country()
        {
          CountryId = Convert.ToInt32(reader["CountryId"]),
          CountryCode = Convert.ToString(reader["CountryCode"]),
          CountryName = Convert.ToString(reader["CountryName"]),
        };
        countries.Add(c);
      }
    }
    return countries;
  }
  public static Country GetCountry(string connectionString, 
    string countryCode)
  {
    Country c = new Country();
    using (IDataReader reader = 
      SqlHelper.ExecuteReader(connectionString,
      System.Data.CommandType.Text,
      "SELECT CountryId, CountryName, CountryCode FROM Country WHERE
      CountryCode=@countryCode",
      new SqlParameter("@countryCode", countryCode)))
    {
      if (reader.Read())
      {
        c.CountryId = Convert.ToInt32(reader["CountryId"]);
        c.CountryCode = Convert.ToString(reader["CountryCode"]);
        c.CountryName = Convert.ToString(reader["CountryName"]);
      }
    }
    return c;
  }
}
  [DataContract]
    public class Country
    {
      [DataMember]
      public int CountryId { get; set; }
      [DataMember]
      public string CountryName { get; set; }
      [DataMember]
      public string CountryCode { get; set; }
}

Setting up PortBridge

Before moving the Web service to the cloud, you need to configure and test PortBridge. Because there are multiple tiers in the architecture, it’s important to get the configuration of all the components right at the start. I usually create a table listing input and output endpoints for each component (see Figure 7).

Figure 7 Endpoints and Locations

Solution Component Input Endpoint Output Endpoint Output Endpoint
Countries Web Service 80   Azure Web Site
PortBridge Agent 1433 1433 Azure Virtual Machine
PortBridge 1433 1433 Azure Virtual Machine
SQL Server 1433   Azure Virtual Machine

This table might seem trivial in this case, but for applications with hundreds of endpoints, a table like this will help you configure services correctly. As discussed earlier, the PortBridge Agent resides in a VM and has two endpoint interfaces—input from the application and output to a Service Bus Relay endpoint. Similarly, the PortBridge service running on your local machine has two endpoints—input from Service Bus Relay and output to the SQL Server database. Once you have the configuration parameters noted down, you can start the deployment process.

Creating a Service Bus Namespace

Because the communication backbone of this solution is Service Bus Relay, I have to first create a Service Bus namespace from the Azure portal (manage.windowsazure.com), as shown in Figure 8.

Creating the Service Bus Namespace
Figure 8 Creating the Service Bus Namespace

The countries namespace in Figure 8 is globally unique and will be used to create a globally unique endpoint for the services. Once you’ve created the namespace, click on the Connection Information button at the bottom of the page to see the namespace’s access infor­mation. You will need the Default Issuer and the Default Key for accessing the namespace, shown in Figure 9, so make a note of these.

Service Bus Namespace Credentials
Figure 9 Service Bus Namespace Credentials

The PortBridge Agent and PortBridge application both require these credentials for connecting to the namespace.

Configuring the PortBridge Application

The PortBridge application has a custom configuration section:

<portBridge serviceBusNamespace="countries" 
  serviceBusIssuerName="owner"
  serviceBusIssuerSecret="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX">
  <hostMappings>
   <!-- Open HTTP, SQL Server, RDP and ElasticSearch ports-->
     <add targetHost="TREDKAR-W530" allowedPorts="1433" />
  </hostMappings>
</portBridge>

The namespace, issuer name, and the secret are required to connect to the Service Bus Relay namespace I created earlier. The host-mapping section lists the target service host. In my example, it’s the local machine and 1433 port. You can add comma-separated ports in order to externalize multiple endpoints on the same host. You can also add multiple target hosts as long as they’re reachable from the PortBridge application machine. Once the configuration is complete, start the PortBridge application on your local machine. If everything goes as expected, you should see a screen similar to the one in Figure 10.

PortBridge Application Running
Figure 10 PortBridge Application Running

The PortBridge application creates an outbound connection to the Service Bus Relay service on the countries namespace and creates a unique name with the format PortBridge/[target host], where [target host] is the targetHost parameter in the PortBridge application’s configuration file, as shown in Figure 11.

Service Bus Relay Connection
Figure 11 Service Bus Relay Connection

The PortBridge part of the name is hardcoded, whereas the target host changes depending on the number of hosts you’ve created. It’s important to note that to keep the name within the namespace unique, every target host will have only one entry in the hostMappings section of the configuration file.  Also note that if you don’t configure the targetHost parameter correctly, the created endpoint will not match the service you want to represent and communications will fail.

For information on outbound ports required by Service Bus Relay bindings, visit bit.ly/1l8lncx.

Configuring the PortBridge Agent

Like the PortBridge application, the PortBridge Agent also has a custom configuration section:

<portBridgeAgent serviceBusNamespace="countries" 
  serviceBusIssuerName="owner"
  serviceBusIssuerSecret=" XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX ">
<portMappings>
<port localTcpPort="1433" targetHost="TREDKAR-W530" 
      remoteTcpPort="1433">
<firewallRules>
  <rule source="127.0.0.1"/>
  <rule sourceRangeBegin="208.0.0.0" 
        sourceRangeEnd="208.255.255.255"/>
...
</firewallRules>
</port>
</portMappings>

The PortBridge Agent has a port-mapping section where you map each input endpoint port (localTcpPort) to the Service Bus relay endpoint port (remoteTcpPort). The target host value must match the target host value you created earlier in the PortBridge application configuration. This value is used for connecting to the appropriate Service Bus namespace endpoint. If these two values don’t match, the application will not work.

In the firewall rules section, you can explicitly list the IP addresses of machines from which the PortBridge Agent will accept requests. Because my PortBridge Agent will be accepting requests from an Azure Web Site, I need to add the IP ranges of the Azure datacenters to the firewall rules. You can download the Azure datacenter IP ranges from bit.ly/1l8yDxV.

To deploy PortBridge Agent in Azure, navigate to the Azure portal, create a new Windows VM, copy the PortBridge Agent files and run PortBridgeAgent.exe. As you create the VM, make sure you open the 1433 endpoint for external access so the Azure Web Site can access that port on the PortBridge Agent VM.

Another option for deploying PortBridge Agent is via Apps for Azure, an easy way to deploy Windows Store apps. Apps for Azure is a free app with prepackaged VMs readily available for deployment, and it has a prepackaged PortBridge VM, as shown in Figure 12. You can install Apps for Azure from appsforazure.com.

Apps for Azure for Deploying the PortBridge Agent
Figure 12 Apps for Azure for Deploying the PortBridge Agent

The prepackaged PortBridge Agent VM by default opens the 1433 and 80 endpoint ports for communication. If you want to custom configure the PortBridge Agent, you need to modify the PortBridgeAgent.exe.config file in the C:\ddapplications folder. After configuring the file, you’ll have to restart the Dynamic­DeployInitService Windows Service.

Testing and Deploying the Countries Web Service

Once the PortBridge components are installed and running, you’ll need to modify the Web service’s database connection string to point to the PortBridge Agent VM:

<add name="DatabaseConnectionString"
  connectionString=
  "Server=tejaswisvm1.cloudapp.net;
  Database=cf10292013;
  Trusted_Connection=True;"/>

Note that apart from the host name, all other parameters remain the same. With this configuration change, you’ll be pointing to the PortBridge Agent VM instead of the database directly. Next, publish the Countries Web API service to Azure Web Sites and test it by navigating to the following URIs:

  1. https://[Azure Web site host]/api/Countries, for retrieving all the countries.
  2. https://[Azure Web site host]/api/Countries/[Country Code], for retrieving a specific country by country code.

If the end-to-end communication is working, both the URIs will return JSON objects for the method invocations. You can now call the Countries Web service from any device and the call will traverse from Azure Web Site to PortBridge Agent, and reach the SQL Server database via the PortBridge application. The data will be retrieved from the database running on your local machine (or datacenter).

Performance Considerations

Because PortBridge is a layer of indirection, it does cause latency when compared to architectures that involve direct database communications or virtual network hybrid connectivity. PortBridge is recommended only in scenarios where virtual network and direct communications aren’t feasible. At the time of this writing, Azure Web Sites don’t support virtual networking and, therefore, Service Bus is the only option for building hybrid connectivity. During my testing, I saw latencies ranging from 50ms to 300ms for the Countries Web service. For some scenarios, I recommend caching the data in the cloud and only periodically reaching out to the services running in remote datacenters.

Security Considerations

PortBridge relies on the security capabilities of Service Bus Relay. By default, PortBridge uses the connection security to connect to the Service Bus namespace, but it does not employ transport or message encryption capabilities. The PortBridge Agent provides a custom IP address filtering mechanism, but this should not be considered as robust as the built-in security mechanisms of Azure Service Bus. I recommend performing detailed thread modeling before deploying the solution in production.

Scaling Out PortBridge

From the architecture in Figure 3, it looks like the PortBridge Agent is a single point of failure. But you can scale out the PortBridge Agent by adding a load-balanced endpoint to the VM and creating multiple instances of the PortBridge Agent VM. The calls coming from the Azure Web Site will then be load balanced across multiple PortBridge Agents. Even though you can scale out PortBridge Agent, the final destination endpoint (the database) should be designed to handle this scale out. You don’t want to scale out the Web and the middle tier but fail on the data tier. Your design should be elastic all the way through.

PortBridge in the Real World

In the past couple of years, based on several customer requests, our team built a messaging hub in the cloud for integrating third-party applications with Azure services. The goal was to create an easy plug-and-play model for collecting data from disparate data sources and then surface the aggregated data to end users on devices of different sizes. The team decided to use PortBridge as the backbone for creating this plug-and-play model because it provided the necessary abstraction for connecting non-WCF services with applications running in Azure. We did modify the PortBridge code for better diagnostics, performance and caching. Some of the scenarios we successfully implemented were:

  1. Creating a unified messaging service that integrated the 19 different data sources of a Fortune 50 company, and providing an on-the-job contextual data mashup to the customer’s field employees. Quick access to necessary infor­mation in the field was the primary goal of this solution.
  2. Retrieving data from thousands of application instances used for managing commercial vehicle inventory and storing the data in a central data repository in the cloud. The captured data was used for predictive maintenance and studying buyer behavior. Without Service Bus (and PortBridge), such an application would have taken 10 times the cost and effort.
  3. Providing case information and timesheet management to thousands of lawyers on their mobile devices anywhere. Before this application, the lawyers had to travel back to the office after every court case to log their time. With the mobile app, they can log their entries directly from the courthouse.

Wrapping Up

Azure Web Sites and Service Bus complement each other in the building of hybrid Web applications. PortBridge is just an abstraction layer on top of Service Bus Relay for conveniently exposing non-WCF endpoints in the cloud. I have dealt with several scenarios that required building aggregation mashup Web sites where data is retrieved from multiple Web services running at different locations. By using PortBridge, these applications could be designed, tested and built without significant changes to the original application code. Once the initial PortBridge configuration is tested, the connectivity works consistently as long as the Service Bus and Web services are available. Even though PortBridge lets you quickly build hybrid applications, keep in mind it does introduce latency in your service calls. If your application is response-time sensitive, PortBridge may not be the right choice and you should evaluate Azure Virtual Network or migrate on-premises services to the cloud. After walking through the solution outlined in this article, you should be comfortable externalizing any on-premises endpoint into Azure and then calling it from an Azure Web Site. Source code for PortBridge and the accompanying Countries Web service is available on GitHub at github.com/dynamicdeploy/portbridge.

Tejaswi Redkar is an author and a software developer. He currently works for Microsoft as a director of application platform strategy and communities. His book, “Windows Azure Web Sites: Building Web Apps at a Rapid Pace” (Dynamic Deploy LLC, 2013), is the most comprehensive and best selling on the topic. Redkar is also the creator of appsforazure.com and dynamicdeploy.com, where he experiences firsthand running production apps in the cloud. You can reach out to him at tejaswi_redkar@hotmail.com and follow him on Twitter at twitter.com/tejaswiredkar.

Thanks to the following technical experts for reviewing this article: Microsoft Azure Product Management Team