Thought Leaders in the Cloud: Talking with Maarten Balliauw, Technical Consultant at RealDomen and Windows Azure Expert
Maarten Balliauw is an Azure consultant, blogger, and speaker, as well as an ASP.NET MVP. He works as a technical consultant at RealDomen. Maarten is a project manager and developer for the PHPExcel project on CodePlex, as well as a board member for the Azure User Group of Belgium (AZUG.BE).
In this interview we discuss:
- Take off your hosting colored glasses – architecting for the cloud yields benefits far beyond using it as just a mega-host
- Strange bedfellows – PHP on Azure and why it makes sense
- Average is the new peak – The cloud lets you pay for average compute, on-premesis makes you pay for peak capacity
- Datacenter API – It makes sense for cloud providers to expose APIs for many of the same reasons it makes sense for an OS to expose APIs
- Data on the ServiceBus – Transferring data securely between your datacenter and your public cloud applications
Robert Duffner: Could you tell us a little bit about yourself to get us started?
Maarten Balliauw: I work for a company named RealDolmen, based in Belgium. Historically, I've worked with PHP and ASP.NET, and since joining RealDolmen, I have been able to work with both technologies. While my day to day job is mostly in ASP.NET, I do keep track of PHP, and I find it really interesting to provide interconnections between the technologies.
And Azure is also a great platform to work with, both from an ASP.NET perspective and a PHP perspective. Also, in that area, I'm trying to combine both technologies to get the best results.
Robert: You recently spoke at REMIX 10, and in your presentation, you talked about when to use the cloud and when not to use the cloud. What's the guidance that you give people?
Maarten: The big problem with cloud computing at this time is that people are looking at it from a perspective based on an old technology, namely classic hosting. If you look at cloud computing with fresh eyes, you will see that it is really an excellent opportunity, and that no matter what situation you are in, your solution will always be more reliable and a lot cheaper if you do it in the cloud.
Still, not every existing application can be ported to the cloud at this time. One important metric to use in choosing between cloud and non-cloud deployments is your security requirements. Do you care about keeping your data and applications on premises versus in the cloud?
Robert: We've obviously engineered Azure to be an open platform that runs a lot more than just .NET, including dynamic languages like Python and PHP that you mentioned, but also languages like Java. But you talk quite a bit about PHP on Azure. From your perspective, why would anyone want to do that when there are so many options for PHP hosting today?
Maarten: You can ask the same question about ASP.NET hosting. There are so many options to host your .NET applications somewhere: on a dedicated server, on a virtual private server, on a cloud somewhere. So I think the same question applies to PHP, Java, Ruby, and whatever language you're using.
Azure provides some quite interesting things with regard to PHP that other vendors don't have. For example, the service bus enables you to securely connect between your cloud application and your application that's sitting in your own data center. You can leverage that feature from .NET as well as from PHP. So it's really interesting to have an application sitting in the cloud, calling back to your on premises application, without having to open any firewall at all.
Robert: In your talk, you also point to the Azure solution accelerators for Memcached, MySQL, MediaWiki, and Tomcat. In your experience, are most people even aware that these kinds of things run on Azure?
Maarten: I'm not sure, because, traditionally, the Microsoft ecosystem is quite simple. There's Microsoft as a central authority offering services, and then there are other vendors adding additional functionality, bringing some other alternative components. In the PHP world, for example, there is no such thing as a central authority, so information is really distributed across different communities, companies, and user groups.
I think some part of the message from Azure is already out there in all these communities and all these small technology silos in the PHP world, but not everything has come through. So I'm not sure if everyone is aware that all these things can actually run on Azure, if you're using PHP or MySQL or whatever application that you want to host on Azure.
Robert: In another talk, you mentioned "Turtle Door to Door Deliveries," and how they estimated needing six dedicated servers for peak load, but because the load is fairly elastic, they saw savings with Azure. Can you talk a little bit more about that example?
Maarten: That was actually a fictitious door-to-door delivery company, like DHL, UPS, or FedEx, which we created for that talk. They knew the load was going to be around six dedicated servers for peaks, but there were times of the day where only one or two servers would be needed. And when you're using cloud computing, you can actually scale dynamically and, for example, during the office hours have four instances on Azure, and during the evening have six instances, and then in the night scale back to two instances.
And if you take an average of the number of instances that you're actually hosting, you will see that you're not actually paying for six instances, so that you're only paying for three or, maximum, four. Which means you have the extra capacity of two machines, without paying for it.
We have done a project kind of like that, for a company in Belgium, doing timing on sports events. Most of these events have a maximum of 1,000 or 2,000 participants, but there are several per year with 30,000 participants.
We used the same technique for that project as in the example in the talk. We scaled them up to 18 instances during peaks, and at night, for example, we scaled them back to two instances. They actually had the capacity of 18 instances during those peak moments, but on average, they only had to pay for seven instances.
Robert: One important characteristic of clouds is the ability to control them remotely through an API. Amazon Web Services has an API that lets you control instances, and you recently wrote a blog post showing a PowerShell script that makes an app auto-scale out to Azure when it gets overloaded. What are some of the important use cases for cloud APIs?
Maarten: If you have a situation where you need features offered by a specific cloud, then you would need those cloud APIs. For example, if you look at the PHP world, there's an initiative, the Simple Cloud API, which is one interface to talk to storage on different clouds, like Amazon, Azure, and Rackspace. They actually provide the common denominator of all these APIs, and you're not getting all the features of the exact cloud that you are using and testing.
I think the same analogy goes for why you would need cloud APIs. They're just a direct means of interacting with the cloud platform, not only with the computer or the server that you're hosting your application on but, really, a means of communicating with the platform.
For example, if you look at diagnostics, getting logs and performance counters of your Azure application on a normal server, you would log in through a remote desktop and have a terminal to look at the statistics and how your application is performing and things like that. But if you have a lot of instances running on Azure, it would be difficult to log in to each machine separately.
So what you can do then is use the diagnostics API, which lets you actually gather all this diagnostic data in one location. You have one entry point into all the diagnostics of your application, whether it's running on one server or 1,000 servers.
Robert: That's a great example. You also wrote an article titled "Cost Architecting for Windows Azure," talking about the need to architect specifically for the cloud to get all the advantages. Can you talk a little bit about things people should keep in mind when architecting for the cloud?
Maarten: You need to take costs into account when you're architecting a cloud application. If you take advantage of all the specific billing situations that these platforms have to offer, you can actually reduce costs and get a highly cost effective architecture out there.
For example, don't host your data in a data center in the northern US and your application in a data center in the southern US, because you will pay for the traffic between both data centers. If your storage is in the same data center as your hosted application, you won't be paying for internal traffic.
There are a lot of factors that can help you really tune your application. For example, consider the case where you are checking a message queue, and you also have a process processing messages in this queue. Typically in this case, you would poll the queue for new messages, and if no messages are there, you would continue polling until there's a message. Then you process the message, and then you start polling again. You may be polling the queue a few times a second.
Every time you poll the queue is a transaction. Even though transactions are not that expensive on Windows Azure, if you have a lot of transactions, it can cost you substantial money. Therefore, if you poll less frequently, or with a back off mechanism that causes you to wait a little longer if there are no messages, you can drastically reduce your costs.
Robert: One of the key challenges for enterprise cloud applications is that they often can't run as a separate island. Can you talk about some of the solutions that exist when identity and data need to be shared between your data center and the cloud?
Maarten: Actually, the Windows Azure AppFabric component is completely focused on this problem. It offers the service bus, which you can use to have an intermediate party between two applications. Say you have an application in the cloud and another one in your own data center, and you don't want to open a direct firewall connection between both applications. You can actually relay that traffic through the Windows Azure service bus and have both applications make outgoing calls to the service bus, and the service bus will route traffic between both applications.
You can also integrate authentication with a cloud application. For example, if you have a cloud application where you want your corporate accounts to be able to log in, you can leverage the Access Control service. That will allow you to integrate your cloud application with your Active Directory, through the AppFabric ACS and an Active Directory federation server.
If you have an application sitting in your own data center but you need a lot of storage, you can actually just host the application in your data center and have a blob storage account on Azure to host your data. So there are a lot of integrations that are possible between both worlds.
Robert: You manage a few open source projects on CodePlex. Can you talk a little bit about the opportunity for open source on platforms like Azure or those operated by other cloud providers?
Maarten: Consider, for example, SugarCRM, which is an open source CRM system that you can just download, install, modify, view the codes, and things like that. What they actually did was take that application and also host it in a cloud. They still offer their product as an open source application that you can download and use freely. But they also have a hosted version, which they use to drive some revenue and to be able to pay some contributors.
They didn't have to invest in a data center. They just said, "Well, we want to host this application," and they hosted it in a cloud somewhere. They had a low entry cost, and as customers start coming, they will just increase capacity.
Normally, if a lot of customers come, the revenue will also increase. And by doing that, they actually now have a revenue model for their open source project, which is still free if you want to download it and use it on your own server. But by using the hosted service that's hosted somewhere on a cloud, they actually get revenue to pay contributors and to make the product better.
Robert: Maarten, I appreciate your time. Thanks for talking to the Windows Azure community.
Maarten: Sure. Thank you.