Thoughts on Testing OSS Applications in Windows Azure
I’ve been spending quite a bit of time lately thinking about the best ways to test PHP applications in a Windows Azure environment. I think that, at a high level, my ideas are applicable to any OSS/Azure application that can be tested using any of a variety of testing frameworks. In this post, I want to share my ideas and get your thoughts. Larry and I will follow up in the coming weeks with language-specific details for each of these approaches.
Before I get to my ideas, let me step up on my soap box and say IF YOU ARE NOT TESTING YOUR APPS, YOU SHOUD BE! I don’t feel the need to justify that exclamation…I’ll let Jeff Atwood justify it though: I Pity the Fool Who Doesn’t Write Unit Tests. If you aren’t actually testing your apps now, thinking about testing your apps is, well, better than nothing (but just barely).
Stepping down now…
So, here are my thoughts on how you might test an OSS/Azure application…from “easiest” to “best”. All of these approaches assume that you are deploying a testing framework and test suite with your application.
1. RDP to staging instance and run command-line tests. This approach involves enabling RDP access to a deployment (I’m assuming deployment to a staging slot), opening a command prompt, and running your tests as (many of you) normally would. The benefit to this approach is that you likely won’t have to deviate much from your normal testing routine. The drawbacks are that you have to manually run your tests and disable RDP access to your instances after you push to production.
Edit: See this post for details on this approach: Running PHPUnit in Windows Azure.
2. Use a web front-end for running tests. This approach assumes you have a web front-end for your testing framework (like VisualPHPUnit is for PHPUnit). The benefits here are that you can run your tests at any time, in exactly the same way you have been (assuming you’ve been using a web front-end all along). The major drawback is that you will need password-protect your test directory, which takes a bit of figuring out if your not used to using IIS.
3. Use a startup script to automatically run tests upon deployment. This approach involves the most work, but I think it is the most powerful, mostly because it’s automated. The idea is that you specify a startup task in your service definition that runs your test suite and writes the results to a file. To get your hands on that file, you’ll need to configure Azure Diagnostics to write that file to your Azure Storage account. To avoid running the tests in production (which you don’t want to do if the tests take a while to run), you’ll need to write code that determines your Azure environment (staging or production) and only runs your tests in the staging environment.
Edit: See this post for details on this approach: Automating PHPUnit Tests in Windows Azure.
What are your thoughts on these approaches? Are there others that I’m missing?
As I mentioned earlier, we’ll follow up with more how-to details for these approaches in the coming weeks.
Thanks.
-Brian