September 2010

Volume 25 Number 09

Test Run - Request-Response Testing Using IronPython

By James McCaffrey | September 2010

I’m a big fan of using the Python programming language for several types of lightweight test automation tasks. In this month’s column, I show you how to use IronPython—a Microsoft .NET Framework-compliant implementation of Python—to perform HTTP request-response testing for ASP.NET Web applications.

Specifically, I create a short test harness script that simulates a user exercising an ASP.NET application. The IronPython harness programmatically posts HTTP request information to the application on a Web server. It then fetches the HTTP response stream
and examines the HTML text for an expected value of some sort to determine a pass/fail result. In addition to being a useful
testing technique in its own right, learning how to perform HTTP request-response testing with IronPython is an excellent way to learn about the IronPython language.

This column assumes you have basic familiarity with ASP.NET technology and intermediate scripting skills with a language such as JavaScript, Windows PowerShell, VBScript, Perl, PHP or Ruby, but I don’t assume you have any experience with Python. However, even if you’re new to ASP.NET and scripting, you should still be able to follow the column without too much difficulty. The best way to see where I’m headed is to examine the screenshots in Figures 1and 2.

Figure 1illustrates the example ASP.NET Web application under test. The system under test is a simple but representative Web
application named MiniCalc. I deliberately kept my ASP.NET Web application under test as simple as possible so that I don’t obscure the key points in the IronPython test automation. Realistic Web applications are significantly more complex than the dummy MiniCalc application shown in Figure 1, but the IronPython testing technique I describe here easily generalizes to complex applications. The MiniCalc Web application accepts two numeric values, an indication to add or multiply the values, and the number of decimals to display the answer to. The app then sends the values to a Web server, where the result is computed. The server creates the HTML response and sends it back to the client browser, where the result is displayed to the number of decimal places specified by the user.

Figure 1 MiniCalc Web App Under Test

Figure 2shows an IronPython test harness in action. My script is named harness.py and doesn’t accept any command-line arguments. For simplicity, I have hard-coded information, including the URL of the Web application and the name of the test case input file. My harness begins by echoing the target URL of https://localhost/MiniCalc/Default.aspx. In test case 001, my harness programmatically posts information that corresponds to a user typing 1.23 into control TextBox1, typing 4.56 into TextBox2, selecting RadioButton1 (to indicate an addition operation), selecting 4 from the DropDownList1 control and clicking on Button1 (to calculate). Behind the scenes, the harness captures the HTTP response from the Web server and then searches the response for an indication that 5.7900 (the correct sum of 1.23 and 4.56 to 4 decimals) is in the TextBox3 result control. The harness tracks the number of test cases that pass and the number that fail and displays those results after all test cases have been processed.

Figure 2 Testing the App Using IronPython

In the sections that follow, I briefly describe the Web application under test so you’ll know exactly what’s being tested. Next, I walk you through the details of creating lightweight HTTP request-response automation using IronPython. I wrap up by presenting a few opinions about when the use of IronPython is suitable and when other techniques are more appropriate. I think you’ll find the information interesting, and a useful addition to your testing toolset.

The Application Under Test

Let’s take a look at the code for the MiniCalc ASP.NET Web application, which is the target of my test automation. I created the app using Visual Studio 2008. After launching Visual Studio with administrator privileges, I clicked on File | New | Web Site. In order to avoid the ASP.NET code-behind mechanism and keep all the code for my Web application in a single file, I selected the Empty Web Site option. I specified https://localhost/MiniCalc in the location field. I decided to use C# for the MiniCalc application, but the test harness I’m presenting in this column works with ASP.NET applications written in VB.NET, and with slight modifications the harness can target Web applications written using technologies such as classic ASP, CGI, PHP, JSP and Ruby. I clicked OK on the New Web Site dialog to generate the structure of my Web application. Next, I went to the Solution Explorer window, right-clicked on the MiniCalc project name and selected Add New Item from the context menu. I then selected Web Form from the list of installed templates and accepted the Default.aspx file name. I cleared the “Place code in separate file” option and then clicked the Add button. Next, I double-clicked on the Default.aspx file name in Solution Explorer to edit the template-generated code. I deleted all the template code and replaced it with the code shown in Figure 3.

Figure 3 MiniCalc Code

<%@ Page Language="C#" %>
<script runat="server">
  private void Button1_Click(object sender, System.EventArgs e)
  {
    double alpha = double.Parse(TextBox1.Text.Trim());
    double beta = double.Parse(TextBox2.Text.Trim());
    string formatting = "F" + DropDownList1.SelectedValue;
    if (RadioButton1.Checked)
      TextBox3.Text = Sum(alpha, beta).ToString(formatting);
    else if (RadioButton2.Checked)
      TextBox3.Text = Product(alpha, beta).ToString(formatting);
    else
     TextBox3.Text = "Select method";
  }
  private static double Sum(double a, double b)
  {
    return a + b;
  }
  private static double Product(double a, double b)
  {
    return a * b;
  }
</script>
<html>
  <head>
    <style type="text/css">
     fieldset { width: 16em }
     body { font-size: 10pt; font-family: Arial }
    </style>
    <title>Default.aspx</title>
    </head>
  <body bgColor="#ccffff">
    <h3>MiniCalc by ASP.NET</h3>
    <form method="post" name="theForm" id="theForm" runat="server"             action="Default.aspx">
      <p><asp:Label id="Label1" runat="server">Enter number:&nbsp</asp:Label>
      <asp:TextBox id="TextBox1" width="100" runat="server" /></p>
      <p><asp:Label id="Label2" runat="server">Enter another:&nbsp</asp:Label>
      <asp:TextBox id="TextBox2" width="100"  runat="server" /></p>
      <p></p>
      <fieldset>
        <legend>Arithmentic Operation</legend>
        <p><asp:RadioButton id="RadioButton1" GroupName="Operation"                 runat="server"/>Addition</p>
        <p><asp:RadioButton id="RadioButton2" GroupName="Operation"                 runat="server"/>Multiplication</p>
        <p></p>
      </fieldset>
    <p>Decimals: &nbsp;&nbsp;
      <asp:DropDownList ID="DropDownList1" runat="server">
      <asp:ListItem>3</asp:ListItem>
      <asp:ListItem>4</asp:ListItem>
      </asp:DropDownList>
    </p>
      <p><asp:Button id="Button1" runat="server" text=" Calculate "            onclick="Button1_Click" /></p>
      <p><asp:TextBox id="TextBox3" width="120"  runat="server" />
    </form>
  </body>
</html>

To keep my source code small in size and easy to understand, I’m taking shortcuts such as not performing error-checking and combining server-side controls (such as ) with plain HTML (such as

). The most important parts of the code in Figure 3 are the IDs of the ASP.NET server-side controls. I use default IDs Label1 (user prompt), TextBox1 and TextBox2 (input for two numbers), RadioButton1 and RadioButton2 (choice of addition or multiplication), DropDownList1 (number of decimals for the result), Button1 (calculate) and TextBox3 (result). To perform automated HTTP request-response testing for an ASP.NET application using the technique I present here, you must know the IDs of the application’s controls. In this situation, I have the source code available because I’m creating the application myself; but even if you’re testing a Web application you didn’t write, you can always examine the application by using a Web browser’s View Source functionality.

To verify that my Web application under test was built correctly, I hit the key. I clicked OK on the resulting Debugging Not Enabled dialog to instruct Visual Studio to modify the Web application’s Web.config file. Visual Studio then launched Internet Explorer and loaded MiniCalc. Notice that the action attribute of my

element is set to Default.aspx. In other words, every time a user submits a request, the same Default.aspx page code is executed. This gives my MiniCalc Web application the feel of a single application rather than a sequence of different Web pages. Because HTTP is a stateless protocol, ASP.NET accomplishes the application effect by maintaining the Web application’s state in a special hidden value type, called the ViewState. As you’ll see shortly, dealing with an ASP.NET application’s ViewState is one of the keys to programmatically posting data to the application.

ASP.NET Request-Response Testing with IronPython

Let’s go over the IronPython test harness program that produced the screenshot in Figure 2. IronPython is a free download available from CodePlex, the Microsoft-sponsored open source project, at ironpython.codeplex.com. I’m using version 2.6.1, which runs on version 2.0 of the .NET Framework and runs on any machine that supports that version of the Framework. The overall structure of my test harness script is presented in Figure 4.

Figure 4 Test Harness Structure

set up imports
define helper functions
try:
  initialize variables
  open test case data file
  loop
    read a test case from file
    parse test case data
    determine ViewState
    determine EventValidation
    construct request string
    send request string to app
    fetch response from app
    determine if response has expected result
    print pass or fail
  end loop
  close test case data file
  print summary results
except:
  handle exceptions

As you can see, my harness script is simple and is driven by an external test case data file. That test case data file is named testCases.txt and consists of:

001|1.23|4.56|RadioButton1|4|clicked|5.7900
002|1.23|4.56|RadioButton2|4|clicked|5.7900
003|2.00|3.00|RadioButton1|4|clicked|5.0000

Each line represents one test case and has seven fields delimited by a “|” character. The first field is a test case ID. The second and third fields are inputs to TextBox1 and TextBox2. The fourth field encodes whether to request Addition or Multiplication. The fifth field is the value for the Decimals DropDownList control. The sixth field (“clicked”) is the Button1 event. The seventh field is the expected result, which should appear in TextBox3. The second test case is deliberately incorrect just to demonstrate a test case failure. For the lightweight testing approach I’m describing here, a simple text file to hold test case data is often a good choice. If I had wanted to embed my test case data directly in the harness script, I could have done so using an array of strings like:

testCases = ['001|1.23|4.56|RadioButton1|4|clicked|5.7900',
             '002|1.23|4.56|RadioButton2|4|clicked|5.7900',
             '003|2.00|3.00|RadioButton1|4|clicked|5.0000']

and then iterated through each test case like:

for line in testCases:
  ...

Python also has a list type that can be used to store test case data.

The first three lines of my IronPython test harness are:

# harness.py
import sys
import clr

Comments in Python begin with the “#” character and extend to end of line. The “import sys” statement allows my script to access resources in the special IronPython sys module.

The locations of these resources can be listed by issuing a sys.path command from the IronPython interactive console. The “import clr” statement allows my script to access and use core .NET CLR functionality.

My next six statements explicitly enable the .NET functionality my harness uses:

from System import *
from System.IO import *
from System.Text import *
from System.Net import *
clr.AddReference('System.Web')
from System.Web import *

The first line here imports System and is similar to the “using System” statement in a C# program. The “import clr” statement includes the System namespace, so I could omit the “from System import *” statement but I prefer to leave it in as a form of documentation. The next three statements bring the System.IO (for file operations), System.Text (for byte conversion) and System.Net (for request-­response functionality) namespaces into scope. The “clr.AddReference(‘Sys­tem.Web’)” statement brings the System.Web namespace into scope. This namespace isn’t directly accessible by default, so I must use the AddReference method before I issue the “from System.Web import *” statement so I can access URL-encoding methods.

Next, I define a helper method to fetch the ViewState information for the Web application under test:

def getVS(url):
  wc = WebClient()
  bytes = wc.DownloadData(url)
  html = Encoding.ASCII.GetString(bytes)
  start = html.IndexOf('id="__VIEWSTATE"', 0) + 24
  end = html.IndexOf('"', start)
  vs = html.Substring(start, end-start)
  return vs

Remember that because HTTP is a stateless protocol, ASP.NET gives the effect of being a stateful application by maintaining the application’s state in a hidden value called ViewState. A ViewState value is a Base64-encoded string that maintains the state of an ASP.NET page through multiple request/response roundtrips. Similarly, an EventValidation value was added in ASP.NET 2.0 and is used for security purposes to help prevent script-insertion attacks. These two mechanisms are key to programmatically posting data to an ASP.NET Web application.

In Python, you must define functions before you call them in a script. Functions are defined using the def keyword. The helper function first instantiates a WebClient object. Next the DownloadData method sends an HTTP request to the Web application given by parameter url, and fetches the HTTP response as an array of byte values. I use the GetString method to convert bytes to a string named html. A ViewState element looks like this:

<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE"
 value="/wEPDwULLTEwNjU=" />

So, to extract the value, I first determine the location of the substring id="__VIEWSTATE "and then add 24 characters. This approach is brittle in the sense that the technique will break if ASP.NET changes the format of the ViewState string, but because this is lightweight automation, simplicity trumps robustness. I determine where the terminating double-quote character is and then I can use the Substring method to extract the ViewState value. Unlike most languages, which use tokens such as begin...end or {...} to delimit code blocks, Python uses indentation. If you are new to Python programming, this may seem odd at first, but most engineers I’ve talked to say they quickly become used to the syntax. Python is supported by a collection of modules, so an alternative to using .NET methods to retrieve the ViewState value is to use functions in the native Python urllib module.

After defining the getVS helper function, I define a helper function to get the EventValidation value:

def getEV(url):
  wc = WebClient()
  bytes = wc.DownloadData(url)
  html = Encoding.ASCII.GetString(bytes)
  start = html.IndexOf('id="__EVENTVALIDATION"', 0) + 30
  end = html.IndexOf('"', start)
  ev = html.Substring(start, end-start)
  return ev

I use the same technique to extract EventValidation as I do to extract ViewState. Notice that Python is a dynamically typed language, so I don’t specify the data types of my parameters, variables and objects. For example, DownloadData returns a byte array and IndexOf returns an int, and the IronPython interpreter will figure these types out for me. Defining two functions, getVS and getEV, requires two roundtrip calls to the Web application under test, so you may want to combine the two helper functions into a single function and name the helper something like getVSandEV(url).

After defining my helper functions, my actual automation begins:

try:
  print '\nBegin IronPython Request-Response testing'
  url = 'https://localhost/MiniCalc/Default.aspx'
  print '\nURL under test = ' + url + '\n'
  testCases = 'testCases.txt'
  ...

Unlike some languages that require an entry point, such as a Main method, Python script execution simply begins with the first executable statement. I use the try keyword to catch any exceptions, then I print a startup message. Python allows the use of either single quotes or double quotes to define string literals, and escape sequences such as \n can be embedded with either delimiter. To disable escape sequence evaluation, you can prepend string literals with lowercase r (“raw”), for example: file = r'\newFile.txt'. I hard-code the URL of the application under test and display that value to the shell. If I wanted to read the URL from the command line, I could have used the built-in sys.argv array, for example: url = sys.argv[1]. Python uses the “+” character for string concatenation. I also hard-code the name of my test case data file, and because I don’t include file path information, I make the assumption that the file is in the same directory as the IronPython script.

Next, I set up counters, open my test case data file and start iterating through the file:

... 
numPass = numFail = 0
fin = open(testCases, 'r')
for line in fin:
  print '==========================================='
  (caseid,value1,value2,operation,decimals,action,expected) = 
    line.split('|')
  ...

Python has several idioms I really like, including multiple variable assignment and concise file operations syntax. The “r” argument in the call to the open function means to open the file for reading. The “for line in fin” statement enumerates through the file one line at a time, assigning the current line of input to variable “line.” Another neat Python construct is the tuple idiom. Tuples are denoted using left and right parentheses and the tuple values are delimited by “,” characters. Here, I call the split method and assign each resulting token to variables caseid, value1, value2, operation, decimals, action and expected, all in a single statement. Very pretty.

Next, I begin to build up the data to post to the Web application under test:

... 
expected = 'value="' + expected.Trim() + '" id="TextBox3"'
data = 'TextBox1=' + value1 + '&TextBox2=' + value2 + '&Operation=' +
 operation + '&DropDownList1=' + decimals + '&Button1=' + action
print 'Test case: ' + caseid
print 'Input    : ' + data
print 'Expected : ' + expected
...

I slightly modify the expected variable to resemble something like:

value="5.7900" id="TextBox3"

So when I search the HTTP response, I’ll be more specific than simply searching for “5.7900.” I associate my test case input values to their respective controls as name-value pairs connected by the “&” character. The first two name-value pairs of the post string simply set TextBox1 and TextBox2 to value1 and value2 from the test case data. The third name-value pair (for example, Operation=RadioButton1) is how I simulate a user selecting a RadioButton control, in this case, the control that corresponds to addition. You might have guessed incorrectly (as I originally did) that the way to set the radio button would be to use a syntax like RadioButton1=checked. But RadioButton1 is a value of the Operation control, not a control itself. The fifth name-value pair, Button1=clicked, is somewhat misleading. I need to supply a value for Button1 to indicate that the control has been clicked, but any value will work, so I could’ve used Button1=foo (or even just Button1=) but Button1=clicked is more descriptive, in my opinion. I echo the values I’ve parsed from the test case data to the command shell, making use of the “+” string concatenation operator.

Next, I deal with the ViewState and EventValidation values:

... 
vs = getVS(url)
ev = getEV(url)
vs = HttpUtility.UrlEncode(vs)
ev = HttpUtility.UrlEncode(ev)
data = data + "&__VIEWSTATE=" + vs + "&__EVENTVALIDATION=" + ev
...

I call the getVS and getEV helper functions defined earlier. The ViewState and EventValidation values are Base64-encoded strings. Base64 encoding uses 64 characters: uppercase A-Z, lowercase a-z, digits 0-9, the “+” character and the “/” character. The “=” character is used in Base64 for padding. Because some of the characters used in Base64 aren’t permitted in an HTTP request stream, I use the HttpUtility.UrlEncode method from the System.Web namespace to convert troublesome characters into a three-character sequence beginning with the “%” character.

For example, a raw “>” character is encoded as %3D and a blank space is encoded as %20. When a Web server receives an HTTP request containing any of these special three-character sequences, the server decodes the sequences back to raw input. After encoding, I append the ViewState and EventValidation values onto the Post data.

Next, I process the Post data to prepare it for an HTTP request:

...  
buffer = Encoding.ASCII.GetBytes(data)
req = HttpWebRequest.Create(url)
req.Method = 'POST'
req.ContentType = 'application/x-www-form-urlencoded'
req.ContentLength = buffer.Length
...

I use the GetBytes method of the System.Text namespace to convert my Post data into a byte array named buffer. Then I create a new HttpWebRequest object using an explicit Create method. I supply values to the Method, ContentType and ContentLength properties of the HttpWebRequest object. You can think of the value of the ContentType as a magic string that’s necessary to Post an HTTP Web request.

Next, I send the HTTP request:

... 
reqst = req.GetRequestStream()
reqst.Write(buffer, 0, buffer.Length)
reqst.Flush()
reqst.Close()
...

The programming pattern for sending a request may seem a bit odd if you’re new to the technique. Rather than use an explicit Send method of some sort, you create a Stream object and then use a Write method.

The Write method requires a byte array, the index in the array to begin writing and the number of bytes to write. By using 0 and buffer.Length, I write all bytes in the buffer. The Write method doesn’t actually send the Post to the Web server; you must force a send by calling the Flush method. The Close method actually calls the Flush method, so my call to Flush isn’t required in this situation, but I include the call for clarity.

After sending the HTTP request, I fetch the associated response:

... 
res = req.GetResponse()
resst = res.GetResponseStream()
sr = StreamReader(resst)
html = sr.ReadToEnd()
sr.Close()
resst.Close()
...

The GetResponse method returns the HttpWebResponse object associated with an HttpWebRequest object. The response object can be used to create a Stream, and then I associate the Stream to a StreamReader object and use the ReadToEnd method to fetch the entire response stream as a single string. Although the underlying .NET Framework cleanup mechanisms would eventually close the StreamReader and Stream objects for you, I prefer to explicitly close them.

Next, I examine the HTTP response for the test case expected value:

... 
if html.IndexOf(expected) >= 0:
  print 'Pass'
  numPass = numPass + 1
else:
  print '**FAIL**'
  numFail = numFail + 1
...

I use the IndexOf method to search the HTTP response. Because IndexOf returns the location of the beginning of the search string within the reference string, a return value >= 0 means the search string is in the reference string. Note that unlike many languages, Python doesn’t have increment or decrement operators such as ++numPass.

Next, I finish my script:

... 
    print '===============================\n'
    # end main processing loop
  fin.close()
  print '\nNumber pass = ' + str(numPass)
  print 'Number fail = ' + str(numFail)  
  print '\nEnd test run\n'
except:
  print 'Fatal: ', sys.exc_info()[0]
# end script

I place a comment at the end of the “for” loop that iterates through each line of the test case data file as an aid to making sure my indentation is correct. Once outside the loop, I can close the test case data file and print the pass and fail counters. Notice that because numPass and numFail were inferred to be type int, I must cast them to type “string” using the Python str function so I can concatenate them. My harness finishes by handling any exceptions thrown in the try block simply by printing the generic exception message stored in the built-in sys.exec_info array.

Quick Automation with a Short Lifetime

The example I presented here should give you enough information to write IronPython test automation for your own Web applications. There are several alternatives to using IronPython. In my opinion, IronPython is best suited for lightweight test automation where you want to create the automation quickly, and the automation has a short (a few days or weeks) intended lifetime. My technique does have some limitations—in particular it can’t easily deal with Web applications that generate popup dialog boxes.

Compared to other scripting languages, one of the advantages of using IronPython for lightweight test automation is that you can use the interactive console to issue commands to help write your script. Additionally, there are several nice editors available, and you can even find IronPython SDKs that let you integrate Iron­Python into Visual Studio. Personally, when I’m going to write Web application test automation, if I want to develop a relatively short harness, I consider using IronPython and Notepad. If I’m going to write a harness that has more than three pages of code, however, I generally use C# and Visual Studio.

Dr. James McCaffrey works for Volt Information Sciences Inc. where he manages technical training for software engineers working at the Microsoft Redmond, Wash., campus. He has worked on several Microsoft products, including Internet Explorer and MSN Search. McCaffrey is the author of “.NET Test Automation Recipes: A Problem-Solution Approach” (Apress, 2006). He can be reached at jammc@microsoft.com.

Thanks to the following technical experts for reviewing this article: Dave Fugate and Paul Newson