• Article

May 2015

Volume 30 Number 5

Modern Apps - End-to-End Testing in Modern Web Sites and Apps

By Rachel Appel | May 2015

Rachel AppelSoftware is more complex than ever before. Whether it’s an app for Windows, iOS, Web, Android, an Internet of Things (IoT) device or a smartwatch—that software is doing a lot. Consequently, the software must be accurate. It must work according to specs. So it’s up to developers to test their software.

Without testing, it’s difficult to verify whether the code does what it should. It’s also harder to isolate code to fix bugs. While it’s important to test, it’s also important to test for the right reasons, not just for the sake of testing or bragging rights about 100 percent coverage. Small smart tests always win over meaningless tests that cover the entire code base. This article will cover how to unit test server- and client-side code, as well as how to automate UI testing with coded UI tests (CUITs). These three basic areas of testing will test the major aspects of your software’s functionality.

Unit by Unit

Unit tests help reduce bugs and provide documentation for the code they’re testing. Regression, user acceptance, UI automation and other quality assurance measures are also important. Budgets are often tight and teams are small, so many times you have to choose one over another. It’s up to you to decide how much quality assurance goes into your app. At the very minimum, unit testing and UI tests will help keep quality high.

There are a number of open source and commercial unit-testing software packages available. Many of the popular testing suites publish a NuGet package, so you can easily download them directly into Visual Studio. A few of the popular C# testing frameworks are xUnit, nUnit, MSTest and TestDriven.NET. For JavaScript testing, there’s qUnit, JSUnit, and YUI Test.

While unit testing is often the default type of test, there are other ways to test your software. Those include UI automation or CUITs, user acceptance testing, regression testing, integration testing and all types of specialty tests. Wikipedia lists dozens of ways to test software at bit.ly/14066q8.

Good unit tests only test one function per test. The single responsibility principle (SRP) is one way to ensure your code is modular. SRP means the code focuses on one thing and does that thing well, whether it’s implementation code or the test itself. Unit tests shouldn’t directly access databases, files or other resources. They should rely on mock objects that simulate those resources and, just as with any other code, unit tests should be small.

Another way to write good tests is by adhering to test-driven development (TDD) principles. TDD is a way to develop code around a suite of tests. The developer creates tests that fail, then writes proper code to conform to the test, then refactors. I call it, “Red, Green, Refactor.” This technique helps you structure and write code consistently. And consistency is a virtue in software development. Although TDD is outside the scope of this article, look into it and try it out with your team.

This article will cover unit testing with xUnit and qUnit. I’ll also look at UI automation testing with CUITs. This way you can become familiar with the most important types of tests you’ll need.

Test Server-Side C# with xUnit

You’ll be pleased to find out how easy it is to integrate a unit-testing package into Visual Studio. Many frameworks are available as a NuGet package. For example, if you’d like to use xUnit, just launch the NuGet Package Manager, search for “xUnit,” then click to install both the core package and its test runner package. You can use xUnit to test ASP.NET Web sites, Windows Forms, Windows Presentation Foundation (WPF), Windows Store and Windows Phone apps, plus any language that compiles to the Microsoft Intermediate Language, or MSIL. You can even unit test F# using xUnit—that’s all the Microsoft development technologies.

To use xUnit, add a Class Library project to your solution. For simplicity’s sake, you might want to keep to common naming conventions and use something like ProjectName.Tests, where “ProjectName” is the name of the project you’re testing. In your test, make sure you add a reference to that project. Then name and add a class file to the test, and a class for testing something.

Figure 1 shows a sample of an xUnit test class with one test, along with the code it tests. The code in Figure 1 determines a customer’s status based on how much they’ve spent. The tests in Figure 1 ensure the code is correct.

Figure 1 Testing a Sample of xUnit Code

// Unit tests that test customer-related features
public class CustomerTests
{
  [Theory]
  [InlineData(99D)]
  [InlineData(1001D)]
  [InlineData(5001D)]
  [InlineData(10001D)]
  public void VerifyCustomerStatus(double TotalSpent) {
    // Arrange           
    var status = new Status();
    var testStatus=OrderForm.Models.Status.StatusLevel.None;
    // Act
    var currentStatus = status.GetCurrentStatus(TotalSpent);
    // Assert           
    Assert.True(testStatus <= currentStatus);
  }
}
// Logic that sets the customer status
public class Status
{
  public StatusLevel GetCurrentStatus(decimal ForAmount)
  {
    var level = StatusLevel.None;
    var amt = Convert.ToInt32(ForAmount);
    if (amt > 1000 && amt <= 5000) { level = StatusLevel.Silver; }
    else if (amt > 5000 && amt <= 10000) { level = StatusLevel.Gold; }
    else if (amt > 10000) { level = StatusLevel.Platinum; }
    return level;
  }
  public enum StatusLevel
  {
    None = 0,
    Silver = 1,
    Gold = 2,
    Platinum = 3
  }          
}

Within unit test methods, you’ll find the popular arrange, act, assert (or AAA) pattern, which is also shown in Figure 1. First arrange the test conditions, then act on those conditions and, last, assert the output.

You can use AAA on anything, but if you want quality software, you must have meaningful tests. Testing frameworks tend to come with a fleet of assertions, such as Equal, NotEqual, Same, NotSame, Contains, DoesNotContain, InRange, IsNotEmpty and several others. Of course, the exact naming or methods won’t be the same, but each framework gives you several from which to choose.

You may have noticed the [Theory] and [InlineData] attributes decorating the test class, instead of the popular [Test] attribute. When test methods have no parameters, in xUnit you can use the [Fact] attribute to mark the method as a test method. These data annotations let the test accept the TotalSpent parameter you see in the test method’s signature in Figure 1. One test will run for each [InlineData] attribute on a method, and display the results of each test in Visual Studio Text Explorer, as shown in Figure 2.

The VerifyCustomerStatus Test Method with Four Data Points in Test Explorer
Figure 2 The VerifyCustomerStatus Test Method with Four Data Points in Test Explorer

You can use a context menu on tests in Test Explorer at any time to reveal the options to run, debug, analyze, profile and so on. Any time you change code or tests, you must compile and run the tests. You can set the option to run tests automatically on each build by going to the Visual Studio Test menu, selecting Test Settings, then selecting Run Tests After Build.

You won’t get far with unit testing before you bump into the need for mock objects. Generally, when you attempt to test a database, you don’t want to modify the actual live data. Therefore, you can use mocking as a way to simulate the actual database, Web service or business object. Mocking is outside the scope of this article, so when you get to that point, search the Web for Moq, Telerik JustMock or Rhino Mocks. All are popular mocking software titles with full documentation and support.

Test Client-Side JavaScript with qUnit

QUnit is a client-side JavaScript testing framework. It’s lightweight, simple and easy to use. You can use qUnit to test any JavaScript or jQuery. Because you’re dealing with client-side JavaScript, there may be activity in the UI happening you don’t need to test. You can skip aspects such as link hovers or changes to aesthetic UI. You can also use UI automation testing to cover UI testing. Your business rules, API code and server-side validation must all have good coverage. Any JavaScript that performs logic and validation needs testing, as well.

Unlike the compiled language tests that integrate directly with Visual Studio, you’ll have to build your qUnit test harness. It’s fairly easy, though, and takes roughly 17 lines of code, which you can copy from Figure 3.

Figure 3 The qUnit HTML Test Harness

<!DOCTYPE html>
<html xmlns="https://www.w3.org/1999/xhtml">
<head>
  <title>Qunit Unit Testing</title>
  <link href="Content/qunit.css" rel="stylesheet" />
</head>
<body>
  <h1 id="qunit-header">QUnit Test Suite</h1>
  <h2 id="qunit-banner"></h2>
  <div id="qunit-testrunner-toolbar"></div>
  <h2 id="qunit-userAgent"></h2>
  <ol id="qunit-tests"></ol>
  <script src="Scripts/qunit.js"></script>      
  <script src="Scripts/logic.js"></script>
  <script src="Scripts/tests.js"></script>   
</body>
</html>

Figure 3 shows references to qunit.css, then qunit.js, logic.js and tests.js. The order is important. You must load qunit.js before your logic, and your logic before the tests. You can name logic.js whatever you want, as it’s your own code. There may be multiple .js files between qunit.js and the tests. The HTML structure of the test harness page contains a few heading tags, as well as an ordered list that displays the individual test results. All the styles for these are in the qunit.css file. As a best practice, put only the JavaScript references at the end of the Web page immediately before the closing </body> tag, as shown in Figure 3.

To test with qUnit, you can place tests in a .js file and name it tests.js. In here, there’s a call to the qUnit test function, which accepts the string name of the function to call, and an anonymous function that actually performs the test. The name passed into the test function shows up as the name of the test in the test runner, as shown in Figure 4. Instead of passing in parameters to test, use the dynamic nature of JavaScript to call the code in question multiple times using multiple asserts. Figure 5 shows the output of multiple asserts in the VerifyCustomerStatus test function.

Figure 4 QUnit Tests the getCurrentStatus Function and the Logic Itself in JavaScript

// Tests to test customer statuses
test('VerifyCustomerStatus', function () {
  var silver = getCurrentStatus(1001);
  equals(silver, "Silver", "Pass");
  var gold = getCurrentStatus(5001);
  equals(gold, "Gold", "Pass");
  var platinum = getCurrentStatus(10001);
  equals(platinum, "Platinum", "Pass");
})
// Logic to set customer statuses
function getCurrentStatus(amt) {
  if (amt > 1000 && amt <= 5000) { return "Silver"; }
  else if (amt > 5000 && amt <= 10000) { return "Gold"; }
  else if (amt > 10000) { return "Platinum"; }
  return "None";
}

Results of qUnit Tests in the Browser
Figure 5 Results of qUnit Tests in the Browser

If you’re writing Windows Store or Windows Phone apps using the Windows Library for JavaScript (WinJS), fear not. There’s a qUnitMetro you can download at bit.ly/1F2RsO7. This package lets qUnit run in the app store container just as it would in a Web page. The coding and the way you write tests are identical to qUnit, because it is indeed qUnit.

Test the UI with CUITs

When you need to automate testing for native UIs, turn to CUITs. CUITs seem to be a little-known testing feature in Visual Studio. You can create CUITs in Visual Studio for Web, Windows Store and Windows Phone apps. Visual Studio CUITs can test any UI on Windows. You don’t even need the source code to test a UI. You can configure the tests to target any running program.

There are Visual Studio templates for your basic program UIs—Web, Windows Store and Windows Phone. When you create a project from one of these templates, Visual Studio will launch you directly into the first dialog windows, shown in Figure 6, asking if you want to record new tests or edit existing ones. Select “Record actions, edit UI map or add assertions,” and a tool window will appear (see Figure 7). This same dialog and tool window will appear every time you add a new CUIT file to a project (File | New Item). At this point, you can click record, and then start interacting with your Web page or the app UI.

The Dialog Visual Studio Displays to Record or Edit Coded UI Tests
Figure 6 The Dialog Visual Studio Displays to Record or Edit Coded UI Tests

The Tool to Record and Edit Assertions
Figure 7 The Tool to Record and Edit Assertions

When you’re done, click the Add and Generate button on the bottom of the window (see Figure 8), add detailed and descriptive metadata, and let Visual Studio generate the code.

Dialogs That Create Coded UI tests
Figure 8 Dialogs That Create Coded UI tests

When you create a Coded UI project and add a Coded UI Test to it, Visual Studio generates a fair amount of code. However, Visual Studio is quite smart about how it generates the test code. The output is clean, especially for generated code. Visual Studio generates the code by recording your actions while using a particular program. If you click record on the toolbox in Figure 7, the recorder tracks what you do to generate code that performs the same action. You’re free to write this code, but it’s easier to have it generated.

Once you’re done with the Coded UI Test Builder, Visual Studio creates a few files, the first being the Coded UI test. This is a .cs file and contains a class with methods that perform the same steps a user would take to use a UI. The other files are map files: UIMap.uitest, UIMap.cs and UIMap.designer.cs. Unlike traditional Visual Studio designers that are notorious for generating too much code, this one behaves much better. It’s easy to modify tests without the designer getting in your way. At any time, you can modify the designer by right-clicking on the .uitest file and selecting Edit with Coded UI Test Builder from the options. This launches the toolbox you saw in Figure 7. Again, you’re free to write your own tests manually. Before deciding, take a look at some of the generated code that performs these four tests:

  1. It tests creating a new customer correctly
  2. It tests creating a new customer without entering a name
  3. It tests creating a new customer without entering an address
  4. It tests creating a new customer by entering an invalid postal code format

You can find the four tests in the designer file, and code in the Coded UI test class calls them in succession, as you can see in Figure 9.

Figure 9 Partial Contents of the Coded UI Test and Map Designer Files

// In the Coded UI Test C# file
[CodedUITest]
  public class CodedUITest1
  {
    [TestMethod]
    public void CodedUITestMethod1()
    {
      this.UIMap.CreateNewCustomerCorrectly();
      this.UIMap.CreateNewCustomerWithoutName();
      this.UIMap.CreateNewCustomerWithoutAddress();
      this.UIMap.CreateNewCustomerWihtoutValidPostalCode();       
    }
    // Some other framework code for Coded UI
  }
// In the Coded UI UIMap.Designer.cs file
public void CreateNewCustomerWithoutName()
    {
      #region Variable Declarations
      // Click 'Create New' link
      Mouse.Click(uICreateNewHyperlink, new Point(51, 8));
      // Type '' in 'Name' text box
      uINameEdit.Text = this.CreateNewCustomerMissingNameParams.UINameEditText;
      // Type '{Tab}' in 'Name' text box
      Keyboard.SendKeys(uINameEdit,
        this.CreateNewCustomerMissingNameParams.UINameEditSendKeys,
        ModifierKeys.None);
      // Type '234 Lane St' in 'Address' text box
      uIAddressEdit.Text =
        this.CreateNewCustomerMissingNameParams.UIAddressEditText;
      // Type '{Tab}' in 'Address' text box
      Keyboard.SendKeys(uIAddressEdit,
        this.CreateNewCustomerMissingNameParams.UIAddressEditSendKeys,
        ModifierKeys.None);
  }
}

Coded UI tests are just like any other test. They’re for interacting with the UI just as a user would. Fortunately, the Coded UI tool is great for recording your actions and generating code that does the same thing. At any time, you can discard and regenerate the tests, or modify the code without worry. Also like unit tests, you can run the Coded UI tests from Visual Studio Test Explorer. They’ll show up next to the unit tests.

Improve the Quality of Your Software

There are real benefits from the various forms of testing. It’s important to test even small Windows Store or Windows Phone apps, especially if you’re selling them in an app store. Tested apps are apps that work reliably—they have documentation and an audit trail.

You can unit test with libraries like xUnit and qUnit, and use automation testing for Web or native apps with CUITs. That’s not to mention other types of important testing such as user acceptance testing, integration testing, systems testing and so on. Adding these once you have a solid set of unit and UI tests will significantly improve the quality of your software.

Rachel Appel is a consultant, author, mentor and former Microsoft employee with more than 20 years of experience in the IT industry. She speaks at top industry conferences such as Visual Studio Live!, DevConnections, Mix and more. Her expertise lies within developing solutions that align business and technology focusing on the Microsoft dev stack and open Web. For more about Appel, visit her Web site at rachelappel.com.

Thanks to the following technical expert for reviewing this article: Oren Novotony