Howie Hilliker's blog

Visual Studio 2012: Removal of the 250 virtual user limit for load testing

Howie Hilliker — Wed, 26 Jun 2013 11:38:07 GMT

The limit on the number of virtual users that you can use in your load tests has been removed in Visual Studio 2012. You no longer have to purchase virtual user licenses to increase the number of virtual users that you can use in your load tests. Although you are now unlimited in the number of virtual users you can use, it is recommended that you use a test controller and test agents if you plan to use thousands of users. You will need to determine this for your particular environment.

For Visual Studio 2010 Ultimate, you were limited to 250 virtual users. This limitation could be overcome by either purchasing additional user pack licenses (1000 users per pack), or if with a MSDN subscription by downloading Visual Studio 2012 Feature Pack 2.

Visual Studio 2010 compatibility with Visual Studio 2012 for web performance and load testing

Howie Hilliker — Wed, 26 Jun 2013 11:27:40 GMT

See the MSDN article I wrote for further details: Upgrading Web Performance and Load Tests from Visual Studio 2010.

Web and load test project template: In Visual Studio 2012, we now have more than one test project template. In order to create new web performance or load tests, you must use the new Web Performance and Load Test project template. However, any test projects you created in Visual Studio 2010 should still work correctly. For more information, see Upgrading Tests from Earlier Versions of Visual Studio.

Running tests in Visual Studio 2012: Visual Studio 2012 no longer includes the Test View window. To run your web performance and load tests, you must now do so from either the LOAD TEST menu. To run your coded web performance tests, you must use the shortcut menu in the code editor and choose Run Coded Web Test.

SQL Server 2012 versions: To use load tests, you can use SQL Server 2012 Express LocalDB which is installed with Visual Studio 2012 Ultimate and is the default database server for load tests (including Microsoft Excel integration). If you upgrade from Visual Studio 2010 Ultimate and SQL Server Express is detected with an existing load test database, Visual Studio 2012 Ultimate will try to connect to and use it. Additionally, if SQL Server Express is detected, Visual Studio 2012 Ultimate will try to create the load test database using SQL Server Express instead of SQL Server Express LocalDB.

Upgrading test controllers: If you are using test controllers from Visual Studio for Web performance or load testing—these test controllers are not configured with Team Foundation Server—then the version of test controller must match the version of Visual Studio. For more information, see Upgrading Test Controllers from Visual Studio 2010 and Installing and Configuring Test Agents and Test Controllers.

Web Performance and Load Testing SharePoint 2010 and 2013 Applications

Howie Hilliker — Wed, 26 Jun 2013 11:10:08 GMT

I've written a new MSDN article on using web performance and load tests with SharePoint applications: Web Performance and Load Testing SharePoint 2010 and 2013 Applications.

Web performance tests can be recorded and then run and analyzed against your SharePoint applications. For example, you can create web performance tests that conduct the following operations on your SharePoint application:

SharePoint 2010

Site rendering
Document library operations
Task, calendar and library operations
Search content
Wikis
Customized web parts
Workflows
Multiple operations from the ribbon
Playing the tests against multiple sites and lists
Office Web Applications (OWA) – Uploaded and downloaded scenarios [Visual Studio 2012.2]
Manage Metadata SA and look up columns in the lists and libraries [Visual Studio 2012.2]
Integration of SharePoint and Excel Services [Visual Studio 2012.2]
For InfoPath – you can use the plug-in (tool) from CodePlex [Visual Studio 2012.2]

SharePoint 2013 (Requires [Visual Studio 2012.2])

All the scenarios listed above for SharePoint 2010
Social
SharePoint Apps

Verifying the Links in your Web Application Using a Web Performance Test

Howie Hilliker — Thu, 24 Mar 2011 13:33:00 GMT

Here are two samples on how you can use a Web performance test to verify that the links are working in your application. In the first sample, the links are verified against a single hard coded request url (https://MyStagingServer.com). In the second sample, the test reads a text file containing a list with multiple url links which you can manage and update. Both samples are simply performing new requests for each HtmlTag containing the string "https://". This can be used to verify that the all the links contained on your page are legitimate (not misspelled, or deleted). In these samples, the ParseDependentRequests is equal to false suppressing dependent request for Image tags, Script files (JavaScript, VBScript), Cascading style sheets, Body background images, Table background images, Image buttons and XML files.

I created this code for testing the Web performance and load testing topics that I author: https://msdn.microsoft.com/en-us/library/dd293540.aspx. I plan on adding these samples to the MSDN library for the product documentation at a later point. I thought I'd post my first draft here. Let me know if you have any comments or feedback.

Sample 1: Verifying Links for a Single Page

{
    using System;
    using System.Collections.Generic;
    using System.Text;
    using Microsoft.VisualStudio.TestTools.WebTesting;
    using Microsoft.VisualStudio.TestTools.WebTesting.Rules;

    public class TestLinks : WebTest
    {
        public TestLinks()
        {
            this.PreAuthenticate = true;
        }

        public override IEnumerator GetRequestEnumerator()
        {
            string linkToVerify;

            WebTestRequest request = new WebTestRequest("https://MyStagingServer.com");
            yield return request;
            request = null;

            foreach (HtmlTag tag in this.LastResponse.HtmlDocument.GetFilteredHtmlTags("a"))
            {
                linkToVerify = tag.GetAttributeValueAsString("href");
                if (! linkToVerify.Contains("https://")) continue;
                WebTestRequest linkRequest = new WebTestRequest(linkToVerify);
                linkRequest.ParseDependentRequests = false;
                yield return linkRequest ;
                linkRequest = null;
            }
        }
    }
}

Sample 2: Read URLs to test links against

namespace TestProject1
{
    using System;
    using System.Collections.Generic;
    using System.Text;
    using Microsoft.VisualStudio.TestTools.WebTesting;
    using Microsoft.VisualStudio.TestTools.WebTesting.Rules;

    public class TestLinksFromFile : WebTest
    {
        public TestLinksFromFile()
        {
            this.PreAuthenticate = true;
        }

        public override IEnumerator GetRequestEnumerator()
        {
            string linkToVerify;
            string[] requests = System.IO.File.ReadAllText(@"c:\public\urllist.txt").Split('\n');

            foreach (string url in requests) // Url entires in the urllist.txt file.
            {
                WebTestRequest request = new WebTestRequest(url);
                yield return request;
                request = null;

                foreach (HtmlTag tag in this.LastResponse.HtmlDocument.GetFilteredHtmlTags("a"))
                {
                    linkToVerify = tag.GetAttributeValueAsString("href");
                    if (!linkToVerify.Contains("https://")) continue;
                    WebTestRequest linkRequest = new WebTestRequest(linkToVerify);
                    linkRequest.ParseDependentRequests = false;
                    yield return linkRequest;
                    linkRequest= null;
                }
            }
        }
    }
}

Help us to help you! Continuous documentation updates for Web performance and load testing

Howie Hilliker — Tue, 10 Aug 2010 16:25:00 GMT

Using our continuous publishing model, which is generally done on a monthly basis, we do our best to update the documentation based on bug fixes and customer feedback. Since Visual Studio 2010 shipped in April, we have updated the docs using this model. You can have direct impact on the docs by using the button on the MSDN library toolbar and telling us both what you like and dislike. Think of it in this way... I can act as your ambassador to the team by taking your comments and chat with the developers, testers and managers within the product unit to try and resolve them. I can't promise I'll get everything resolved, but I'll do what I can as I have a real passion for doing this type of thing.

Here is what I have done for the Visual Studio Ultimate product and testing documentation since we shipped:

June Updates

In June, I added a new topic based directly from customer feedback that asked for information on the protocols and ports that are used for the test controller so he could properly configure it around his firewall. The topic is Configuring a Test Controller and Test Agent Across a Firewall. I also made several minor updates based on customer feedback in both the Visual Studio 2010 and Visual Studio 2008 content.

July updates

I had some extra time to focus on customer feedback issues for the July update. All the topics in the following table are based on customer feedback that the team has received over time.

New topic	URL
How to: Specify the Maximum Size for the Log File	https://msdn.microsoft.com/en-us/library/ff829315.aspx
Troubleshooting Network Emulation in Load Tests	https://msdn.microsoft.com/en-us/library/ff817099.aspx
Walkthrough: Adding a Loop to a Web Performance Test	https://msdn.microsoft.com/en-us/library/ff820570.aspx
How to: Run a Load Test Containing Web Performance Tests that Collects ASP.NET Profiler Data	https://msdn.microsoft.com/en-us/library/ff805009.aspx
How to: Manually Create a Load Test Performance Report Using Microsoft Word	https://msdn.microsoft.com/en-us/library/ff805009.aspx

In additon to the new topics, I also accomplished the following, which I hope will benefit you:

Updated topics:

Added information on ClassInitialize and TestInitialize attributes:

Anatomy of a Unit Test

Added C# and Visual Basic code snips displaying accessors

Unit Tests for Private, Internal, and Friend Methods

Added screenshots and information for using SQLExpress

How to: Select a Load Test Results Repository

Added codes snip for using "Tag Inner Text" validation rule and added links to validation and extraction walkthrough

Using Validation and Extraction Rules in Web Performance Tests

Added note regarding IIS application pool credentials (closed PU bug)

How to: Collect IntelliTrace Data to Help Debug Difficult Issues

Added Tip note suggesting the use of the ColorWebApp ASP.NET sample created in the walkthrough to experiment with

How to: Create a Coded Web Performance Test

Cleaned up art - numbering sequence

Running Unit Tests on Multiple Machines Using a Test Controller and Test Agents

Walkthrough: Using a Test Controller and Test Agents in a Load Test

Using Test Controllers and Test Agents with Load Tests

Configuring Test Controllers and Test Agents for Load Testing

Reported bug - waiting on resolution

How to: Specify Test Agents to Use in Load Test Scenarios

Fixed confusion over "Performance session" vs. "ASP.NET Profiler adapter" & added links

How to: Start a Performance Session on a Web Application from the Web Performance Test Editor

Load Test Analyzer Overview

Added Links to the following topics

Added link to new related walkthrough topic

Adding Loop and Branch Logic to Web Requests in Web Performance Tests

Added links to the various walkthrough topics on binding to db, csv and XML data sources.

Binding a Data Source to a Web Performance Test

Added links to "Binding a Data Source to a Web Performance Test"

Walkthrough: Adding Data Binding to a Web Performance Test

Walkthrough: Adding Data Binding to a Web Performance Test (CSV File)

Walkthrough: Adding Data Binding to a Web Performance Test (XML File)

Added link to new walkthrough on using a loop condition in a Web performance test

Web Performance Test Walkthroughs

What's New for Web Performance and Load Testing

How to: Add Loops to Web Performance Tests Requests

Added information on DLL version error work-around

How to: Create a Load Test Plug-In

How to: Create a Recorder Plug-In

How to: Create a Request-Level Plug-In

How to: Create a Web Performance Test Plug-In

Added link to new topic "How to: Run a Load Test Containing Web Performance Tests that Collects ASP.NET Profiler Data"

How to: Configure ASP.NET Profiler for Load Tests Using Test Settings

How to: Create a Test Setting for a Distributed Load Test

Added link to new topic "Troubleshooting Network Emulation in Load Tests"

How to: Add Networks Using the Load Test Editor

Specifying Virtual Network Types in a Load Test Scenario

Troubleshooting Load and Web Performance Tests

Troubleshooting Load Tests

Running Load Tests

Added link to new topic "How to: Specify the Maximum Size for the Log File"

Modifying Load Test Logging Settings

Added link to new topic "How to: Manually Create a Load Test Performance Report Using Microsoft Word"

Reporting Load Tests Results for Test Comparisons or Trend Analysis

How to: Create Load Test Performance Reports Using Microsoft Excel

August Updates

How to: Install the Required Software to Create a Video

Updated the existing topic on installing the encoder used to record video sessions. The topic describes the steps required to install the new Expression Encoder 4.0 diagnostic data adapter which is replacing the Windows Media Encoder diagnostic data adapter. This is critical because the Windows Media Encoder is no longer supported and any new video links and video recordings contained in a work item bug that are filed from an instance of Microsoft Test Manager with the new diagnostic adapter installed, will not function when accessed from a machine that does not have the new diagnostic data adapter installed.

How to Record a Video of Your Desktop as You Run Tests Using Test Settings

Updated the existing topic on recording the desktop while performing a test demonstrating the configuration changes due to using the new Expression Encoder 4.0 diagnostic data adapter.

Test Controller and Test Agent Requirements

New topic that covers the hardware and software requirements for test controllers and test agents with tables that provide guidance based on the number of virtual users and other requirement considerations.

How to: Specify the Network Adapter to Use on a Test Controller or Test Agent with Multiple Network Adapters

New topic that describes how to edit the associated XML configuration files for test controllers and test agents in situations where the machine has more than one network adapter card. This helps to resolve the following error:

o Error 8110. Can not connect to the specified controller computer or access the controller object.

How to Specify Timeout Periods for Test Controllers and Test Agents

New topic that provides information on editing the associated XML files for test controllers and test agents to modify key timeout values. The various timeout settings are covered in tables with the XML key name, descriptions and the value settings. This information is helpful when resolving errors and issues caused by timeouts. For example, the test controller’s key name AgentConnectionTimeoutInSeconds is used to specify the Number of seconds to wait for agent ping request before connection is considered lost.

September Updates

For September, my focus was on some coded UI testing topics:

Walkthrough: Creating, Editing and Maintaining a Coded UI Test" title="https://msdn.microsoft.com/en-us/library/ff977233.aspx">Walkthrough: Creating, Editing and Maintaining a Coded UI Test

New walkthrough which demonstrates creating a simple WPF application and creating a coded UI test for it. The Coded UI test is altered to demonstrate some common editing and maintenance issues. The alterations include refactoring of a control and using the UITestControl.ControlWaitForEnabled() method to force the test to wait on a control to be enabled before proceeding to the next step in the test.

Making Coded UI Tests Wait For Specific Events During Playback" title="https://msdn.microsoft.com/en-us/library/gg131072.aspx">Making Coded UI Tests Wait For Specific Events During Playback

New topic that provides information on all the various UITestControl.WaitForControlXXX() methods that you can use to avoid common coded UI test playback issues.

How to: Add UI Controls and Validation Code Using the Coded UI Test Builder

Updated the topic with screenshots to help clarify the steps used with the Coded UI Test Builder tool.

What's new in the Visual Studio 2010 Web performance and load test documentaion (Part Two - Screenshots with callout art)

Howie Hilliker — Tue, 10 Aug 2010 13:44:09 GMT

In What's new in the Visual Studio 2010 Web performance and load test documentation (Part One), I talked about how we took a new approach to how the documentation is presented by identifying key customer scenarios to create content around. This strategy was developed by soliciting feedback from numerous customers in various venues. As part of this strategy we used a new model for our high-level introduction topics. The new topics provide much more detail and tables with information about the content contained in the section. This blog post, part two is regarding how we incorporated the use of screenshots with call-outs in procedural topics (How to and Walkthrough topics). We used screenshots with callouts for two approaches:

For situations where the user can quickly learn the steps to accomplish the task
For situations where the steps are complex or not intuitive

Quickly Learn the Steps

As an example for the first approach, we used screenshots with call-outs extensively for the documentation coverage of the new manual test runner. A customer scenario we identified is a company or organization hiring temporary workers to run manual tests. Having these simple quick reference type of screenshots with call-outs should help in bringing workers up to speed and provide them with a quick reference should they forget any of the steps. To evaluate these illustrations, see the various How To topics under Running Manual Tests Using Test Runner.

Here is perhaps one of the simplest examples I created for the topic How to: Insert Additional Scenarios to an Existing Load Test. In this case, the reader can immediately identify the steps required to perform the steps to add a new scenario to their load test without having to take the time to read the full procedures.

Note We still include the full detailed list of procedural steps below the illustrations for readers who are visually impaired and to include additional information.

Complex Steps

For the second approach, we added screenshots with call-outs to steps or procedures that would benefit from having illustrations. For example, here is the screenshot with call-outs I created for the topic How to: Run a Load Test Containing Web Performance Tests that Collects ASP.NET Profiler Data:

Please let us know if you like these, dislike these or have other suggestions for the documentation by clicking the Add Content button on the MSDN library toolbar and providing us feedback.

Thanks,

Howie Hilliker

Senior Programmer Writer - Visual Studio Team Test

What's new in the Visual Studio 2010 Web performance and load test documentaion (Part One)

Howie Hilliker — Wed, 26 May 2010 15:52:33 GMT

This blog is about what we've tried to do to improve the overall documentation. If you want to see the documentation on the new features for Web performance and load testing, see: https://msdn.microsoft.com/en-us/library/ff400219.aspx.

For *Visual Studio 2010 Ultimate, the documentation team (includes me) took a new approach to how the documentation is presented. In prior releases, we generally were much more feature centric, but for Visual Studio 2010 we identified key customer scenarios and tried to create content around them. This strategy was developed by soliciting feedback from numerous customers in various venues. I for one, talked with dozens of customers and received some great feedback to help us improve the documentation. As part of this strategy we used a new model for our high-level introduction topics. The new topics provide much more detail and tables with information about the content contained in the section. For example, compare these two topics from Visual Studio 2008 and Visual Studio 2010:

Visual Studio 2008: Editing Load Tests https://msdn.microsoft.com/en-us/library/ms182583(VS.90).aspx
Visual Studio 2010: Editing Load Tests Using the Load Test Editor https://msdn.microsoft.com/en-us/library/ff406975.aspx

You can also see that the new table of contents (below) is structured around user scenarios or tasks. Now, when you search the internet (Bing of course), you'll likely land on a topic and have supporting content in the same vicinity. The tables in the new introduction topics also contain links to related tasks which should make navigating to related content easier. Additionally, we also attempted to make our topic titles more specific in order to achieve better search results.

Note Consider using the feedback option within the following topics to give me your opinion on this strategy in addition to any errors of omissions you might find.

Testing Application Performance and Stress

In part two, I'll blog about our extensive use of screenshots in the content.

* I specifically contributed to the Visual Studio Ultimate product documentaion. This new appraoch was also braodly applied to other Visual Studio content too.