Load Agent and Load Controller Installation and Configuration Guide

Microsoft® Visual Studio Team Test Load Agent Products

The Team Test Rig enables Team System users to run tests on one or more remote computers. A rig is made up of a single test controller and one or more test agents. The test controller can be installed on either one or two computers, and like Team Foundation Server, has both application and data tiers. A single test controller is used to coordinate the execution of multiple test agents running tests on multiple computers.

Supported Operating Systems

The following operating systems are supported for the agent and controller:

2008 release
Controller	Windows Server 2008Windows Server 2003 (SP1)Windows VistaWindows XP Professional (SP2)
Agent	Windows Server 2008Windows Server 2003 (SP1)Windows VistaWindows XP Professional (SP2)

 

2005 release
Controller	Windows Server 2003 (RTM)Windows XP Professional (SP2)
Agent	Windows Server 2003 (RTM)Windows XP Professional (SP2)Windows 2000 (SP4)

Hardware Requirements

For the test agent, memory and CPU typically control how much load can be generated. The test controller does not generate load, but does collect load test statistics from the agents and performance counter data from the agents and system under test. Therefore, database used by the controller requires considerable resources both for the amount of data stored and the number of agents involved in the load test scenario. Use the following table as a guide for determining hardware requirements for the controller and agent:

 

	Controller Application Tier		Controller Data Tier		Controller Application/Data Tier
	Min	Recommended	Min	Recommended	Min	Recommended
CPU	600 Mhz	1 GHz	600 Mhz	1 GHz	600 Mhz	1 GHz
Disk – system	1 GB	1 GB	1 GB	1 GB	1 GB	1 GB
Disk - install	1 GB	8 GB	8 GB	40 GB	8GB	48 GB
Memory	256 MB	512 MB	512 MB	1 GB	512 MB	1 GB

 

Agent	Minimum	Recommended
CPU	600 MHz	2 GHz
Disk - system	1 GB	1 GB
Disk - install	1 GB	5 GB
Memory	256 MB	1 GB

 

By default, SQL Express is installed with the controller and is used as the default store for load test results. Therefore, the controller runs an instance of SQL Express.  For more information, see “SQL Sizing Considerations” later in this document.

Supported Configurations

Visual Studio, the Test Controller, SQL, and the Test Agent can all reside on the same computer or be installed on different computers. Typical computer configurations are:

Recommended Configuration:

· Computer 1: Visual Studio

· Computer 2: Controller, SQL Express

· Computer 3-n: Agents

Note   Installing an agent on the same computer as the controller can interfere with results collection. If you choose to do this, use the Agent Weighting property to reduce the amount of load generated by the agent installed on the same computer as the controller.

It is recommended that you install the agents on their own computer. Anything that is processing at the same time the agent is running affects the accuracy of the test. Agent weighting reduces the impact, but inaccuracies are still introduced.

The following additional configurations are available. They are listed in order of recommendation.

Alternative Configuration 1

· Computer 1: Visual Studio, Controller, SQL Express

· Computer 2-n: Agents

Alternative Configuration 2

· Computer 1: Visual Studio

· Computer 2: Controller, Agent

· Computer 3: SQL Express

· Computer 4-n: Agents

Alternative Configuration 3

· Computer 1: Visual Studio

· Computer 2: Controller, SQL Express, Agent

· Computer 3-n: Agents

Performance Considerations

Here are some additional considerations to consider when sizing the hardware:

 

	Agent	Controller Application Tier	Controller Data Tier	Controller AT/DT
CPU	Depending on the test, the CPU is frequently the limiting factor	Not heavily used	Not heavily used	Not heavily used
Disk	Not heavily used	Not heavily used	10 GB space required for 24 hours of test data	10 GB space required for 24 hours of test data
Memory	Depending on the test, memory will be the limiting factor	Not heavily used	Heavily used by SQL	Heavily used by SQL

 

The amount of load that a specific load agent can generate varies widely from test to test. Most tests are bound by the CPU. CPU use is directly proportional to requests per second (RPS). For other load tests, memory is the limiting factor. The RPS you can expect to drive from a load agent depends upon many factors. These include the following:

· User load

· Think time

· Authentication scheme

· Size of requests and responses

· Response time

· Level of response validation

· Test Type under load (either Web Test or Unit Test)

Think time is the primary factor for determining the number of users on a CPU-bound test. Changing think times from 2 seconds to 10 seconds allows you simulate 5 times more users, but the RPS being generated will be the same. If your goal is to simulate real users, set the think time to a value that reflects how you believe users will behave on your Web site. Increasing think time and the number virtual users will not necessarily place additional stress on your Web application.

Agents can be bound by memory on tests that use the Connection Per User connection mode. Two connection modes can be configured in the load test run settings. In Connection Pool mode (the default), connections are pooled, but each user still uses two connections when active. In this mode, all virtual users are multiplexed over the connection pool. This allows you to have 1000 active virtual users sharing 100 connections. In Connection Per User mode, each user has a connection that consists of two actual connections open to the server.

If load testing against a typical ASP.NET application with 3 to 5 second think time using Web tests, you can simulate around 1000 users from a single-processor load test agent with a 2 GHz CPU and 1 GB RAM (recommended configuration). The number of users supported is a function of the think time. With longer think times, more users can be supported.

SQL Sizing Considerations

By default, SQL Express is installed on the controller and is used by the controller as the default SQL store for load test results. The SQL Express database is license-limited to store 10 GB of data, which is around 24 hours of load test data for a typical load test. The space that is required for load test data varies greatly depending on the test.

During a load test, samples are collected for each counter instance on each computer. Therefore, the amount of space that is required in the database depends on the following factors: the number of counters collected, the number of computers involved in the test, and the number of samples taken, as controlled by the sample rate.

If appropriate, consider using a separate database to store the load test data. The database can be stored on either the controller computer or on a different computer. To change the data store, submit the SQL commands contained in the .sql file to the SQL server instance you want to use for the load test results store. These are two ways to do this. One way is to use the command sqlcmd from a command prompt and specify the options needed to connect to the desired database. Use the –i option to specify the path to loadtestresultsrepository.sql. Another way is to use one of the GUI interfaces to SQL, such as query analyzer, and open the .sql file and submit the connects.

Test Rig Software Requirements

Setup installs the following required components:

 

Component	Requirements	Version
Controller	SQL Express	2005
	.NET Framework	2.0
Agent	.NET Framework	2.0

 

Test Rig User Accounts

The following types of accounts exist in the rig. Before installing, create the required accounts on the rig computers.

 

Account Name	Description	Requirements
Setup User	User who installs the controller and agent.	This user must be an Administrator on the computer they are running Setup on, and be an Administrator on the Controller.
Controller Service account	User the controller is running as. During Setup, you are prompted for this user account.	This user must have access to read performance counters from computers under test during a load test because the controller service collects performance counters during a load test. If running in Workgroup (non-domain), a local computer account that has the same user name and password must exist on all agents. To restart the rig, the Controller Service User is added to the Agent Administrators group during Agent Setup.
Agent Service account	User the agent service is running as. During Setup, you are prompted for this user name.	By default, tests are run as this user. (There is a way to run tests as a different user). Therefore, this user needs the access as required by the tests you are running. If you turn on Agent Logging, this user needs write access to the installation directory.

 

Requirements for Workgroups

If the computers in a rig are running in Work Group mode, that is, they are not in a domain, or they are running in different non-trusted domains, create local computer accounts on the controller. The accounts should have a matching password for each user who will access the controller, including the Agent service users.

For example, you have three computers, Controller1, Agent1, and Agent2. You must create local computer accounts on each computer for the agent service user and controller user. You also must make sure an administrator account exists on all three computers with the same username and password.

 

Computer Name	User Account	Password
Controller1	ControllerService	ControllerServicePassword
Agent1	ControllerService	ControllerServicePassword
Agent2	ControllerService	ControllerServicePassword
Controller1	AgentService	AgentServicePassword
Agent1	AgentService	AgentServicePassword
Agent2	AgentService	AgentServicePassword

 

Note: If you are running on Windows XP in a work group, you must turn off Simple File Sharing. See this KB for info on how to do that: https://support.microsoft.com/kb/307874

Open Windows Explorer, choose Tools, and then choose Folder Options, Next, select the View tab, and turn off the "Use Simple File Sharing (Recommended)" option.

Accessing the Controller

After the controller is installed, access to the controller is limited to members of the TeamTestControllerUsers and TeamTestControllerAdmins groups that were created during Setup, and to the Administrators group. Add appropriate users and/or groups to these groups to allow them to access the controller. Members of the TeamTestControllerAdmins group or the Administrators group can administer the controller by clicking the Test menu, and then choosing Administer Test Controller in Visual Studio. Members of the TeamTestControllerAdmins group must also be power users or administrators on the controller computer.

Team Test Load Controller Installation

Warning   Set up the Controller before setting up any agents.

Note   Load controllers cannot be installed on domain controllers.

1.      Start setup.exe located in the vs\controller folder.

The Welcome to Setup page appears.

2.      Click Next.

The End-User License Agreement and Product Key page appears.

3.      Review the End-User License Agreement (EULA) and select I accept the terms of the license agreement, if appropriate.

4.      Enter the product key information and then click Next.

The Destination Folder page appears.

5.      Click Browse to select a different destination folder, or click Next to use the default folder.

The Service Account page appears.

6.      Enter the account name and password for the controller service account and then click Next.

The Ready to Install page appears.

7.      Review the installation information and then click Install to continue.

The Installing Components page appears.

8.      Upon successful installation, the Setup Completed Successfully page appears. Click Finish to complete the Setup program.

During Setup, you are prompted for the controller service account. The controller service account collects performance counters on other computers during a load test. Use a domain account that can easily be granted access to many computers. Grant the controller user rights to monitor performance on all the systems under test for the built-in performance monitoring capabilities to work.

Setup creates three local groups:

· TeamTestControllerAdmins

· TeamTestControllerUsers

· TeamTestAgentService

Add users who run tests to TeamTestControllerUsers. Add users who administer the test rig to TeamTestControllerAdmins. Agent setup automatically adds the users who Agent Services run as to TeamTestAgentService.

NoteThe user running the controller setup must be a member of local Administrators group.

The controller runs as a Windows Service on the computers on which it is installed. To view the properties of the service use Control Panel. In Control Panel, choose Administrative Tools and then choose Services. The name of the controller service is the Visual Studio Team Test Controller. The controller services can be started and stopped like other services in Windows.

The controller and agents can be configured and monitored by using the Administer Test Controller dialog box, available on the Test menu. For more information, see “How to: Administer the Rig” in the product documentation.

By default, the controller uses SQL Express to store load test results. SQL Express is license limited to store 4GB of data, which is approximately 10 hours of load test data. If you choose to use another SQL database to store the results, you can set up the Load Test Results Store after you have installed the controller. For more information, see “How to: Configure a Results Store using SQL” in the product documentation.

After you have configured your Load Test Results Store, you must indicate to the controller where the results are to be stored. For more information, see “How to: Identify a Load Test Results Store” in the product documentation.

Load Controller Firewall Exceptions

If you are running Windows Firewall, the controller setup adds the following exceptions to your firewall:

 

Exception	Reason
QTControllerService.exe	Enable communication with the controller
SQLExpr.exe	Enable Visual Studio Load Test Results Analyzer to access load test data stored in SQL Express
SQLBrowser.exe	Enable Visual Studio Load Test Results Analyzer to access load test data stored in SQL Express
File and Printer Sharing TCP 139	Enable Restart Rig in Administer Test Controllers dialog to work. Enable Agent setup to automatically add the Agent service user accounts to the TeamTestAgentService user group. Warning   Setup does not add this firewall exception. Add this exception manually.

 

If you are running on Windows XP in a work group, you must turn off Simple File Sharing. See this KB for info on how to do that: https://support.microsoft.com/kb/307874

Team Test Load Agent Installation

Warning    Set up the Controller before setting up any agents.

1.      Start setup.exe located in the vs\agent folder.

The Welcome to Setup page appears.

2.      Click Next.

The End-User License Agreement and Product Key page appears.

3.      Review the End-User License Agreement (EULA) and accept the terms of the license agreement, if appropriate.

4.      Enter the product key information and then click Next.

The Destination Folder page appears.

5.      Click Browse to select a different destination folder, or click Nextto use the default folder.

Note   If you are installing the Agent on the same computer as the Controller, the option to choose a new location is disabled.

The Specify Controller page appears.

6.      Enter the computer name of the Team Test Load Controller computer and click Next.

Note   You can locate the name of the controller computer by clickingMy Computer, right-clicking, choosing Properties and viewing the Computer Name tab.

The Ready to Install page appears.

7.      Review the installation information and then click Install to continue.

The Installing Components page appears.

8.      Upon successful installation, the Setup Completed Successfully page appears. Click Finish to complete the setup.

Following this, you will be prompted to restart your computer to complete the installation.

NoteThe user running the agent setup must be a member of local Administrators group and a member of the controller’s Administrators group.

During setup, you are prompted for the name of the controller computer that this agent will be associated with. Agent setup communicates with the controller to add the agent to the controller.

During setup, you are prompted for the Agent user. This user is automatically added to TeamTestAgentService group on the controller. Only Agent Service Users who are a member of this group are allowed to communicate with the controller.

The Agent service is named Visual Studio Team Test Agent. To view the properties of the service use Control Panel. In Control Panel, choose Administrative Tools and then choose Services. The agent services can be started and stopped like other services in Windows. Also the agents can be administered using the Administer Test Controller dialog box, available on the Test menu. For more information, see “How to: Administer the Rig” in the product documentation.

Note   AgentConfigUtil.exe is a command line utility that can be used to assign an Agent to a different Controller, or to add an Agent to a Controller if Agent setup fails to do so.

Load Agent Firewall Exceptions

If you are running Windows Firewall, Agent setup added the following exceptions to your firewall:

 

Exception	Reason
QTAgentService.exe	Enable communication with the controller
AgentServiceUI.exe	Enable communication with the controller
QTAgent.exe	Enable communication with the controller
AgentConfigUtil.exe	Enable communication with the controller
File and Printer Sharing Port 139	Enable Restart Rig in Administer Test Controllers dialog to work Enable collection of performance counters Warning   Setup does not add this firewall exception. Add this exception manually.

 

If you are running on Windows XP in a work group, you will also need to turn off Simple File Sharing.

Troubleshooting

Note   To verify the installation, run Visual Studio and launch the Administer Test Controller dialog box (available from the Test menu), connect to the controller, and check the status of the agents. Agents should be in the Ready state. If an agent is not in the list of agents, reinstall the agent or run AgentConfigUtil.exe after verifying that the controller is installed correctly. If the agent is in the Disconnected state, check that the agent service user account has been added to the TeamTestAgentService user group on the controller. Also check the <installdir>\Microsoft Visual Studio 2005 Team Test Load Agent\LoadTest\agentConfigUtilSetup*.log file for errors.

“Rig failed to restart for controller <controller computer name>. The following error was reported Cannot open VSTTController service on computer <controller computer name>.”

This error occurs when a user tries to restart the rig from the Administer Test Controller dialog box, but while this user has been added to the TeamTestControllerAdmins group, they have not also been added to the Power User or Administrator group.

Comments

• Anonymous
  February 21, 2007
  Here is the install guide for the agent and controller. It contains a bunch of useful sizing and configuration

• Anonymous
  March 08, 2007
  Hi Ed, I am trying to set up an environment where I will have a test controller communicating with agents through NAT. The agents will work in workgroup mode as you specify. However I need to know two things to verify if my approach will work:

1. Do TCP connections only go from Controller to Agent, or does sometimes an Agent initiate communication with the Controller? (If yes, then I would need to work around it in NAT config somehow)
2. What exact ports need to be opened between the test controller and agents? You only specify that there are some exceptions, but do not give info what exact exceptions are needed. I need this information to set up the firewall rules that isolate the test controller and test agents. Another approach that I am considering is placing the Test controller on the same network as agents, but then I have the issue of Test Controller <-> TFS communication. Do you know if it is possible to run such configuration if the test controller is in one domain and the TFS server in another untrusted domain? Which approach would you recommend? The general problem is that the development environment where the TFS server sits is another domain and another network separated by NAT and firewall from the test environment. I would imagine that is a pretty common scenario and there must be a solution to that. Thanks in advance for any information, Andrzej

• Anonymous
  June 11, 2007
  Ed, thank you SO much.   This is exactly what I was looking for.  Keep up the great work! Mark.

• Anonymous
  June 17, 2007
  Hi Ed Thanks for the Agent and controller help. I have downloaded visual studio team system  2005 180 days trail version. and now I want to install Agent and controller. As you mentioned that setup files are present in the folder vsAgent and VSController. But In the VSTS setup folder i have not found the vsagent folder and vscontroller folder so please explain me from where i will find this folder for the vsts Agent and Controller installation. Also explain me that VSTS 2005 180 days trial version supports agent and Controller functionality.? Thanks Nilesh

• Anonymous
  October 02, 2007
  In my previous blog article, I pointed out that in a workgroup environment, Windows XP has the force

• Anonymous
  October 02, 2007
  In my previous blog article, I pointed out that in a workgroup environment, Windows XP has the force

• Anonymous
  November 24, 2008
  Load Agent and Load Controller Configuration Guide Published On: Ed Glas&#39;s blog The Team Test Rig

• Anonymous
  November 24, 2008
  Load Agent and Load Controller Installation Guide Published On: Ed Glas&#39;s blog Team Test Load Controller

• Anonymous
  February 08, 2009
  [ Nacsa Sándor , 2009. február 8.] Ennek a processzoronként licencelt szoftvernek a segítségével skálázni

• Anonymous
  March 17, 2009
  Hi Ed. We have a setup where the controller is within the test domain A and the load agent has 2 NIC cards to access domain A while on domain B. The test rig thus has the controller in domain A while it controls the load agent which is in domain A&B. The interesting thing that happens here is that if you initiate a load test from the load agent side, the test rig shows all is connected but it remains in a pending state and does not start the actual load test. When you initiate the same test from the Controller the test rig starts the tests with no problem. We have narrowed the problem down to what we think the problem is and that is the fact that we have not setup Kerberos delegation in our environment and initiating the test from the load agent side means a hop from the agent to the controller and than back to the load agent again, where starting the test from the controller is only one hop from the ctlr to the agent. Please comment on this theory, as we don't find any error logs or exceptions to prove that. Also are there any other issues or tips you can recommend to solve this other than setting up Kerberos or moving the load agent onto domain A only ? Regards. David.

• Anonymous
  April 18, 2009
  http://blogs.msdn.com/edglas/pages/load-agent-and-load-controller-installation-and-configuration-gui...

• Anonymous
  May 22, 2009
  I cannot seem to find what is the recommended # of NIC's for an Agent?

• Anonymous
  June 18, 2009
  The comment has been removed

• Anonymous
  June 18, 2009
  The comment has been removed

• Anonymous
  November 12, 2009
  I have setup the controller and a remote agent.  There is a firewall in between with the required ports open but the agent is still showing as disconnected.  These are two different domains and a trust is not going to be allowed.  All user accounts have been setup as local users and all match with passwords.  Please help as this is delaying testing. Thanks

• Anonymous
  July 19, 2010
  According to given installation procedure this tool accepts only .Net framework with version 2.0. does it accepts the versions also. i want to know whether it accepts .Net 3.5 also

• Anonymous
  October 26, 2010
  Hi I have installed VS Ultimate -2010, and could someone guide how to install a controller and UserAgents using Ultimate.

• Anonymous
  October 26, 2010
  Hi Installed VS Ultimate 2010.  Could someone help how to install Controller and Vu Gens. on top of Ultimate? or I need to uninstall ultimate and install VSTS.  Please guide. -ASdf

• Anonymous
  March 27, 2011
  Is it possibe to have both a 2007 agent and a 2010 agent on the same computer?  Doing this will help keep our VMWare deployment as small as possible.

• Anonymous
  March 30, 2011
  While configuiring Agents for Load testing what option is recommended amongts the both:

1. Run the Test agen as "Service" OR " Interactive process" I am in middle of configuring the agents. it will be great if you can let me know the suitable option to select.

• Anonymous
  July 20, 2011
  Hey!! This is really good.. Could you come up with such a requirement for Visual Studi 2010 as well.

• Anonymous
  October 09, 2011
  Hey ED !!, i want to know that how to add the Agent computers while prepareing to perform the Load Testing in VSTS 2008.

• Anonymous
  May 11, 2015
  Please tell me how i install Agent and controller visual studio 2008 into my system, send me the link which i install everything..