Packaging and Installing Test Type Extensions in Visual Studio Team System
This topic describes the scenarios, components, packaging options, and installation details that are required for packaging test type extensions and installing them into Visual Studio Team System.
Test Type Extensions
Test type extensions provide Visual Studio Team Edition for Testers and Visual Studio Team Edition for Developers with new test functionality. A new test type allows integration into the user experience and also provides for new types of tests that can be authored, edited, and run within Visual Studio 2008.
Tests from test type extensions are visible and usable from the Test View and Test List Editor windows, just like the built-in test types. You can build custom editors as part of a test type extension and integrate them into Visual Studio 2008. You can customize test execution for each test type and then you can build it into a test type extension. As a result of the integration of the new, custom test type, test results that it produces appear in the Test Results window. Test results for all test types can be published for team sharing and for reporting through the Team Foundation Server data warehouse.
For more information about test type extensions, see Creating Custom Test Types in Visual Studio Team System.
Test Type Scenarios
There are four scenarios where a custom test type is used during the life of a test.
Tests are authored, organized, and executed in Visual Studio 2008. The Visual Studio 2008 test experience is the richest of the four scenarios and includes all aspects of a test type extension.
Tests can be executed in a test lab. Test runs are scheduled on a remote controller and then distributed to agent computers for execution. Tests that are run in a test lab must be portable from the computer on which they were written and need full execution support.
Tests can be run as part of a team build on the Team Foundation Build computer. Similar to running in a test lab, tests are run on the Team Foundation Build computer and must be portable from the computer on which they were written and need full execution support.
Test results can be published for sharing and reporting. Test results are uploaded from Visual Studio to the application tier for team sharing and storage in the data tier, and for inclusion in the data warehouse.
Test Type Components
To make it easier to understand how to factor test type components and which components are required to satisfy the scenarios described in Test Type Scenarios, the components that make up a test type are divided into three components:
Core: the components that implement and describe the core information about a test type. At a minimum, this means that you must implement ITestElement, the test-element object, and ITestResult, the test-result object.
Execution: the components that implement the execution functionality for the test type. At a minimum, this means that you must implement ITestAdapter.
Client: the components that implement the client-side functionality, typically, the authoring and editing of tests. At a minimum, this means that you must implement ITip, which provides for loading, saving, and creating test elements, and ITuip, which provides the user interface.
You might have created a client component with the purpose of packaging and distributing it to customers. However, customers can use it only if you sign it with a package load key (PLK) before you distribute it. For more information about signing packages, see Adding a Package Load Key.
Options for Packaging Test Type Components
To make sure that the correct functionality is installed for the scenarios described in Test Type Scenarios, and to keep unnecessary functionality and surface area exposure at a minimum, you should factor test types into three assemblies: core components as one assembly, client components as a second assembly, and execution components as a third assembly.
However, other packaging options will work. The best two-assembly option is to package client components as one assembly and core and execution components as a second assembly.
A single assembly option is also possible, where all components are built into one assembly.
Installing Test Type Components
After you have packaged the components of your custom test type into assemblies, install them onto the computers that require them. The following sections describe your installation options:
Install Components on Test, Development, and Build Computers
Install Components on a Controller Computer
Install Components on an Agent Computer
Install Components on the Application Tier
Install Components on Test, Development, and Build Computers
On a computer that is running Visual Studio Team Edition for Testers or Visual Studio Team Edition for Developers or a Team Foundation Build computer, you must install all components, core, client, and execution. On the Team Foundation Build computer, all components are required because a full installation of Visual Studio Team Edition for Testers is required to run tests as part of a team build. For more information about the core, client, and execution components, see Test Type Components.
Test type extensions will almost always have a dependency on the core test tools components that are part of Visual Studio Team Edition for Testers or Visual Studio Team Edition for Developers. To help you determine whether the core test tools components are already installed, a license key has been created. Check the following registry key for the value "twepoiuadidasjgfksjdfnmncxzbxiuwpaituaqskg": HKLM\Software\Microsoft\VisualStudio\8.0\Licenses\02990959-292B-4b46-BCF1-0E31989AEEB4
Install test type extension assemblies in the Visual Studio 2008 "PrivateAssemblies" subdirectory. You can retrieve the installation path for Visual Studio 2008 from the registry key HKLM\Software\Microsoft\VisualStudio\8.0\InstallDir. Test type extension assemblies should be installed in the "PrivateAssemblies" directory, which is a subdirectory from this path.
Install Components on a Controller Computer
On the test lab controller computer, only the core components are required. To communicate between the Visual Studio 2008 computer and the controller computer and between the controller computer and the agent computers, .NET remoting is used. The act of serializing and deserializing data through .NET remoting requires that the core components be installed.
To determine whether the controller is installed on the computer, a Setup program can check for the existence of the following registry key:
HKLM\System\CurrentControlSet\Services\VSTTController
Test type extension assemblies must be installed in the "Load Test" subdirectory where the controller is installed. You can retrieve the installation path for the controller from the following registry key:
HKLM\System\CurrentControlSet\Services\VSTTController\ImagePath
Test type extension assemblies should be installed in the "Load Test" subdirectory of the controller installation path.
Install Components on an Agent Computer
On the test lab agent computer, both the core components and the execution components are required.
To determine whether the agent is installed on the computer, a Setup program can check for the existence of the following registry key:
HKLM\System\CurrentControlSet\Services\VSTTAgent
Test type extension assemblies must be installed in the "Load Test" subdirectory where the agent is installed. You can retrieve the installation path for the agent from the following registry key:
HKLM\System\CurrentControlSet\Services\VSTTAgent\ImagePath
Test type extension assemblies should be installed in the "Load Test" subdirectory of the agent installation path.
Install Components on the Application Tier
Both the core components and the client components are required on the application-tier computer. To determine whether the application tier is installed on the computer, MsiQueryProductState can be used as follows:
MsiQueryProductState("{FF8C520A-74E4-4DFA-9624-560FD61B0826}") == INSTALLSTATE_DEFAULT
Test type extension assemblies must be installed to two locations on the application-tier computer:
To support the publishing Web service, install test type extension assemblies to this folder: Program Files\Microsoft Visual Studio 2005 Team Foundation Server\Web Services\Build\bin. This service publishes test results from Visual Studio to the application tier.
To support the data warehouse adapter, install test type assemblies to this folder: Program Files\Microsoft Visual Studio 2005 Team Foundation Server\Web Services\Warehouse\bin\Plugins. This adapter pushes test result data into the data warehouse for reporting.