Mahipal Kante's Blog on Visual Studio 2010

Visual Studio 2010 Load Test – Out of Memory Exceptions On 32 bit Machines OR 32 bit Run

Mahipal Kante - MSFT — Thu, 10 Jun 2010 04:22:00 GMT

If you see large number of “Out of Memory” Exceptions in your Load Test results, do not quickly conclude that there is a memory leak! First make sure you are running Agents on 64 Bit OS in a 64 Bit Test Run. Note the two terms with 64 bit in them, which I have explained below.

If you want to scale up the VUsers to 1000 and beyond, one fundamental requirement is that your Agent machines should be of 64 Bit OS. For each Agent machine there is only one process that is running the load test, which needs to spawn thousands of threads. Due to this the memory consumed by Agent process increases linearly as the number of VUsers increase. For a large VUser load, the memory consumed can easily go much beyond 2 GB, which is the limit for any process on a 32 bit OS. This is why for a large VUser volume, Load Test will start throwing “Out of Memory” exceptions. These exceptions are not thrown by OS itself, but the Load Test engine starts throwing these exceptions as soon as the process reaches 1.75 GB.

So, even if your machine has huge memory and CPU, load test will not be able utilize it well. Hence, it is imperative that you change the machine OS to 64 bit.

An important thing to remember once upgrading your OS to 64 bit is to make sure to change the Test Run Settings also to 64 Bit. Otherwise, your Load Test will continue to run in 32 bit mode on 64 Bit OS, and you will not be able to go past the out of memory exceptions. So, if you are browsing the Load Test results, and you see Out of Memory exceptions, be sure to look for “QTAgent32” in the results as shown in the above picture. If you see this, you should go and change the run settings as shown below to 64 Bit. On a 64 bit run, the Agent process is listed as “QTAgent” with no “32” suffix!

Finally here is a graph of a successful 64 bit run for a 1000 Vusers, which clearly shows that the Virtual Bytes of the QTAgent process has gone up close to 5 GB. Notice the quick linear jump when VUsers were increased.

VS 2010 Load Test Results Database Recovery Model

Mahipal Kante - MSFT — Thu, 10 Jun 2010 01:09:00 GMT

To get the best performance while storing Load Test Results into Database, we need to make sure that the recovery model on the database is set to “SIMPLE”. Sometimes when the DB create script, loadtestresultrepository.sql, that is in ~Program Files\Microsoft Visual Studio 10.0\Common7\IDE folder is used to create the results database, the recovery model is erroneously set to “FULL” which is not a desirable setting for Load Test Results DB.

To check if the recovery model is set to “SIMPLE” and to fix, follow these steps:

Step 1: Run “Visual Studio Command Prompt 2010” in Administrator mode

Step 2: Open “Notepad checkDB.sql”

Step 3: Enter the following query into the file and save:

SELECT name,

CAST(DATABASEPROPERTYEX(name, 'Recovery') as nvarchar(30))

FROM master.dbo.sysdatabases;

Step 4: Run the following command:

sqlcmd -S .\sqlexpress -i checkDB.sql

Step 5: Make sure the Load Tet results DB is listed as "SIMPLE" for recovery model

master

SIMPLE

tempdb

SIMPLE

model

SIMPLE

msdb

SIMPLE

LoadTest2010

SIMPLE

Step 6: If it is NOT set to SIMPLE, enter the following SQL into "fixDB.sql"

ALTER DATABASE LoadTest2010 SET RECOVERY SIMPLE

Step 7: Run command:

sqlcmd -S .\sqlexpress -i fixDB.sql

Step 8: Repeat step 5 to make sure it is now changed to "SIMPLE".

The reason why we want to set recovery model to SIMPLE is that the Load Test results are always bulk loaded into database. So we do not need to maintain database logs, which makes the DB operations slow. You can read more about the recovery models on MSDN

VS2010 Web Test Plugins for Web Service Messages

Mahipal Kante - MSFT — Tue, 06 Apr 2010 23:53:00 GMT

When testing Silver Light applications, we frequently run into dealing with non-Html content types. This is because most of the Silver Light applications use WCF RIA web services. Typically, this will be a binary format called MSBin1. Also, Silver Light applications may use other content types such as JSON, SOAP and REST to communicate with server. In general there is significantly greater use of Web Services and JSON in today's web applications.

To test these applications under various scenarios we really need to edit, parameterize, extract values from responses and data drive these messages rather than simply playing back the same messages that are captured by Web Test recorder.

There is currently no editing capability for MSBin1 content in Web Test and JSON messages are difficult to read or edit. Also, in general Xml needs better editing capabilities than simple text editor that we have currently in Web Test.

Luckily there is a good set of extensibility points in Web Test for providing additional support for these content types. There are a number of plugins available to be downloaded from Code Plex project Web and Load Test Plugins for Visual Studio Team Test

The Web Test Plugins for Web Service Messages leverage these extensibility points to provide the following functionality for all these content types:

Editor for Request body: The plugins provide a tree view of the structured data which allows easy editing, parameterization and data binding.
XPath based Extraction and Validation Rules: XPath is a powerful tool to extract required data from Xml. This provides an additional capability for Xml and JSON based messages which can use existing extraction and validation rules that work on text. But for Msbin1, none of the existing rules work, because it is binary. So, these plugins are only way to extract and validate data from MSbin1 content types.
Data Driving for MSBin1: Using these plugins, you can parameterize and data drive MSBin1 content.
Additional tab in Web Test results that shows Request and Response in hierarchical view side by side. Using this user can add extraction/validation rules on response messages and parameterize requests.

When installed, the following components will be added to the user's file system:

Plugin	Assembly/Files	Folder	Functionality
Visual Studio Addin - Additional tab in Web Test Result	WebServicePluginFor WebTestResultTab.AddIn WebTest.WebService. Plugin.ResultTab.dll	\Visual Studio 2010\Addins	Adds a new tab called "Web Service" into Web Test results
Msbin1MessageEditor (IBinaryHttpBodyEditorPlugin) XmlMessageEditor (IStringHttpBodyEditorPlugin) WebServicePostRecording (WebTestRecorderPlugin)	WebTest.WebService. Plugin.MessageEditors.dll	~Program Files\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies \WebTestPlugins	WebTest Editor Plugins - Request editors for various content types
XPathExtractionRule (ExtractionRule) XPathValidationRule (ValidationRule) WebServicePlugin (WebTestPlugin)	WebTest.WebService. Plugin.Runtime.dll	~Program Files\Microsoft Visual Studio 10.0\Common7\IDE\PublicAssemblies	Runtime assembly -contains extraction/validation rules, data binding for binary content types.

All three assemblies work together, so all of them need to be present when authoring Web Test. Only runtime assembly (WebTest.WebService. Plugin.Runtime.dll) is required to run Web Test.