• Prerequisite: Create a PERFMON log file (.blg) by running a trace in a server, to do this you need to create a Perfmon Data Collection Set, there are a few templates by default or you can create or lookup a specific template. Once created you need to run the Data Collection for an given amount of time (e.g. 30min while a Load Test is running). Here is some additional information from TechNet.
• Optionally: Once you have one or several .blg files, you can join them together into a single .blg file for easier handling, or you can chose to keep them separate. For this you can use relog.exe, it's important to point out that you can use relog.exe from the Command Prompt console, or you can use a PowerShell script as follows for easier editing of the attributes and local paths:

• Step 1: You will need to create a new database in SQL (e.g. PerfmonDB). For this example I'm using SQL Server 2016, I have also successfully loaded Perfmon data to a SQL 2012 database.
  • Note: SQL Express editions do not work since it you would need to create the ODBC connection thru the SQL Native client (to localDB) and that will create problems with relog.exe since it won't recognize the DSN source.
• Step 2: Create a SQL Server ODBC- System DSN source named Perfmon_DSN that points to your SQL Server and Database
  • To create the ODBC-System DNS connection:
    • Run odbcad32.exe, go to the System DSN tab and click on 'Add' to add a new Source
    • Select SQL Server (not native client) and type the connection configuration details (e.g. Name: Perfmon_DSN, Description: Perfmon, Server: <yourSQLServer>)
    • Preferably chose NT Authentication and change the default database to PerfmonDB, keep all other defaults.

• Step 3: Use relog.exe to load the combined, or single .blg file into the SQL database thru your DNS connection.
  • To do this you can run relog.exe from PowerShell as shown on the code below.

• You can validate that the script worked by checking your PerfmonDB database. Tables CounterData, CounterDetails and DisplayToId should have been created.
• Step 4. Now that the data has been loaded into the SQL Database, we can start working in pulling it into Excel reporting purposes. Let's do the Connection/Query first:
  • You will need Excel (I'm using 2016 for this example).
  • First you need to create a connection. In a new Spreadsheet, go to Data > Get External Data > From Other Sources > From SQL Server.
    • Type your SQLServer name and log on credentials.
    • Select your database PerfmonDB, there is no need to specify a default table just click next and then finish.
    • Cancel the select table dialog.
  • Now that you have a connection, you need a proper SQL query that pulls only the information that you need.
    • In excel, go to Data > Connections, select your connection and click Properties.
    • Go to the Definition tab and in Command Type, select SQL
    • Under Command Text copy the SQL query below and click the OK button. On the query, the interval is set at every 1 minute, you can tweak that according to your needs (e.g. every 1hr), also this query will pull all objects imported from the Perfmon logs, if it is too much information you can always filter out in the WHERE clause by adding (for example): AND objectname = 'Processor'.

• Step 5 - Let's use the query results to build a Pivot Table and Charts
  • First go to Insert > Charts > PivotChart > PivotChart & PivotTable
  • In the Create Pivot Table dialog, select "Use External Data Source" and "New Worksheet". Click ok.
  • That should show you an empty Table, empty Chart and the PivotTable Fields Selector. In the Selector, use the configuration shown below. Basically you need the internal field in the Rows area, the counterValues in the Values area and all other fileds in the Columns area.
    • Note: For the counterValues field you need to summarize the value field by average. To do this right-click in the counterValues field and go to Properties, then from the list select Average.
  • Now you should have all the information you need for the Charts, however it may have too many objects, counters, or Machines for the chart to make sense. In the table use the Column Labels filters to chose only the objects that you need. Please note that you can change the field on top of the picture below in the Select field dropdown:
  • A much more rewarding way to do this is by adding Slicers. To do this, in excel go to PivotTable Tools > Analyze > Filter > Insert Slicer, then select the counter(s) that you need for easier filtering. It should look something like this:
  • Now you can play around with selecting one or more combinations of Machine/Object/Instance and Counter. You can do multi-select with the button on top of each selector.
    • Keep in mind that not all Counters have an Instance.
  • Using the selectors and some formatting on the Chart, you should be able to easily come up with a Chart similar to the one below:

$categoryName = "UI Scripts"

$categoryHelp = "Transaction Response Times for UI Scripts"

$categoryType = [System.Diagnostics.PerformanceCounterCategoryType]::MultiInstance

 

$categoryExists = [System.Diagnostics.PerformanceCounterCategory]::Exists($categoryName)

 

 

If (-Not $categoryExists)

{

  $objCCDC = New-Object System.Diagnostics.CounterCreationDataCollection

 

  $objCCD1 = New-Object System.Diagnostics.CounterCreationData

  $objCCD1.CounterName = "Transaction Response Time"

  $objCCD1.CounterType = "NumberOfItems32"

  $objCCD1.CounterHelp = "Duration in Miliseconds of a UI transaction"

  $objCCDC.Add($objCCD1) | Out-Null

 

  [System.Diagnostics.PerformanceCounterCategory]::Create($categoryName, $categoryHelp, $categoryType, $objCCDC) | Out-Null

}

 

$perfInst1a = New-Object System.Diagnostics.PerformanceCounter($categoryName, "Transaction Response Time", "Transaction1", $false)

$perfInst2a = New-Object System.Diagnostics.PerformanceCounter($categoryName, "Transaction Response Time", "Transaction2", $false)

$perfInst3a = New-Object System.Diagnostics.PerformanceCounter($categoryName, "Transaction Response Time", "Transaction3", $false)

$perfInst4a = New-Object System.Diagnostics.PerformanceCounter($categoryName, "Transaction Response Time", "Transaction4", $false)

$perfInst5a = New-Object System.Diagnostics.PerformanceCounter($categoryName, "Transaction Response Time", "Transaction5", $false)

$perfInst6a = New-Object System.Diagnostics.PerformanceCounter($categoryName, "Transaction Response Time", "Transaction6", $false)

$perfInst7a = New-Object System.Diagnostics.PerformanceCounter($categoryName, "Transaction Response Time", "Transaction7", $false)

 

$checkIfSame1 = ""

for ($i = 1; $i -lt 100000; $i++){

    $timings = import-csv "C:\Users\edwinh\Desktop\test.csv"

    $checkIfSame2 = $timings[$timings.Length-1].'Transaction1,' -as[int]

    If($checkIfSame1 -ne $checkIfSame2){

        $perfInst1a.RawValue = $timings[$timings.Length-1].'Transaction1' -as[int]

        $perfInst2a.RawValue = $timings[$timings.Length-1].'Transaction2' -as[int]

        $perfInst3a.RawValue = $timings[$timings.Length-1].'Transaction3' -as[int]

        $perfInst4a.RawValue = $timings[$timings.Length-1].'Transaction4' -as[int]

        $perfInst5a.RawValue = $timings[$timings.Length-1].'Transaction5' -as[int]

        $perfInst6a.RawValue = $timings[$timings.Length-1].'Transaction6' -as[int]

        $perfInst7a.RawValue = $timings[$timings.Length-1].'Transaction7' -as[int]

    }

    $checkIfSame1 = $checkIfSame2

Start-Sleep -s 2

}

 

 

Read-Host

 

Visual Studio

The first thing you will need is Visual Studio. The Web Load & Performance Testing features of Visual studio are only available on versions: Visual Studio Enterprise 2015 or Visual Studio Ultimate 2013.

Other type of tests such as Coded UI, Manual tests and Unit testing are also available on Visual Studio Test Professional 2015 however these are all functional tests. We'll focus on performance web tests for now, this is NOT available on VS Test Professional. You can find a feature comparison among VS versions here: Compare Visual Studio 2015 Offerings.

Virtual User Licenses

On premises. No virtual user licenses are needed. You can execute load tests with any number of virtual users (as far as your hardware resources allow you) as long as you have any of the following:

- Visual Studio Enterprise 2015 with MSDN

- Visual Studio Enterprise 2015 annual and monthly subscribers (check Visual Studio 2015 Licensing White Paper)

- Visual Studio Ultimate 2013 with MSDN (check Visual Studio 2013 Licensing White Paper)

* In Visual Studio Enterprise (or Ultimate) trial version, the virtual user count is limited to 250.

Cloud-Based testing. No extra software installation needed. You do need:

- A Visual Studio Online (VSO) account (get one here).

- That gets you 20,000 virtual user minutes every month to load test at no extra charge (check how virtual users minutes work and test duration limitations here: VSO Virtual users minutes).

Controller and Agents Software

On premises. If you are planning to load test an application in an on-premises environment, you will need a Controller and at least 1 agent machine (check next section for hardware requirements). You can find the installable software for these two components based on your Visual studio version here:

- Download Agents for Microsoft Visual Studio 2015  - Update: Don't use the Test Agents for VS 2015 for load testing. It is meant for continuous tests on build scenarios, it doesn't have a test controller or a configuration tool. Instead use VS Test Agents 2013 Update 5:

- Download Agents for Microsoft Visual Studio 2013 Update 5

 Cloud-Based Load testing. No additional software installation needed.

2. Infrastructure/Hardware requirements

Visual Studio (IDE) System Requirements

• Windows 7 SP1 or above, or
• Windows Server 2008 R2 SP1 or above.
• A minimum of 1.6GHz, 1.0 GB of RAM (1.5 if working on a VM).
• 10 GB of available HD space (20 GB for VS 2013).
• Check details here:
• Visual Studio Enterprise 2015 with Update 1
• Visual Studio Ultimate 2013 with Update 5

Controller and Agents Hardware Requirements

On premises. If you are planning a load test project in a on premises environment, you will need a controller and at least 1 test agent, to set this up you will need the following:

• OS/Framework requirements
  • Controller: Windows 7 SP1 / 8 / 8.1 or Windows Server 2008 R2 SP1 / 2012 / 2012 R2
  • Test Agents: Windows XP SP3 / 7 SP1 / 8 / 8.1 or Windows Server 2008 R2 SP1 / 2012 / 2012 R2.
  • Both: .NET Framework 4.5
  • More information here: System requirements for Controller and Agents

You can also execute local test runs (without controller or test agents) directly from your Visual Studio host, this is done normally for debugging purposes since you are limited by resources.

• Hardware sizing

The table below shows the recommended hardware requirements. You can use this table to plan how many test agents and what size of controller suit your load testing needs:

• Load Test Repository

Load tests results may be stored in the Load Test Results Repository, which is a SQL database. The Results Repository database is created by setup for controllers (or created automatically on the first local run of a load test), a the database will be created automatically if the load test schema is not present.

SQL Server 2012 Express LocalDB, which is installed with Visual Studio is the default database server for load tests. If SQL Server Express is detected with an existing load test database, Visual Studio Enterprise will try to connect to it and use it.

However, for heavier database needs you should consider upgrading to a full SQL Server to provide further scaling potential. Also, SQL Express is limited to using a maximum of 4 GB of disk space. If you will run many load tests over a long period of time, you should consider configuring the load test results store to use an instance of the full SQL Server product if available. More on Load Test results repository.

Note: A single 1 hour load test can take from 100MB to 500MB depending on the number of webtests, performance counters, VMs you are monitoring, etc. Plan storage needs accordingly and take in consideration that this information is temporarily stored in the Test Agents, collected in the Controllers RAM memory before finally being placed in the SQL Repository. Plan RAM Memory needs accordingly.

• Alternative: Testing using VSO or TFS instead of Controller

For test scenarios (still on premises) using Visual Studio Online (VSO) or Team Foundation Server (TFS) 2015, you won't need a test controller because Agents for Microsoft Visual Studio 2015 handle orchestration by communicating with VSO or TFS 2015. For example, you're running automated tests with your build and release workflows in VSO or TFS 2015.

Cloud-Based Load testing

You can use cloud-based load testing to avoid having to use your own resources and machines. In this case you don't need a Controller or Test Agents. The cloud-service provides virtual machines in the cloud that generate the load needed to test your target application/website.

All you need is a Visual Studio Online (VSO) account. Check 'Getting started with Load Tests' further down in this document.

3. Getting started with Webtests

Basic components

At the most basic level you will need to do the following steps to get started:

• Create a web performance and load test project
• Record a web performance test.
• Create a load test.
• Run and analyze your load test

 Follow this article to learn how to create the most basic components.

Validation

Parametrization

o    Constant parameters. You can create a Context Parameter for any value in a request of a WebTest. The constant parameters are displayed at the bottom of your webtest. A simple case where you would want to do this is for host or website URLs. Read this article for more information.

o    Dynamic parameters. Web sites and applications use dynamic values, this is a value that is generated every time that a user runs the application. A dynamic parameter can cause your Web performance test playback to fail because it has a different value every the test is run. Therefore, you cannot play back recorded values. (E.g. a session ID). To solve this situation you have a couple of options (follow link to articles):

- Auto-detect dynamic values and promote them to WebTest parameters. Use VS auto-detect feature to identify and parameterize dynamic values (it may not catch all values).

- Add an Extraction Rule. Find dynamic values by using 'quick find' on recorded results then add an extraction rule:

• Extraction rules extract data from the responses to Web requests and store the results in the test context as name/value pairs. Extraction rules are predefined in Visual Studio (you can also create your own, check Customization section), you can use them to extract form fields, text, attributes, headers, regular expressions, and hidden fields, Check full article here.

Data Binding

 You can bind parameters (e.g. form post parameters) on the requests of your WebTest to data sources to provide different values to the same test. This makes your load test more solid and realistic. You have several options for the data source type:

- SQL database (several versions permitted, check FAQ)

- CSV text files

- XML files (or SOAP XMLs)

 Check this article for a step by step guide.

4. Getting started with Load Tests

Load Test setup

Load Test can be created in a Load Test projects. If you followed the article at the start of Section 3 of this page you must already have a simple Load test. Some of the Load test settings and scenarios can be setup on the initial Wizard or directly on the load test afterwards. Below are descriptions for the main sections:

Scenarios

Load tests contain scenarios, which contain Web performance tests. Scenarios are important because they give you flexibility in configuring test characteristics that allow for simulation of complex, realistic workloads. You can have more than one scenario per load test.

 Use the Load pattern, test mix model and test mix to specify what webtests to use, the manner in which the load will be applied to your application and pacing.

 You can also use Browser mix, Network mix to make your workload more realistic. Check MSDN article on editing Load tests scenarios.

Counter Sets, Counters and Mappings

Counter sets that are useful when you analyze performance counter data. The counter sets are organized by technology such as Application, .NET Application, IIS, or SQL. You can create counter sets during the initial New Load Test Wizard or later on the load test file itself.

There is a set of predefined counter sets or you can create your own. You can also add counters to the default Counter Sets or to your own. The counters are added from Perfmon.

 On the Run settings section of a load test you can specify Counter Set mappings where you can add computers (by DNS or IP) and map them to Counter sets. Visual studio will collect data for every counter on a set for each computer where it is mapped to.

 To learn more about Counter Sets, thresholds, sampling and other tasks, check this article on MSDN.

Run Settings

Run settings are properties that organize the manner in which a load test runs. You can have more than one run setting in a load test but only one can be active at a time while the other settings provide a quick way to select an alternative settings.

Settings are used to specify things like test duration, Logging, SQL server collection, warm up duration, sampling rate or validation setting between others. You can also add Counter Set Mappings to map computers to Counter Sets.

Check this article for full description of tasks available.

Running Load Tests On Premises

Test Settings file

Make sure your Load test is configure to run On premises by checking the General section of your active .testsetting file on your Solution. Test Settings are not the same as Run Settings.

 Other items to configure on this window are Roles and Timeouts. On the Roles section, test execution method should be set to 'Remote execution' if you want to use your Controller and Test Agents. Check this couple of article to learn more about Test settings: Managing Test Controllers and Test Agents with Visual Studio, Edit Test Settings.

Controller-Agents connectivity

Also make sure the Controller can communicate with the Test Agents by checking the Load Test > Manage Test Controllers. That dialog should show if the controller has any Test Agents connected to it. More information here: Manage Test Controllers.

Running the Load Test

From that point forward you should be ready to run a Load Test. During or after run time you have access to Graphs and Tables with throughput, response times information, performance counters for the computers you configured and details on vusers activity. Check this tutorial to learn more: Run and analyze your load test.

Running Load Tests using the Cloud Service

There are a couple of options for running a Load test with the Cloud Service:

• You can run a Load test from Visual Studio (Enterprise 2015 or Ultimate 2013) and Visual Studio Team Services.
  • As mentioned above you will need a VS Team Services account.
  • Your solution needs to be connect to TFS.
  • In your solution; if you open your active .testsettings file, on the General section select 'Run tests using Visual Studio Online':
  • 
  • From there you can specify the Location you want your load to come from on the Load Test file and run the Load test. Check the full tutorial here, specifically the 'Run and analyze your load test' section.

• Your second alternative is to run a basic (and much more limited) load test directly in Visual Studio Team Services. The up side is that you don't need a load test project (or Visual Studio). You do need a Visual Studio Enterprise monthly or annual subscription or a MSDN subscription.
  • To do this you need to go to the Load Test Hub on your Visual Studio Team Services account page.
  • Setup is simple, just specify URL, Load location and some simple Test settings. Check the com article here.

5. Customization

• Create a custom plug-in for your Web Performance test: You can use Web performance test API to create a custom plug-in to add more testing functionality to your Web performance test, including at the request level. You can also create a Web Service test.
  • How to: Use the Web Performance Test API
  • How to: Create a Web Performance Test Plug-In
  • How to: Create a Request-Level Plug-In
  • How to: Create a Web Service Test
  • How to: Create a Recorder Plug-In

• Create a custom plug-in for your load test: You can use load test API to create a custom plug-in to add more testing functionality to your load test.
  • How to: Use the Load Test API
  • How to: Create a Load Test Plug-In

FAQ

How many vusers can I run per agent?

What version of SQL do I need?

Can load tests use other test types in their test mix besides web performance tests?

Can virtual users simulate pausing between test steps?

Can I mix versions of TFS, Microsoft Test Manager, the test controller, and test agent?

 https://msdn.microsoft.com/en-us/library/dd648127.aspx#Anchor_2

What firewall permissions do I need?

What databases can I use as a data source?

• Microsoft SQL Azure.
• Any version of Microsoft SQL Server 2005 or later.
• Microsoft SQL Server database file (including SQL Express).
• Microsoft ODBC.
• Microsoft Access file using the .NET Framework provider for OLE DB.
• Oracle 7.3, 8i, 9i, or 10g.

What is a good place to learn more?

https://blogs.msdn.microsoft.com/visualstudioalm/2015/12/10/visual-studio-2015-test-tools-getting-started-content/