In this post we are going to cover the differences in the Load Testing of Visual Studio 2010 (VS10) and Visual Studio 2012 (VS12).

 In VS10 you have to create a  New Test project and then Add Load/Performance test, this feature is changed in VS12, you can directly add Web Performance and Load Test Project . Once you add Web Performance and Load Test Project you will see Web Performance Test, this is the default behavior. Once, Test Project is created then you can add more web performance / unit / load tests to it by right clicking on test project and selecting appropriate test type. Below screen shots show the difference between creating a Test Project in VS10 and VS12.

Note here, ‘Add New Test’ window is deprecated in VS12 and you will not see the below screen that is part of VS10 ( Navigation: Test – NewTest). The only way to create a new test in VS12 is by right clicking your project and then adding a new test.

 There have been some changes in the way we associate TestController in VS12. Once we Create a Load Test in VS12, you can navigate to the LoadTest tab of IDE and click on ‘Manage Test Controllers’.  In VS10 you have to navigate to Test tab in IDE and select ‘Manage Test Controllers’. Similarly, selecting Active Test Settings file can be done from the Load Test tab of VS12.

Some additional options that are available in VS12 while creating a Load Test is the availability of IE 9.0 and IE 10.0 support under the Browser Mix tab in Load Test Wizard.

In this post we shall cover about the changes to the Code Coverage in Visual studio 2012 (VS12) from  Visual Studio 2010(VS10).

In VS 10, we needed to set the code coverage setting in a .testsettings file, it is in this file where you had specified the targets dlls for code coverage. In VS 12 the experience is far more transparent, your code is instrumented as the code gets loaded by the run time. All the code / binaries that is targeted by your tests will be instrumented and code coverage results can be viewed.

In VS12 to open the Code Coverage results navigate to Test->Analyze Code Coverage -> and select the test for which you would need to test the code coverage. This is much better than having to set up flags in a .testsettings file as done in VS10.

This will start off a new run for the scripts and after the scripts are done executing; we see the results as follows. The results are shown in terms of covered blocks and can be drilled down to specific methods and clicking on the method takes you to the source code, which itself is color coded blue to show the lines are covered by the tests and red to show the code that is not covered.

A Note about .testSettings file, it is still editable in VS2012, which is where you can set the targets for code coverage, you still have the option to do it.  However, it is a deprecated feature, and these targets do not work. This feature is just to help edit the file and the UI clearly shows that this setting is only for backward compatibility for VS10.

If you need to specify the Targets of Code coverage in VS12, you need to do this via a .runsettings file. This is a new file that has been introduced in VS12. This is done, because the .testSettings file only supports MSTEST.exe. Since VS12 supports other custom test runners (eg NUnit, xNunit) these settings will not be applied for those test runners.  You can find more details on how to work with a .runsettings file from this msdn link here.

Finally a word about the .coveragexml files. In VS12, the coverage data is all present in one single file and has all the data that is required. It has the information required to show what blocks are covered. Hence this file is all required to share to people who have the source code to show the covered blocks (colored). It’s not necessary to share the instrumented dlls to show the coverage offline.

This post gives a high level idea about profiling on Windows 8 and Windows Server 2012 platforms. Content of this post comes from one of the A-Team members Harsh Jain.

Windows 8 and Windows Server 2012 Apps (Apps) can be written in
a. JavaScript and HTML5
b. Visual C++, Visual C#, or Visual Basic

When profiling a Windows Store app that is written in JavaScript and HTML 5, Instrumentation method is used to collect data for the JavaScript code. For Windows Store app that is written in Visual C++, Visual C#, or Visual Basic, CPU sampling method is used to collect data for the native and managed code.

Windows Store apps run on a wide variety of devices, form factors, and processor types which need not have Visual Studio installed. Profiling on Windows Store Apps can be done on local machine as well as on a remote device. Profiling on Remote Device has been introduced in VS12 for Windows Store Apps.
Remote profiling lets you collect performance data on any device that is connected to your Visual Studio computer over a network or that is connected directly through an Ethernet cable. When you profile remotely, Visual Studio deploys the latest build of your app. You can start your app immediately, or you can profile your app after it is started by a contract or some other process.

Steps for Profiling Win8
App written in JavaScript and HTML5 on Local Machine:-

1. Select ANALYZE option from Menu Bar. Click on Launch Performance Wizard.

2. Specify the Profiling method you want to use. Instrumentation option is selected by default (and recommended) when we launch Performance Wizard. Click Next.

3. Choose the module to profile. Project app will be selected by default. Click Next. In the next screen, click Finish. If you don’t want to start profiling immediately, Uncheck “Launch profiling after the Wizard finishes” and click Finish.

4. Application will be launched with profiler. Execute the functionality of app.  Use profiler session screen to pause/stop profiling as shown below

5.Report will be displayed once you stop profiling.

For Remote Devices

Following configuration changes are required to profile a Windows 8 App on Remote Device

1. In Visual Studio, open the project that you want to profile.
2. On the Debug menu, choose Project Properties.
3. Choose Debugging in the Configuration Properties node.
4. From the Debugger to launch list, choose Remote Machine.
5. In the Machine Name row, type the name or IP address of the remote device.

If the remote device is on the same subnet as the Visual Studio computer, or if the device is directly connected to the Visual Studio computer, you can choose the down arrow in the row, choose the <Locate...> item, and then select the device on the Remote Debugger connections dialog box.

6. From the Debugger Type list, choose whether to profile JavaScript or Visual C++, Visual C#, or Visual Basic code in app components:
1. Choose Script only to profile JavaScript.
2. Choose any other value to profile C#/VB/C++ components of the JavaScript app.

Rest of the steps are same as profiling on Local Machine.

Steps for Profiling Win8 App written in Visual C++, Visual C#, or Visual Basic on Local Machine :

1.Select ANALYZE option from Menu Bar. Click on Launch Performance Wizard

2.Specify the Profiling method you want to use. CPU Sampling option is selected by default (and recommended) when we launch Performance Wizard. Click Next.

3. Choose the module to profile. Project app will be selected by default. Click Next. In the next screen, click Finish. If you don’t want to start profiling immediately, Uncheck “Launch profiling after the Wizard finishes” and click Finish.

 
4. Application will be launched with profiler. Execute the functionality of app.  Use profiler session screen to pause/stop profiling as shown below

5. Report will be displayed once you stop profiling.

For Remote Machine
Following configuration changes are required to profile a Windows 8 App on Remote Device

1. On the Debug menu, choose Project Properties.
2. Choose the Debug menu item.
3. From the Target Device list, choose Remote Machine.
4. In the Remote Machine box, type the name or IP address of the remote device.

If the remote device is on the same subnet as the Visual Studio computer, or if the device is directly connected to the Visual Studio computer, you can choose Find, and then select the device on the Remote Debugger connections dialog box.

Require Authentication should always be set to Yes. Not requiring network authentication puts your network at significant risk of attacks by malicious users.

Rest of the steps are same as profiling on Local Machine.

As compared to normal profiling options and features in VS10, following are not supported when profiling Windows Store apps in VS12:

• Profiling JavaScript apps using the sampling method.
• Profiling managed and native code using the instrumentation method.
• Concurrency profiling
• .NET memory profiling
• Tier interaction profiling (TIP)
• Sampling options, such as setting the sampling event and timing interval, or collecting additional performance counter data.
• Instrumentation options, such as collecting performance and windows counter data, or specifying additional command-line options.

In this particular post, we will cover the basic differences between VS2012 and VS2010 Test features, mostly with respect to navigation, running the tests, results and some features that have been deprecated in VS12. Let us now dive into this.

Test List Editor which is very commonly used to run tests in VS10 is deprecated in VS12 and is replaced by Test Explorer. Below screenshots show this difference with a red highlight.

Next question would obviously be, how do I run my tests from Test Explorer and how will this be different from VS2010. The experience of running the tests in VS12 is seemingly different from VS10 for a first time user. When you select Test Explorer in VS12 as shown in the above screen shot (Test – Windows – Test Explorer), it will open the Test Explorer window and you will see ‘Run All’ or ‘Run’ option to execute your tests as compared to ‘Run Checked Tests’ or ‘Debug Checked Tests’ in VS10. Below screen shots will show this difference

In VS12 ‘checkbox’ option is removed which means that if you want to select any particular tests to run, you have to click ‘CTRL + Select’.

Now that we understand the differences on how to run the tests in VS12, let us look at the features that have changed in VS12 while the tests are in progress. Unlike, VS10 where the progress of the test is shown with a distinct ‘In Progress’ icon, this icon is replaced in VS12 with a simple status bar. We have highlighted the differences in the below screenshots

To this point, we have seen the changes on how to Open the tests, run the tests and progress state of the tests in VS12. Now, let us look at the changes that are included after running the tests.

In VS10 test results would open in a TRX file when you click on the Passed/Failed tests, where you can find more details, this is replaced in VS12 with simple intuitive message at the bottom of ‘Test Explorer’ window. This window also has a stack trace, in case of a failed test. Highlighted part in the below window will show the difference.

That is it in this post, in the next post, we will be covering the differences in the Load Test of VS12 retrospect to VS10.