Visualizers for native types are specified in .natvis files. A natvis file is simply an xml file (with a .natvis extension) with its schema defined in <VSINSTALLDIR>\Xml\Schemas\natvis.xsd. Visual Studio ships with a few natvis files in <VSINSTALLDIR>\Common7\Packages\Debugger\Visualizers folder. These files contain visualization rules for many common types and can serve as examples when writing visualizers for new types.

Not trying to provide all the details about .natvis files (as it is a topic for entire separate blog/article), I will rather try to demonstrate briefly the flexibility and power of the framework that can be used beyond simplistic visualization of data.

Let’s pretend that you have your own data type:

When you look at variables of this type in debugger Watch, Locals or Auto window, you see the expected data:

Now copy the below XML snippet into the file with extension .natvis, and place it in %USERPROFILE%\My Documents\Visual Studio 11\Visualizers\. Besides many different ways to display your types in friendly ways, you can create synthetic expansions:

mytypes.natvis:

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="https://schemas.microsoft.com/vstudio/debugger/natvis/2010">
  <Type Name="Rectangle">
     <Expand>
       <Synthetic Name="Am I a square?"
       Condition="height==width">
         <DisplayString>Yes I am.
         </DisplayString>
       </Synthetic>
       <Synthetic Name="Am I a square?"
       Condition="height!=width">
         <DisplayString>No I am not.
         </DisplayString>
       </Synthetic>
     </Expand>
  </Type>
</AutoVisualizer>

And now your visualization is richer and allows you to see not only raw data of your objects, but also some post-processing of it (which can be very handy, especially if you are dealing with complicated data types that require some non-trivial evaluation).

Enjoy!

Update: more data on this subject is available here: https://code.msdn.microsoft.com/Writing-type-visualizers-2eae77a2

then you end up with an error like that:

The rationale behind usage of local drives is that VS Profiler attempts to make logging to the file fast, while usage of network drives might introduce logging delays (up to blocking your application execution in some cases).

However, if you absolutely have to use your network drives for that purpose, you can achieve that through usage of Symbolic Links. A symbol link can be easily created with mklink utility. For example,

mklink /D C:\Foo \\MyNetwork\MyValut

After that you can use C:\Foo as a location for profiling reports. Keep in mind, though, that your network access can be slower than a local drive access, and it can impact your profiling session.

If you install recently published VS2010 SP1 Beta, you can discover that VS Profiler team enriched Silverlight profiling user experience. Previously I published few posts about VS2010 usage for those purposes, either through VS Profiler command line tools or VS2010 UI. Both approaches led you to the final results or figuring out the bottlenecks of your Silverlight application, but they require number of manual steps to be done.

In VS2010 SP1 however Silverlight projects are fully recognized by VS Profiler, and you can conduct your performance investigations much easier.

For example, you might use the Silverlight Text Editor Sample, and open this project in VS2010 SP1:

Now, if you wish to create a performance session for this project, you can do it in a usual way through Performance Wizard:

You will see your project in the list of possible targets (it was not there before the final integration of Silverlight projects and VS Profiler), and you can select it there and proceed with the Wizard that creates and starts a performance session for you:

When you close your application – the session is over and profiling results are presented so you can analyze the data:

Enjoy!

While using Visual Studio Profiler either from Visual Studio UI or through command line tools, you may wish to collect more/different information using specific CPU Performance Counters. The collected data depends on the profiling mode. As usage of CPU Performance Counters considered an advanced technique, let me explain what is the proper way to use them and how to interpret the results.

First of all, CPU Performance Counters are not operating system performance counters, as they are implemented by CPU vendors inside of a CPU die.

If you run inside of a Virtual Machine or on a Hyper-V enabled machine – you have no access to CPU Performance Counters (as I mentioned in my previous post). This is the current limitation of virtualization technologies.

Let’s review 2 basic modes of VS Profiler – Sampling and Instrumentation – and discuss how the counters are used in each one of them.

Sampling

In Sampling mode, the profiler periodically interrupts an execution of your application in order to take a sample of it. The regularity of interrupts is achieved through usage of one of the CPU Performance Counters. By default – it is 10,000,000 Non Halted CPU Cycles. You can change the predefined sampling interval or the counter itself. When you change the counter, that means that you switch to a different event as a trigger of your samples. If the default Non Halted Cycles event causes your samples being collected when your application is actively using CPU, then switching to Last Level Cache Misses event, for example,  will cause samples being generated on cache misses (so, if there are no cache misses – no data is collected).

VS Profiler supports sampling on only one counter at a time. Therefore, you can use only one counter when you configure your sampling session.

When you look through results of sampling, you will not see separate data columns with the counter name. Remember, that usage of specific counter in this scenario means that your samples were triggered by the counter event. So interpret your samples (inclusive, exclusive, function samples, module samples, call trees etc) appropriately.

To select a specific counter for your Sampling session in Visual Studio, go to Properties of your Performance Session (right click –> Properties), and select the Sampling tab. In Sample event drop down list switch default Clock cycles to Performance counter.

Now you can select one of the provided counters and set its Sampling interval.

The full set of counters provided by CPU vendors is quite big. VS Profiler suggests a sub-set of them, with an ability to add more counters to the selection if necessary. You will see all currently supported counters in VS UI. If you use command line tools for your data collection, you can see the counters through the following option:

C:\MyProjects>VSPerfCmd.exe /querycounters
Microsoft (R) VSPerf Command Version 10.0.30319 x86
Copyright (C) Microsoft Corp. All rights reserved.

Portable counters:
        InstructionsRetired
        NonHaltedCycles
        L2Misses
        L2References
        ITLBMisses
        BranchesRetired
        MispredictedBranches
Platform counters:
        DCULinesIn
        DCUModifiedLinesIn
        . . .

To use one of them, you need to provide the counter name in /COUNTER option of VSPerfCmd.exe when you start your data collection. VSPerfCmd.exe /? will give you an exact syntax. For example:

VSPerfCmd.exe /attach:MyApplication.exe /COUNTER:L2Misses,1000

Instrumentation

In instrumentation mode, VS Profiler records timestamps at function entries and exits. If you add CPU Performance Counters to the session, their values will be collected at the same time, and you will have separate columns of data for each counter. Unlike the sampling mode, you theoretically can add to instrumentation profiling session as many CPU counters as you wish. Practically, however, the number of counters you can use simultaneously is limited by a number of registers that are implemented in your CPU for that purpose. Sometimes it could be as few as two registers only, and VS Profiler always tries to acquire one of them for internal usage.

To add some Performance Counters to your Performance Session, go to the session Properties and select CPU Counters tab. There you can enable and select counters you want to add to your instrumentation data collection.

When you look at collected results, you will see new columns of data:

If number of counters that you select exceeds current CPU limitation, some of them will be dropped from the collection and you will get a WARNING message in the Output window:

Warning VSP2318: The monitor was unable to initialize counter 'Mispredicted Branches'.

While using command line tools, you can have multiple /COUNTER options to add several counters to your instrumentation session:

VSPerfCmd.exe /start:trace /output:MyFile.vsp /COUNTER:c1 /COUNTER:c2

Enjoy!

Does your managed code uses C# lock statement or Monitor class?

If yes, and if you would like to get a clear picture of all contentions in your application, you might want to check out the new profiling mode of VS2010 Profiler – Resource Contention concurrency profiling. It collects and presents information about all contentions, when synchronization resources cause your threads to be blocked in an attempt to acquire the resource.  Being contended, a thread doesn’t run. It waits. It wastes the time. It decreases the performance you wanted to gain from your super powerful multi-core machine, when you designed your application as a multithreaded in the first place.

For detailed explanation about how to collect contentions data, and what are the best ways to investigate it, follow the links.

1. Resource Contention Concurrency Profiling in Visual Studio 2010 – June 2010 MSDN article
2. Resource Contention Concurrency Profiling in Visual Studio 2010 – Performance Investigation Flows – a follow up blog to the MSDN article.

Resolve synchronization issues, if your code has any, and boost the performance of your application!

We will use the same Breakout Game example, which is open as a project in Visual Studio at the beginning:

As Silverlight project is not currently recognized by the Performance Wizard, you better start with creating an empty Performance Session. You can do it from the Analyze menu (Analyze –> Profiler –> New Performance Session):

or by clicking a New Performance Session button in Performance Explorer:

When a new Performance Session is created, it has empty Reports and Targets nodes. Now right click on Targets, select Add Target Binary and pick up c:\Program Files\Internet Explorer\iexplore.exe (or c:\Program Files (x86)\Internet Explorer\iexplore.exe, if you work on x64 machine):

Next step is to right click on iexplore.exe node, select Properties, and then select Launch tab in the Property Pages window. The “Executable to launch” is set already, but you should add Arguments and Working directory for your application.

The last preparation before you can start measurements is to make sure your Symbols setting includes the binary directory where the application is compiled. This is currently required for correct symbols resolution. You can find the Symbols setting in Tools –> Options –> Debugging –> Symbols (it is under Debugging and not Performance Tools, as both Debugger and Profiler share the same setting for Symbols).

Now you can click Start Profiling button in the Performance Explorer, and Internet Explorer will be started under profiler, running your application. As I mentioned in the previous post, there could be number of IE processes while your application is running. Yet Visual Studio attaches Profiler to only first of them, and doesn’t reattach it automatically to other spawned processes. Thus, you have 2 options here – either to eliminate creation of child processes by IE through the registry (modifying TabProcGrowth value), or attach profiler to additional IE process (one or many) before you start running your scenario. Simultaneous profiling of several processes is supported, and you will actually profile all IE processes that you attach the profiler to. Those processes that run your application will generate a valuable profiling data, while rest could be later ignored.

You can attach to additional processes using Attach/Detach button of Performance Explorer:

In our example, you can see in “Attach Profiler to Process” dialog that the profiler is attached already to one IE process, and now you will attach it to yet another one.

After you finish your scenario, close IE window and profiling results will be automatically open for you in Visual Studio, so you can navigate it using all views as usual:

Enjoy!

This feature, however, is only partially integrated with Visual Studio UI, and you need to use command line tools to collect the data. After that, you can open the resulting .VSP file in Visual Studio as you normally do after other types of profiling, and enjoy rich visualization experience while investigating your performance problems.

Yes, we know it is not as cool as clicking VS buttons. We are working on a better experience. Yet I thought it is worth to emphasize that there is a way to use command line tools for customers that seek performance improvement in their Silverlight applications.

Let assume you develop an application like Breakout Game.

You compile it inside VS2010, run it, and want to profiler it. Performance Wizard tells you that this project can not be launched by the profiler. Alas. Time to go to the command line tools. Click you Start button, go to All Programs and find there Microsoft Visual Studio 2010. There, go to Visual Studio Tools and launch a Visual Studio Command Prompt (2010).

This command line window has predefined PATH that includes profiler command line tools:

Now go to the directory where your Silverlight binaries are compiled:

>cd c:\Breakout\Breakout\Bin\Release

We are ready to start our profiling session.

1. VSPerfClrEnv /sampleon
2. "c:\Program Files (x86)\Internet Explorer\iexplore.exe" C:\Breakout\Breakout\Bin\Release\TestPage.html
3. VSPerfCmd /start:sample /output:MyFile /attach:<PID of iexplore.exe process>
4. Run your scenario
5. VSPerfCmd /detach
6. VSPerfCmd /shutdown
7. VSPerfClrEnv /off

Steps above require some explanation. In 1 – you prepare environment that is needed for Silverlight profiling, and in 2 you start your hosting process (iexplore.exe, in our example) within that environment. This is essential, as if you start IE by clicking an icon on your desktop, you will get no Silverlight profiling data at the end. Step 3 starts profiler in Sampling mode, and attaches it to the process of your choice. Keep in mind that you might have multiple IE processes running, so you need to choose one that runs your code. If you are not completely sure which one is it, or if your code runs in several IE processes – well, attach to all of them! You can do it either running “VSPerfCmd /attach:<PidN>” several times, or enumerating all PIDs in a single command: “VSPerfCmd /attach:PID1,PID2,PID3”. After profiler is attached, you run your scenario in step 4, and detach profiler from all target processes in step 5. Detaching is not enough, as you need to shutdown the profiler in order to finalize your result file (step 6). Step 7 cancels changes of the environment that are made in step 1.

It could be that you need to profile your application from the very beginning. In this case steps 2 and 3 can be combined into one, where you launch IE under profiler:

VSPerfCmd /start:sample /output:MyFile /launch:"c:\Program Files (x86)\Internet Explorer\iexplore.exe" /args:”C:\Breakout\Breakout\Bin\Release\TestPage.html“

The caveat here is that if IE process spawns a child process that will run your application, the profiler will be still attached to the “father” IE process, as it doesn’t re-attach itself automatically to spawned processes. You may, however, eliminate creation of child processes by IE8 through the registry, modifying TabProcGrowth value.

Please make sure that you run all the above steps from the directory where your binaries are located. Until we complete the integration of this feature with VS, your function names will not be properly resolved otherwise.

You can kill you IE process now. Open the MyFile.VSP in VS2010 as you would do normally for profiling results.

Formally, that would conclude an official support of Silverlight profiling in VS2010 – Sampling mode through command line tools.

However, if you want to try .NET memory or concurrency resource contention profiling of your Silverlight application – I won’t stop you. This is not currently supported and tested modes of work, but they might work for you and help you to see your performance problems from a different angle.

For .NET memory profiling, use “VSPerfClrEnv /samplegc” or “VSPerfClrEnv /samplegclife” in your step 1 to get either allocation profiling or allocation and lifetime profiling.

For concurrency resource contention – use “/start":concurrency,ResourceOnly” in the step 3 above.

Oren Nachman from Sliverlight team is another excellent source for Silverlight profiling information. In his great article he provides several additional tricks and options that you can find quite useful.

Enjoy!  ]]> https://docs.microsoft.com/archive/blogs/mgoldin/resource-contention-concurrency-profiling-in-visual-studio-2010-performance-investigation-flows Thu, 22 Apr 2010 09:02:23 GMT https://blogs.msdn.microsoft.com/mgoldin/2010/04/22/resource-contention-concurrency-profiling-in-visual-studio-2010-performance-investigation-flows/ Visual Studio 2010 brings a number of innovations to multithreaded and parallel computing. One of them is Resource Contention Concurrency Profiling. John Robbins gives a really great first look at this feature in his Wintellect VS 2010 Beta 2 Concurrency Resource Profiling In Depth First Look post. The June 2010 issue of MSDN magazine contains the Resource Contention Concurrency Profiling in Visual Studio 2010 article with a detailed explanation of this profiling mode, including various ways of data collection and presentation and some guidance to efficient data analysis. Here I would like to append the article with some additional valuable information.

The table below presents a subset of VS2010 performance data views. Those views are useful while investigating results of Resource Contention Concurrency Profiling.

The Resource Details view, for example, is the following one:

There are several built-in transitions between profiling data views inside VS2010, and they are explained in the article. Let me briefly summarize those transitions through the below diagram, which presents possible performance investigation flows. When you analyze your data, do not feel obligated to use VS2010 views in that exact order or sequence – you can always select any desired view through a drop-down combo box of VS2010 that enumerates all views. The diagram is just a suggestion how that can be done.

The transitions that are enumerated on the diagram are:

1. Move from Summary view to Resource Details view (chart) by clicking a resource name in the Most Contended Resources table

2. Move from Summary view to Thread Details view (chart) by clicking a thread ID in the Most Contended Threads table

3. Switch from Resource Details view to Thread Details view by clicking on a chart label, representing the thread

4. Switch from Thread Details view to Resource Details view by clicking on a chart label, representing the resource

5. Move from Resource Details view to Function Details view by double clicking on a function name in Contention Call Stack panel

6. Move from Thread Details view to Function Details view by double clicking on a function name in Contention Call Stack panel

7. Move from Resource Details view to Contention view by clicking on a Total label of the chart

8. Move from Contention view to Resource Details view through a context menu (right-click) of a resource

9. Move from Thread Details view to Processes/Threads view by clicking on a Total label of the chart

10. Move from Processes/Threads view to Thread Details view through a context menu of a thread

When you open the profiling results file, you see a Summary view that helps you to select a possible investigation direction. Some options to consider here are:

• Start in Summary view, detect a contention resource or a thread you want to inspect closely (because it has too many contentions), then explore data through the transitions:
  • 1 – to see resource details, or 2 – to see thread details
  • 4 and 3 – to pivot between resources and threads details
  • 5 and 6 – to see details of function causing the contention
• Start in Summary view, use drop-down combo box to switch to Contention view, sort all resources, select one of them, then use transition 8 to get to the resource details, followed by transitions 4, 3, 5 and 6 as required.
• Start in Summary view, use drop-down combo box to switch to Processes/Threads view, sort all threads, select one of them, then use transition 10 to get to the thread details, followed by transitions 4, 3, 5 and 6 as required.

In post-VS2010 Beta 2 releases you can use Last Level Cache Misses and Last Level Cache References counters for that purpose:

Profiling started.
Profiling process ID 2752 (<Your Application Name>).
Process ID 2752 has exited.
Data written to C:\<some path>\Report100114.vsp.
Profiling finished.
PRF0025: No data was collected.
Profiling complete.

There could be number of reasons for that.

The first question I will ask: “Is your application CPU bound?”. If your process doesn’t use CPU intensively enough during the profiling session (it waits on IO instead, for example), than the process becomes “invisible” for default profiler sampling (as events that trigger a sample happen when other applications are running – see my previous post explaining different samples and what they mean). If you use advanced sampling, configuring some specific CPU Performance Counter (through either /COUNTER command line option, or using Visual Studio UI) – for example, L2 Cache Misses – then again, if your application doesn’t generate samples of that specific event, no data will be collected. In those cases you may try to switch to Instrumentation mode.

If, however, you are sure your application is CPU bound as you see CPU spikes during its execution, yet Visual Studio claims “no data was collected” – please look into Properties of the collected file (you can get it by pressing F4). In some rare cases you will see all collected Samples are set to 0. Which means no sample was collected at all (normally, even if your application is not CPU bound – samples are collected, but they belong either to kernel mode, or other applications; Total number of samples equal to 0 is something really strange). The question I will ask in this case is: “Did you use any other diagnostic tool recently? Especially those, that are not developed by Microsoft?” And if you answer “Yes”, please reboot your machine and try again. The reason is that some tools may leave your machine hardware in a state that breaks functionality of Visual Studio profiler. Reboot will fix it for you.