NEW FEATURE AREA Historical Debugger

• Explore important diagnostic events that happened in your application every time you press F5.  Think of this as having things like Procmon/Filemon/Regmon and Fiddler integrated directly into your debugging experience!  Except we do ones like exceptions and events specific to the libraries you are using like WinForms, Webforms, WPF etc.
• Go back to previous break states in the debug history.  Ever written a value from the autos window or locals window down on a post-it and continued on debugging?  Now just go back to the point you stopped at and see the value.
• Turn the historical debugger dial to high and see a history of the code execution, the method calls, the object states and call sites in the tree view of the Debug History window.  Walk back to those points in time and understand what happened.
• Open logs of both diagnostic events and application recording created from running the new Visual Studio Test tools so you can figure out why test are opening this issue on you when clearly - “It works on my machine”.

NEW FEATURE AREA Test Impact Analysis

• Understand which tests to run to verify the changes you are making, as opposed to running every test.
• Understand the specific methods changed that are causing the requirement to run a specific test.
• Ask the question – if I changed this method – what tests would need to be run to verify it?

Debugger

• New: Tear off your datatips and leave them floating on the page while you debug.  No need to go and hover over things time and again.
• New: WPF visualizer.  Inspect the state of visual elements in WPF applications. No need to hunt down the specific property you have set based on which base class it is set in, use the filter mechanism in the visualizer to find it.  See the rendered visual of the element in the preview pane.
• New: WPF logging.  Track issues such as WPF DataBinding rather than asking “Why is may control/window black”
• New: minidump summary page.  Use File->Open to open a minidump and see the data about the environment and modules that were present in the minidump.  Search for your module in the list.
• New: Managed minidumps.  Create minidumps for managed code running on the v4.0 runtime.  Open those minidumps in Visual Studio and explore them.
• New: Thread window.  A whole new thread window designed to help you find the important threads and figure out what they are doing.  Show threads from all the processes you are debugging.  Show the callstacks for all of those threads in the threads window itself.  Group them by flagging them.  Filter threads to only threads that have a particular pattern in their stack.
• New: Export your breakpoints.  Tag and filter them in the breakpoints window.  Export and give to a co-worker to let them import them and figure out a bug in some code you know
• New: Support for tasks in the task libraries as first class citizens
• New: New parallel stacks window shows the stacks of your application in a graphical format
• Improved: Support for opening executables as a project and using that as the debug target
• Improved: Support for inter-op debugging of x64 processes. 
• Improved: C++ – support strcmp/strncmp in conditional breakpoint expressions

Profiler

• New: Tier interaction profiling.  Understand the cost of the calls to the database from your mid-tier. See the SQL queries that make up those costs.
• New: Performance rules. Get guidance on common performance issues that the profiler spots in the performance results.
• New: Thread contention profiling.  Understand which resources your threads are most contending on.  See the contention pattern between threads fighting over locks, and drill down directly to the line of code trying to acquire the resource.
• New: Parallel performance analyzer.  Understand core utilization and how parallel your app is and could be.
• Improved: Directly attach to managed CLR 4.0 applications without pre-setting environment variables.
• New: Profiler works on VPC, Hyper-V and VMWare
• New: Just my code support. 
• New and improved: Integration with LoadTest.  Switch on the profiler in the test settings for the load test.  Open the load test result after the test and click on the profiler results icon in the tool bar.  Drill right down to the code hotspots for your code under load.
• Improved: Summary page shows CPU usage charts and hot stacks rather than just methods.  An action link pane provides support for “one click to action”.
• New: Function details page.  Understand graphically how time is being spent in this method, and the children of that method.  Visualize the source code and show the hot lines line by line with performance statistics visualized in the code at the line level.

Code Coverage

• Improved: Support for code coverage on 64bit code under test.

Whew!  That should keep my blog full for a few months anyway! 

Please pull down the Beta and give it a spin.  It’s available from the product page here:

https://www.microsoft.com/visualstudio/en-us/products/2010/default.mspx

So far I have been running Beta1 and VSTS2008SP1 Side by Side with no issues.  We spent a lot of time in this release focusing on setup, side by side and uninstall so I do think this is way better than in previous releases.

Fix those bugs, and stay posted for more deep dives in all of these areas.  Sorry I didn’t drill deeper, but I wanted to get this post out today.

John

A
B
C
A
B
C 
...

My trick here is simple, and it's to do with "hit count" breakpoints. Set a breakpoint in one of the methods, but go ahead and add a hit count.  You do this via:

Right click on the breakpoint->Select Hit Count...->hit when the hit count is greater than or equal to

Crank the hit count up to something insane (I tend to use 65536).  Now let the program run to the crash point and see that the breakpoint never fires.  Now go to the breakpoints window and look at the hit count for your breakpoint and it will tell you how many times that the code executed that breakpoint.  Now go ahead and change the breakpoint hit count to only hit at that count or one or two below, and now you can debug through the last throes of the stack overflow....

Happy slaying! (Sorry for the lack of pictures)...

John

Prof. Zeller's talk was on the subject of mutation testing to understand test suite effectiveness.  You can find out more, including watching a pre-record of pretty much the same talk I attended, here:

https://www.st.cs.uni-saarland.de/mutation/

While I had some questions about the mutations that Prof. Zeller's team used, and my own concerns about how well test team's manage to make their test suites model after user's behaviours, there is no doubt that this research area is very interesting.  The team is as yet unable to make any conclusions about the effectiveness of these techniques over existing coverage metrics.

My own perspective here is that anything that gets people thinking more about the test effectiveness, and less about specific numbers and metrics in things like statement coverage, the better.  I'm continually asked about "what number of coverage should I shoot for" and "how much coverage does Microsoft aim for".  This is a fruitless task in my opinion.  Our coverage tells you one thing and one thing only - what code was not exercised at all by my test bed.  You then need to make a judgement call about what to do about it.  Is the functionality important?  Is it really dead code?  All of these go into the mix when making the decision to do work and write new tests.

Whatever your own opinion on these, Prof. Zeller's work continues to be thought provoking, so please go check it out,

John

https://channel9.msdn.com/pdc2008/TL59/

In the version of the talk for this year [DTL323], I'll have similar content on VS2005 and VS2008, but will have an updated version of VS2010 to show off some new debugger features if you all are interested.  I may even do a booth talk on VS2010 in more depth since I want to keep the breakout talk balanced with things you can take advantage of today.  If you are headed to TechEd, please go ahead and sign up for this session early.  Last time the room was way too small, and I really like the opportunity to have a meaningful Q&A session during and after the talk.  Not much ability to do that when you are a mile away!

See you all there!

John

John

In the debugger, we always try to minimize the impact we introduce on the debuggee, but there is always some work that we do in a synchronous way.  We alway deal with breakpoints, even when they are in "tracepoint mode",  we always do work when receiving a module load or thread create event.  As I say, we try to minimize the impact of this work, and are constantly looking to improve the performance (and therefor usually the perturbance) on the target application.  However the biggest contributor I see to app performance degradation is the fact that when launching under the debugger, the operating system uses the debug heap. 

The VS debugger uses the standard Windows CreateProcess call to create the debuggee.  See docs for that here: https://msdn.microsoft.com/en-us/library/ms682425(VS.85).aspx. We pass in the create flag DEBUG_PROCESS as documented here: https://msdn.microsoft.com/en-us/library/ms684863(VS.85).aspx

When we do that, Windows enables three heap debugging features:

1. Enabling a tail check. This put a marker on the end of each allocation and when it is freed, the heap manager checks the marker is intact.
2. Enabling free check.  If the process writes to the block after free, the heap manager detects this.
3. Checking paramaters.  The heap manager checks for valid parameters in each call to any of the heap allocation/free functions.

1. and 2. have a sizable performance impact when freeing memory.  For more information on this subject, read Mark Russinovich and David Solomon's book "Windows Internals" Chapter 7.

There are two workarounds for this issue. 

1. Use CTRL+F5 and attach to your application instead.  I realize this is not always ideal.
2. Add the setting _NO_DEBUG_HEAP=1 to the environment settings in the debugger settings for your launch project.  This only works in C++ projects today.  You could force inheritance here by adding that to Visual Studio's environment, which your target application inherits from.

Remember however that you are giving up a lot of validation that the OS does on your behalf.  Catching memory corruption issues early is very important, so don't take this step blindly.

 Happy hunting

John

I was watching a debate on one of the internal support email lists for debugger related things last year when an interesting event occured.  There is currently a limitation in the Windows OS that prevents ModuleLoad events being sent after ~500 of those events have been sent.  The conversation wandered into why you get some gains from performance if you keep the number of dlls down, and ensure they have their base adresses set.  To which there was a reply of (paraphrase):

 "I don't want to think about that, the operating system should keep all that from me, if I want to have thousands of dlls I should be able to"

Which set off alarm bells for me.  The hyperbolae of this statement is of course "don't bother me with constraints".  It also surfaces in how people treat memory allocation and usage in managed code ("Hey - there's a garbage collector - there are no leaks.  Um - why am I running out of memory - it must be a CLR bug").  Complete ignorance of the constraints of systems is a dangerous place to be.

I self identify as an engineer.  But of course many of us who are in professional software development came through Computer Science courses as their degrees.  Looking back on my own education, there were very few courses that armed me with the engineering skills I grew in my first few years as a developer.  I learned (the hard way in many cases) what it takes to engineer solutions that work within the constraints of the systems my customers expected.  I'm in no way bashing Computer Science.  Where would our industry be without the deep thinking and experimentation that carries on to this very day.  But when onboarding software developers, I think we need to do more to make them aware of engineering practices and principles in a more structured way.  In our own team we do this through a combination of mentoring and training, but by no means do we have it down pat (who does).  Different situations may call for different solutions. Until recently I never thought of how much that science vs. engineering discipline is actually implicitly demonstrated as opposed to being explicitly stated, but now I think on it, it's a useful conversation to have with those new hires.

Something to ponder as the next wave of scientist/engineers walk through the doors into industry from university.  Please discuss through the comments section!

 John

Eliminate the “no-repro” bug with application flight recording.

How many times has a developer used the phrase “works on my machine” when responding to a bug?  I know I have :) In many cases that may be true.  By default the VSTS 2010 testing system will record a host of information:

1)      Screenshots of the test in progress so developers can see the symptoms

2)      An environment snapshot to aid in configuration mismatch diagnosis

3)      An application flight recording that can be replayed in the debugger to allow developers to quickly get to the root cause of any issue

The application flight recorder technology is a key element to actually finding and solving bugs faster.  Not only can developers see symptoms, they can understand what their code was up to at the time and get to the problem faster.  To do this, we  took advantage of one of Visual Studio’s core experiences – the VS debugger (which we are responsible for too).  Many developers spend a large portion of their time using the VS debugger for both software construction and debugging.  So we now provide a new set of debugging experiences that focus on looking at the application through the recording, and seamlessly integrate that with the debugger’s existing views, as opposed to building a new tool that people would have to learn and get some experience with.  Any developer will be able to get value directly out of the gate.  This also frees testers from extra rounds of validation (did that bug really exist), or working to reproduce a bug that only show up intermittently or even going to the trouble of setting up a test environment for the dev to repro and debug on live. 

Understanding the test impact of code changes

A common question that testers in software organizations ask is “which tests should I run to ensure quality”.  This is especially critical for teams with a test bed of many thousands of tests as they approach the release point of their software.  Code changes can break the product, but also simply break the tests associated with them.  All of this is cost, and as time progresses from the date the issue was added, it becomes more expensive to understand and fix.

VSTS 2010 intends to provide guidance around which tests have been impacted by changes in newer builds.  As tests run in either automated or manual test scenarios, information is being recorded about which tests touch which code elements in a given project.  This mapping is then transmitted to the test server which maintains a database of mappings.  Team System then makes this information available to all the participants in the software process.

1)      Test managers have a built in report of the test impact of any given new build.  As well as understanding the tests impacted by changed source code, they can see which requirements have changed, or which bugs have been resolved, and by extension the test cases that are related to those.  They can then decide to promote a new build to be “the testable build”

2)      An individual tester can see this same information for their area, and make decisions on which of their tasks they ought to work on first

3)      Developers can see the upstream impact of any change they make, and potentially run the impacted tests to understand if there is a breaking change, or if they have broken the product.

As we get closer to Beta, we should be going deeper on how these features work and what you can (and can't  :) ) do with them, but until then, please go test drive them and give us the feedback!

Keep on fixing,

John

https://social.msdn.microsoft.com/Forums/en-US/vsdebug/thread/f3fcb4fb-8a08-4fa0-8d58-9ed6f3eb1193

The symptoms are that while managed debugging some step commands do not complete as expected, and some breakpoints may be missed if running on hyperthreaded, multi-core or multi-proc hardware. We finally got to the root cause, and I'm glad to announce that we have now made a fix available for this issue.  You can grab that here:

https://code.msdn.microsoft.com/KB957912/Release/ProjectReleases.aspx?ReleaseId=1796

John

 https://channel9.msdn.com/pdc2008/TL59/

The topics I covered tricks in stepping, breakpoints, symbols, multi-threading and post-mortem.  Judging from the evals the reception was pretty good, and I really wish I had the time to go deeper and go over more stuff.  I was especially pleased with being highest rated talk in the TL track for job relevance :).  I'd like to hear if folks would like to see more of this at PDC09.  We are considering whether we should have pre-con sessions on debugging, profiling and diagnostics to really go deep.  For the chap "D" from company "C" who was able to answer my opcode question, please get in touch.  Other talks  that I would encourage you to check out:

• Steve and Ed talking about the great performance tools coming in VSTS 2010 https://channel9.msdn.com/pdc2008/TL24/
• Habib and Justin talking about our test impact and historical debugging features in VSTS 2010 https://channel9.msdn.com/pdc2008/TL03/ 
• Brian "I just beat JoC in the evals" Harry talking about upcoming Team Foundation Server goodness https://channel9.msdn.com/pdc2008/TL52
• Gert "I dream in T-SQL" Drapers talking about upcoming goodness in the data tools in VSTS 2010 https://channel9.msdn.com/pdc2008/TL45

Steve doing his "Let's pretend this says System.String" line is one of my favourites of all time.

I'm not sure hte title of Habib and Justin's talk did justice to what we were showing off, which was the first introduction to historical debugging coming on line in VSTS 2010.  However we received the highest evaluation scores for all of PDC (by some margin) for the Hands On Lab that Justin produced to demo this feature.  We literally think it will change the way people debug, and will become increasingly important in diagnosing issues as platforms become more distributed and cloud based, e.g. Windows Azure.

Cheers

J