Unit Testing

Customer visits are one of the more fun parts of my job.  Last week, Scott Wiltamuth and I visited three large enterprise customers to understand how they are using (or not using) our tools and technologies, what works well for them, what doesn’t work well for them, what some of their major pain points are, etc. Scott runs the C# team and is a Developer Division veteran.  Each customer had their own set of constructive feedback, but a common theme emerged – all of them were interested in knowing more about Visual Studio 2005 Team System and how that can help them with team development.  These three customers were particularly interested in how Visual Studio 2005 Team System handles Unit Testing.

The Visual Studio 2005 Team Test Edition equips customers with tools to manage a wide array of unit tests, manual tests and web tests. Unit Tests work on functions and methods and Visual Studio 2005 Team System makes it easy to manage and run unit tests. The customers reiterated basic problems most people face with unit testing – test creation, test organization, result reporting etc. Visual Studio 2005 Team System has implemented features to address all these problems. For example, to create unit tests, simply right-click in your code and select Create Tests. The Test Manager allows you to select which tests you need to run. You can group tests to test data access, business logic or other functionality in your code. Test results are easily published to the Team Foundation Server enabling you to share results via a portal site. Visual Studio 2005 Team System treats testing as a top-tier activity helping reduce the risks inherent with delivering complex applications. For more information, see Visual Studio 2005 Team System: Enabling Better Software through Better Testing.

Namaste!

Comments

• Anonymous
  March 24, 2005
  Considering the latest (heated) debate around MS pricing of Team System I'd like you to think of this - how many ENTERPRISE developers who have ANY SAY in decisions about purchasing Team System do you think are reading this blog? These would be the ones to follow the links and learn more (I find it hard to beleive that there are still people who don't know what Team System is, though).
  My guess is that this post will only attract more revolt from small and medium size shops who are dissapointed with MS position of marketing and intent of selling Team System to big corporations only.
  I really don't understand MS position. If you want to sell Team System to "big guys" only, why are you promoting it through blogs? Note that for example I hadn't even heard about Rational's product (and others) that is supposed to compete with Team System up until Team System's pricing was announced and the whole debate started. In contrast, I have been flooded with information on Team System. In a way, MS did "here's our new beatiful system, developers; you can look, but you can't touch".
  
  I think this is a wrong channel for the message you're trying to send. Don't "sell" Team System to your blog readers - they are hard working developers, small ISVs and people generally excited about technology, but not the ones who will be able to use it nor decide on if to buy it.
• Anonymous
  March 24, 2005
  Somasegar,
  
  I think its really great that you are proactively going into the market and gathering feedback from your users. You make comment to the fact that you visited 3 LARGE Enterprise customers for your market study... and you do so in the context of Unit Testing.
  
  My question for you is do you believe Unit Testing is only reserved for those large enterprise corporations? Small development shops, ISV's, consulting companies, independant contractors, students, hobbyists... what is there role in your vision of Unit Testing?
  
  Is software testing only done in the context of large teams, or is a part of general software engineering that teams and individuals use to build software?
  
  I think people like want to know what we are going to get when VS 2005 ships... without Team Services, will I have unit testing – test creation, test organization, result reporting etc. without having to buy Team Systems and a supporting OS?
  
  JavaKid
• Anonymous
  March 24, 2005
  The comment has been removed
• Anonymous
  March 24, 2005
  VisualStudio tools and the .NET Framework have revolutionized software development. It is now possible for a lone developer to produce a commercial-quality application.
  
  How many visits have you made to companies with one or two developers in the past three months? And does your answer have anything to do with the mess VSTS is in?
  
  -Don
• Anonymous
  March 25, 2005
  Okay, maybe this isn't the place to ask this, but does anyone know if you can use VSTS Developer edition without a server and still create and manage unit test projects, etc.
  
  Sure you won't get all the features enabled with server, but that would be great if I could just have the integrated unit testing and code coverage.
• Anonymous
  March 25, 2005
  Mark- Yes, you can use the Developer edition without the VSTS TFS server (for example, you can use VSTS developer edition with SourceSafe). I haven't seen a formal list of which features require the server, but presumably unit testing and code coverage will run without the server.
  
  -Don
• Anonymous
  March 25, 2005
  Somasegar,
  
  I felt I had to make another comment; not just to criticize Microsoft, but I really want you guys to improve.
  
  Anyway, your blog is beginning to sound like a PR exercise. If we take the latest entry there are two problems with it:
  
  1. You make it sound as unit testing is something brand new; whereas it has been around for a few years now and there are even tools that work with .NET.
  2. You appear to duck the heated issue with Microsoft’s approach. Months ago when Microsoft announced unit testing would be in Team Systems there was a huge furor as to why it was only in Team Systems and not other VS products. By at least articulating your view on this in your blog you would have gained more credibility. Even if the view was “tough, that’s our decision and we are sticking to it” or hopefully “over time we will push unit testing down the VS stack”. Any view is better than avoiding the issue.
  
• Anonymous
  March 26, 2005
  
  Looks like this blog did hit a raw nerve with a bunch of folks. Thanks for the feedback. Let me try to address some of the key feedback points that I think I am hearing. I was away at an offsite for the last 3 days and hence the delay in responding to the feedback.
  
  Unit testing is not something new. We are providing unit testing for the 1st time as part of Visual Studio Team System. Of all the things that are coming out in VSTS, unit testing was the one constant thing that these customers wanted to talk a lot about, and hence I wrote about that.
  
  Whenever I am on the road, I meet with customers and ISVs (small, medium and large). With Visual Studio, I really want us to be providing tools that are relevant to all kinds of developers - whether they are in a small shop, or a large enterprise.
  
  3 weeks ago when I was outside the US, I met with a handful of what I call "small" ISVs.
  
  Unit testing or software testing in general, is done by ALL development teams - small or large. Microsoft is a classic example. We have teams that are literally thousands to developers to teams that have a handful of developers. I want our testing tools to be used by all the teams. The same is true for developers outside as well (whether they work alone, in small teams or as part of a large team).
  
  In v1 (VS2005), these tools are going to be available only on VSTS Developer Edition and VSTS Suite. Over time, we will need to figure out which tools (and when) we can push down the VS stack. We won't get there for v1.
  
  You do not need the TFS server to be able to use our testing tools, including unit testing. This is also true of the statis code analysis tools, code coverage tools, profiling tools, that are available as part of VSTS Developer Edition.
  
  - Somasegar
  
  
  
  
• Anonymous
  March 31, 2005
  The comment has been removed
• Anonymous
  April 01, 2005
  Michael - appreciate your words of support. Can I get your email id? would love to get more feedback on the MSDN events. I work as a Business Manager in the Developer Division. My email id = devic@microsoft.com