Scripting Exploratory Smack Down

Vinayak writes: "I do feel Exploratory Testing has importance in SCRUM-like projects and also scripted testing has its role in specific contexts. I would really like to see your views on how do you think test design, test documentation, and test execution can optimally take place in SCRUM-like situations."

It's not either-or, it's both.

Scripted tests can be useful, especially if they are automated and cover Paths Which Must Always Work. Unit tests tend to be scripted tests, and I imagine that even the most ardent Scripted Tests Are Evil proponent would agree that unit tests are A Good Thing. Especially if you realize and remember that scripted tests will only ever find regressions in your program's behavior, not heretofore unfound bugs. If your scripted tests all pass, that merely means that your application has been trained to pass them. It doesn't (necessarily) mean that your product will pass any other test you might dream up.

Exploratory testing can also be useful, in part precisely because its very nature means your application can't be trained to pass it. I don't see how testers using their brains to identify The One Test Which Seems Most Useful To Run Right Now could ever be a bad thing. Great Testers are always exploratory testing, even when they are supposed to be executing scripted test cases. Their first test may follow the script exactly and ensure that (Presumably) Very Important Path works as expected; after that, however, Great Testers use the script as a starting point for identifying other tests which seem interesting.

By the way, I don't know how to write a scripted test case without doing exploratory testing as I write it.

As for how testing changes when you're doing Scrum, I don't see that using Scrum, or any other program management methodology for that matter, changes The Right Way To Do Testing. Mostly because there is no Right Way! I find that every situation is different and requires a different solution. My current process goes like this:

1. Get a feel for the problem however seems to make sense: read specifications, poke around the application for a bit, talk with the developers, talk with whomever tested it last time, and so on.
2. Formulate a plan. I almost certainly don't know enough at this point to create The Perfect Plan, so I don't attempt to do so. Instead I build a plan that seems Mostly Good Enough, For Now Anyways.
3. Start executing on the plan. As I do this I will likely discover that portions of it are suboptimal.
4. Go to Step 2.

Note that this process never reaches The Perfect Plan! Even if I did somehow manage to construct one that is exactly right today, by tomorrow something will have changed and my plan will no longer be perfect. My plan enables me to find issues today, so it is plenty good enough.

Let me know how this process works for you!

*** Want a fun job on a great team? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.

Comments

• Anonymous
  February 20, 2008
  >>Great Testers are always exploratory testing I would (and have) even said that good testing is always exploratory. Which, of course, always makes me question the excitement around the term. You just can't test well if you don't approach testing in some exploratory fashion - but the same is true in nearly everything else I do. I just see no need to use exploratory as a term to describe driving, cooking or any other task I perform while thinking.

• Anonymous
  February 25, 2008
  The comment has been removed

• Anonymous
  February 29, 2008
  Dead on.  There's a third tier to these two, which is the constant process of expanding the testers' knowledge; from knowledge comes new insights on exploring the product; from new paths in exploratory testing come test scripts.  (Just remember to cull old dead test cases every so often!)