Chia sẻ qua

The purpose of a good test case

  • Bài viết

Many experts have written articles and devoted chapters of books on the attributes of what constitutes a ‘good’ test case. Unfortunately, most books repeat a common (yet limited) perspective that a good test case has a high probability of finding bugs, and Kaner goes to the extreme by stating “A test that did not reveal a problem was a waste of time.” I pondered this for a while and thought, if I run a test to prove that a feature functions as expected is that really a waste of time? Isn’t it a good thing that testers actually spend some time executing tests that demonstrates the product does what the customer expects it to do? Or do we simply want to restrict the value of testing and reduce our testers to keyboard monkeys who bang away at the keyboard in search of bugs? (I believe limiting the perspective of the purpose of a test in testing literature has only perpetuated the faulty assumption that the purpose of testing is to simply find bugs; a point I previously dismissed). So, what is the purpose of a test?

Jorgensen states in his book, Software Testing: A Craftsman’s Approach that“a test case has two purposes: either to expose an error, or to demonstrate correct execution.” This explanation broadens the purpose of a test case to include both verifying the product meets the requirements, and revealing potential errors or defects. I tend to agree more with Jorgensen in this matter, but also think the purpose of a test case involves even more.

There are several objectives of any effective test whether manually executed or automated. The rationale for test case usually falls into one of seven categories. Each category provides value to the organization in different ways, but they all essentially function to reduce risk and qualify the testing effort. So, a good test provides some measurable value to the organization with the explicit purpose of:

  • Verifying conformance to applicable standards and guidelines and customer requirements

  • Validating expectations and customer needs

  • Increasing control flow coverage

  • Increasing logic flow coverage

  • Increasing data flow coverage

  • Simulating 'real' end user scenarios

  • Exposing errors or defects

Comments

  • Anonymous
    September 17, 2006
    The comment has been removed
  • Anonymous
    September 18, 2006
    The comment has been removed
  • Anonymous
    September 20, 2006
    The comment has been removed
  • Anonymous
    September 21, 2006
    The comment has been removed
  • Anonymous
    September 29, 2006
    >(BTW..this is why I disagree with Kaner's assertion that we test to "maximize bug count.")

    You seem to miss the point that this is a single example on a list of many possible motivations for testing.  <i>Sometimes</i> we <i>might</i> test with the goal of maximizing the bug count.  Moreover, I don't think he's talking at that point about artificially or misleadingly inflating the bug count; I think he's making a distinction between, given the same amount of time finding a few really super-important bugs, investigating and reporting them in great detail; and finding a whole bunch of bugs (with distinct root causes) irrespective of their relative importance, and investigating and reporting them in somewhat less detail than in the first case.

    Note that in some contexts, management does indeed value the fallacious idea that good testing equals the number of bugs discovered.  There is much education to be done there, for sure.  In my view, counting bugs (or test cases) is a practical guarantee that someone will be misled.

    ---Michael B.
  • Anonymous
    October 01, 2006
    The comment has been removed