Share via

The purpose of software testing

  • Article

For the past few years I have been teaching new testers at Microsoft. Before beginning the Microsoft ‘assimilation’ process I always ask each new group to define testing in one sentence and list the primary objectives of software testing. Invariably, the majority of folks new to software testing believe the purpose of software testing is to ensure quality and the primary objective is to find bugs. Unfortunately, this is an antiquated view of testing, and perpetuating these limited perspectives of testing simply retards the role of testing from maturing into a professional discipline in the field of computer science.

 

Testing doesn’t ensure quality

 

The purpose of software testing is to assess or evaluate the capabilities or attributes of a software program’s ability to adequately meet the applicable standards and customer needs. So, when I hear a tester say the purpose of testing is to ensure quality I immediately ask them, “how can testing ensure quality?” Think about this rhyme for a moment:

 

I don’t get to drive the train,

I don’t get to ring the bell,

But, if the train jumps the tracks

See who catches hell!

 

I don’t get to discuss the needs of the customer, I don’t get to translate those needs into requirements, I don’t get to design or code the solution, and when I find a bug in the software I don’t even get to fix it. I can’t even demand that every bug I find gets fixed. In fact, I know that a number of reported bugs (including ones I think should get fixed) simply won’t get fixed. So, how in the hell can I ensure quality?

 

What I can do as a professional tester is to design the appropriate tests to evaluate the attributes and capabilities of a software program’s ability to handle valid and invalid inputs under various conditions. This approach not only has the potential to demonstrate a program’s ability to meet customer needs and expectations, but also includes identification of flaws or deficiencies that could negatively impact the customer (or long term maintenance of the product).

 

The purpose of testing is not to find bugs

 

The primary objectives of testing include:

  • Qualifying a software program’s quality by measuring it’s attributes and capabilities against expectations and applicable standards
  • And (probably the most important role of testing is simply to) provide information

 

Although I vehemently disagree the primary objective of software testing is to find bugs, I am not surprised when new testers state bug-finding as a primary purpose of testing because it is so heavily emphasized in many books and proselytized by so-called ‘experts’ in the field. Even Cem Kaner states in his “best selling” book on software testing, “A test that reveals a problem is a success. A test that did not reveal a problem was a waste of time.” Bugs are simply a by-product of testing.

 

So riddle me this, if testers are spending all day beating on the product in attempts to find bugs, who in the hell is verifying the product does what it is supposed to do? I guess performance testing is a waste of time if the performance measurements never fall below acceptable levels. But wait, how would we know if we didn’t “waste the time” to measure performance to begin with? I guess we would have to waste time to actually measure the capabilities against the expectations or applicable standards, or say…”yep, in my ‘expert’ (biased and subjective opinion) it’s good enough!

 

My teammate Alan also recently wrote an excellent post entitled Testing != bugs, where he talks about pushing quality upstream through defect prevention. This is a common theme we discuss in our training for new software testers. We advocate the expansion of the testing role beyond bug-finding. We encourage testers to explore opportunities, seek out solutions to difficult challenges, and unlock their potential to become a greater asset to the organization throughout the entire software process. Fortunately, other companies besides Microsoft are beginning to understand that a highly trained and competent test team can provide much greater value to the organization beyond the simple finding bug job.

 

But, personally, I think even many experienced testers have yet to realize their full potential or the significant impact the testing role can provide beyond finding bugs. But, I do think within the next few years we will see a transition in the industry away from hiring a plethora of “expert bug-finders” towards the maturation of a respected discipline of professional software testers.

Comments

  • Anonymous
    July 12, 2006
    The comment has been removed

  • Anonymous
    July 14, 2006
    The comment has been removed

  • Anonymous
    September 15, 2006
    Many experts have written articles and devoted chapters of books on the attributes of what constitutes...

  • Anonymous
    October 24, 2006
    The comment has been removed

  • Anonymous
    October 25, 2006
    The comment has been removed

  • Anonymous
    January 20, 2008
    PingBack from http://softwareinformation.247blogging.info/i-m-testy-the-purpose-of-software-testing/

  • Anonymous
    May 01, 2008
    The comment has been removed

  • Anonymous
    May 02, 2008
    'i.m.test' you said that """ The purpose of software testing is to assess or evaluate the capabilities or attributes of a software program’s ability to adequately meet the applicable standards and customer needs""" when u say this u are actually negating your own statement " Testing doesn’t ensure quality  " ... any step you are taking to ensure that customer needs+expectation are met , u are actually ensuring the customer satisfaction... and customer satisfaction is linked with the quality attribute of any product or service... its similar to a mathematical rule of equality that if x=y and y=z then x=z. No doubt simple testing (part of software testing process) plays some part of ensuring the quality since its a part of Quality Control area ... where as Quality Control + Quality Assurance covers major part of Total Quality Management ... which on the basis of continual improvement ensure the customer satisfaction.. in short what i am trying to say is that... < if somebody says that when we do software testing we are not ensuring the quality rather simply collecting information, infact he is ensuring quality because like implementing best practices in other area of SDLC we can ensure quality of product and service similarly by implementing best practices of software testing we no doubt impart quality attribute in product or service and ensure the customer satisfaction... this is what TQM says...

  • Anonymous
    May 02, 2008
    The comment has been removed

  • Anonymous
    May 04, 2008
    hi i.m.testy its nice to read ur good comments.... No doubt testing conformance to requirement is a part of the whole testing picture ... other activities of Software testing process are helping to minimize the cause of any NC to req.. + help manage the Testing part of SDLC... imagine if we have processes to manage different parts/phases of SDLC in more effective way dont u think we will be able to fulfill the requirements of users with confidence.. this is quality ... yes.. i object on this statement that "Testing doesn’t ensure quality"" .. some part of reason  lies in above written lines :) secondly, i would like to refer ISO 9001-2000 in this regard... in clause 7 (product realization) ISO demand for VERIFICATION (i.e. specifically 7.3.5 -Design & dev verification and 7.3.6- design & dev validation...) similarly CMMI have one process area at ML3 - Verification and Validation... one can say that... in short verification and Validation (testing) is important for quality and it ensures the quality attribute of any product and service.... lastly... i agree with u that mkt analyst, product designer, decision maker , product maker (developer) play their vital role in achievement of customer satisfaction (if they are following the process approach)...and u cannot eliminate/neglect testers from this chain of quality producers if they are performing their activities as per any well defined testing process.. because.. effective system approach (in any org) without critical testing process is incomplete ... as result Customer satisfaction will b incomplete / uncertain... its similar to say that Quality of product/service is not only responsibility of QA department rather every one is and has to play his or her role in producing quality product.

  • Anonymous
    May 05, 2008
    Quality is perhaps the most misused word in the English language (with the exception of perhaps one other word which has come to be used as a noun, adverb, verb, and interjection). Yes, I agree that everyone in any company the manufacturers some thing does their part and has (or should have) a vested interested in wanting the company to succeed. I am not discounting what ISO and CMMI suggest. In my opinion, functional or engineering 'quality' attributes or capabilities of a thing are not directly synonymous with customer satisfaction, but are indirectly related. Additionally, during the SDLC testers do not usually measure customer satisfaction (nor should they), and customer satisfaction is a subjective measure at best and is only realized after the product is released. I think we essentially agree that testing is necessary and critical to the success of any software project

  • Anonymous
    May 29, 2009
    PingBack from http://paidsurveyshub.info/story.php?title=i-m-testy-the-purpose-of-software-testing

  • Anonymous
    June 19, 2009
    PingBack from http://debtsolutionsnow.info/story.php?id=11728

  • Anonymous
    September 11, 2011
    The majority of the answers so far are saying that private methods are implementation details which don't (or at least shouldn't) matter so long as the public interface is well-tested and working. That's absolutely correct if your only purpose for testing is to guarantee that the public interface works. Personally, my primary use for code tests is to ensure that future code changes don't cause problems and to aid my debugging efforts if they do. I find that testing the private methods just as thoroughly as the public interface (if not more so!) furthers that purpose. Consider: You have public method A which calls private method B. A and B both make use of method C. C is changed (perhaps by you, perhaps by a vendor), causing A to start failing its tests. Wouldn't it be useful to have tests for B also, even though it's private, so that you know whether the problem is in A's use of C, B's use of C, or both? For m ore detail click on edugoing.com/.../index.php

  • Anonymous
    April 17, 2012
    thank you for guide concerning software tasting.