Functional techniques are more than black box techniques

Too often many tester's mistakenly assume that functional techniques such as equivalence class partitioning, boundary value analysis, combinatorial analysis, etc. are simply "black-box" testing techniques. I suspect this rather narrow perspective of functional testing techniques is due to a lack of in-depth understanding of testing throughout the product lifecycle, and confusion between test design and test execution. I also suspect this incorrect assumption is perpetuated by 'testers' whose only approach to testing is interacting with the software via the user interface with the intent to find bugs. Of course when a person can only 'test' via the user interface then everything is a "black-box."

When I teach software testing courses I reinforce the concepts of black, white, and gray boxes as perspectives for test design; not for test execution. Using these concepts as test design approaches designing tests from black box test design makes no assumptions about the code and focuses on inputs and outputs from the end user interface, white box test design explicitly uses the code to help the tester design more effective tests, and gray box principles use an in-depth understanding of the entire system to design effective tests within the context of the software solution.

Functional testing techniques were derived years ago by industry experts and the founding fathers of software testing to provide systematic procedures that help identify specific categories of problems based on fault models that are commonly encountered during the development process. Certainly, functional techniques such as combinatorial analysis are very effective when applied from the end-user interface; however, we all know the fault model which combinatorial testing is effective in identifying involves semi-coupled or directly dependent unordered parameters. So, in order for this functional technique to be applied correctly the tester must have an in-depth understanding of the domain space and know which unordered parameters are semi-coupled or directly dependent. Boundary testing is very effective in evaluating how specific linear variable conditions of independent parameters are handled and is useful for identifying problems with overflowing intrinsic data type ranges, casting between data types, incorrect usage of relational operators, etc. Effective boundary testing means the tester must have some knowledge of data types and programming concepts and insight into the code is helpful in order to design the most effective tests.

Many of the functional techniques we apply at the end-user interface actually require a solid understanding of the underlying system to be most effective. So, designing tests using many functional testing techniques actually approaches test design from a gray-box perspective. These functional testing techniques can also be effectively applied during unit testing to drive quality upstream and prevent problems from getting into the product. Unit tests, of course, that include comprehensive boundary value analysis are tests designed from a white box perspective.

This doesn't mean that functional testing techniques shouldn't be executed from the end-user interface. But, the purpose of functional testing techniques is to help testers design tests from multiple perspectives in order to perform a more in-depth, systematic analysis of software, and potentially expose very specific categories of problems based on empirical fault models. The execution of those tests may be applied either at the code level or from the end-user interface. But, effective application of functional techniques requires in-depth system level knowledge and cognitive skills that extends below the end-user interface.

Testers must embrace and extend the role of testing beyond the simple ability to find bugs at the end-user interface. Testers must start testing earlier as a partner in the process rather than an adversary of the development team. Testers must become proficient in various approaches necessary to provide a wide variety of information to the key decision makers throughout the development lifecycle instead of at the end of it. Quite simply, professional testers need to break beyond the "black-box "barrier!

Comments

• Anonymous
  October 23, 2008
  The comment has been removed
• Anonymous
  October 23, 2008
  The comment has been removed