Exploratory testing and philosophical psycho-babble

Last year James Bach wrote that he “…"invented" testing…mainly by discovering that the problems of testing have already been solved in the fields of cognitive psychology, epistemology, and general systems thinking.” Well, there are simply 2 things wrong with the 2 points James makes in his statement. First, I am pretty sure James didn’t invent testing. Although, he has been very successful at taking the most commonly used testing method, wrapping it with the fancy moniker of ‘exploratory testing,’ and having us believe that we really don’t understand it (until of course we drink the 'kool-aid' and concede exploratory testing is the best thing since sliced bread). Secondly, the problems of software testing are complex and they certainly aren’t already solved. I won’t pretend to know what the solutions are to the problems we encounter in software testing, but I am pretty confident the solutions are not going to be found by studying cognitive psychology, epistemology, or systems thinking.

Now, don’t assume that I simply dislike James Bach. I agree with James on several points regarding the need for better tester education, the value (or lack thereof) of tester certifications, and some of his earlier writings. However, I do admit that James and I disagree on several issues. Professional disagreement is not bad; in fact it sometimes sparks healthy debate, and motivates thought so we can reach our own logical conclusion and reconcile alternative points of view.

One point of contention is the correlation of software testing with branches of philosophical, psychological, and sociological fields of study. The idea that solutions to the problems in software testing practices and processes can be found by studying philosophy, psychology, and systems theory is twaddle. These are interesting topics and can certainly heighten a person’s ability to think abstractly and question assumptions. But, are these subjects really unique to software testing or do they also apply as equally to development, or to any other job requiring a high degree of innovation and ingenuity?

Epistemology

Epistemology is the philosophical study of the origin, the nature, and the scope of knowledge. When I find a defect in software testing, I must admit I really don’t spend a lot of time philosophizing how I know that I know this is a defect? Current theories in epistemology tend to focus on the subjects of truth and belief. Imagine a Venn diagram that illustrates an intersection between belief and truth as knowledge. So, I guess that if I believe an unexpected behavior to be a defect, and the requirements confirm it to be a defect, then I now know it is a defect. But, of course, if the requirements are wrong, then I can only theorize it is a defect, but I may be wrong because then it is only my belief (which may or may not be correct). Frankly, I am not too sure that understanding the difference between priori knowledge versus posteriori knowledge will help me decide whether or not the unexpected behavior is a defect. I am also not too sure how understanding the differences between the foundationalism and coherentism approaches to knowledge will help me design better software tests. Personally if I were to correlate software testing to a non-engineering field of study, I would say that software testing is more akin to archaeology than it is to philosophy. To quote Indiana Jones, “Archaeology is the search for fact... not truth. If it's truth you're looking for, Dr. Tyree's philosophy class is right down the hall.”

Cognitive psychology

Cognitive psychology is the branch of psychology that studies mental processes. Basically, it studies how we learn and process that knowledge into behavior. As an educator and consultant I am very interested in how we learn and how we use the knowledge we acquire to reach logical conclusions, make reasonable decisions, and solve complex problems. Understanding cognition helps me design and develop better curriculum to train new software testers into professional testers and work with teams to resolve problems or implement new ideas. But, do new testers really need to study how people learn and how they process information, or do they simply need the capacity to learn engineering principles and apply that knowledge to design effective tests.

Systems thinking

Systems thinking is related to general systems theory which studies the organization and interdependent relationships of systems. This seems logical in computer science, but general systems theory is usually applied to natural sciences and systems thinking is typically used to model human organizational behavior. Systems thinking views the ‘big picture’ of organizational relationships and the environment on a system. Systems thinking is mostly applied to change management processes because small events in an organization can sometimes cause catastrophic consequences. Effectively communicating decisions and continuously managing the processes effecting the change in an organization to minimize isolation of any part or element of the organization. Consultants must understanding change management and system thinking in order to be successful because their primary role is to introduce change to an organization. But, from a software testing viewpoint I am thinking that systems thinking may not be the best field of study to pursue. Perhaps a better focus would be systems philosophy, which is a form of systems thinking that focuses on design and root cause analysis. Systems philosophy studies the development of systems expressed as models. We often create models to help understand complex systems. In software testing we create models to help us design state transition tests.

I am not an expert in systems thinking, cognitive psychology, or epistemology. And, the simple fact is that I don’t need to be an expert in social sciences to be successful as a professional tester.(Of course taking courses in social sciences in university helps one think abstractly.) And I certainly wouldn’t try to convince anyone who is serious about a career in software testing that they should spend a lot of time studying various branches of philosophy, psychology, or other human sciences to be an effective tester. As I said above, while these topics are certainly fascinating, they are of no greater importance to the practice of software testing then they are to software development or any other discipline that requires a high degree of intellectual horsepower and creative innovation.