We're Gonna Test Like It's 1999 (yeah try not to sing it!)

Modern software developers bear little resemblance to our forebears. We’ve forsaken their jackets and ties in favor of hoodies and t-shirts. We’ve quit their offices and cubicles to occupy hacker hostels and corner cafés. They had floppies and sneakernet. We have Github. They printed and stored; we share and post. They worked for big companies with distribution channels. The world is our distribution channel. Where, with all these changes, do we stand with software testing?

Let’s face it, the 1990s were the golden age of software testing. As an industry we were still figuring things out. Global or local data? File and variable naming conventions. Time constraints versus memory utilization. Library, procedure or inline code? Use or reuse? And the granddaddy of them all: how do we fix bugs that occur in the field when the only way to get bug reports is by phone or email and the only way to update our software is by mailing a new set of floppies? We were at once not very experienced in writing code and after shipping that code, fixing it was a really, really painful proposition.

No wonder we put so much time and effort into testing. We had no choice but to double check developers’ work and try to ensure that as few bugs as possible made it into the released product. Like I said, it was a golden age for software testers. Small chance of getting it right, large chance of expensive rework. Testers were the insurance policy no company could afford to decline.

But then the world changed. First it was the web which made software updates a small matter of refreshing a web page. All those floppies were F5-ed into oblivion. And then along came mobile apps which could collect their own user telemetry, create their own failure reports and prompt a user to update them when necessary. At the same time the risk of shipped defects was decreasing dramatically. So-called waterfall software development models were replaced with agile methods that created better code out-of-the-box. A collective intelligence around how to code and the body of knowledge of coding practices matured. The art of coding has become downright pedestrian.

Quality is no less important, of course, but achieving it requires a different focus than in the past. Hiring a bunch of testers will ensure that you need those testers. Testers are a crutch. A self-fulfilling prophesy. The more you hire the more you will need. Testing is much less an actual role now as it is an activity that has blended into the other activities developers perform every day. You can continue to test like its 1999, but why would you?

You can’t test in quality, but you can code it in.

And at the tail end of the lifecycle, testing can now involve users at a level that it never could in the past. Who, after all, is the better judge of a bug: the user who is honestly trying to use the software to get work (or pleasure) done or a tester who has a preconceived (and unavoidably biased) notion of how the software is supposed to work? Why must a tester serve as the intermediary between the developer and the user when the user is only a click away? Can you imagine the impact on quality when developers and users have no middleman getting in their way?

Quality, and therefore testing, is not something separate from software development unless your software is going into a nuclear power plant, medical device or an airplane where it is difficult (for now) to recall post-deployment. For the vast majority of app development on this planet, software testing is an activity within the development process and keeps happening after the software is released. Modern testing is an activity and doesn’t require a separate role to perform it. It’s time to bring quality into the 21^st century where testing is such an integral part of software development that you’ll often forget you are doing because it has become so familiar. Hey wouldn’t that be awesome … that testing gets done without making such a big fuss about it?

This is not your father’s application development process. It’s yours. Own it.

Comments

• Anonymous
  April 11, 2014
  Hey James, glad to hear you talking about testing and quality again! I will point out that even airplanes get Tested in Production, it’s called a test flight. :-) What you are saying is very well aligned with the message Ken Johnston is delivering on EaaSy (Everything as a Service), most recently at ALM forum (www.alm-forum.com/.../pr_test).  Myself, I’ve been preaching similar gospel about Data-Driven Quality (www.setheliot.com/.../your-path-to-data-driven-qualityalm-forum-2014) which focuses on production or near-production

• Anonymous
  April 11, 2014
  Not to monopolize the comments, but perhaps even better thematically aligned with your message, I cautioned against returning to the 90’s with our testing practices www.softwaretestpro.com/.../5683

• Anonymous
  April 14, 2014
  The comment has been removed

• Anonymous
  April 15, 2014
  Based on the assumption that development is way more simple than it is and a smooth process? This is fancy from an purely idealistic world. The risk is reduced 100 fold when releasing bugs but watch bugs in any software break trust and respect for the users. Saying that testing can become just an activity for programmers is like saying dentists should pull their own teeth. There are a myriad of reasons why that notion is unrealistic and impractical.

• Anonymous
  April 16, 2014
  I agree with Jim. Tasking developer to do the testing themselves will just make there's no testing at the end because of time pressure. And the managements don't want to invest in testing too. I've had a project that taken about a year to develop. When I try to add a week for writing tests to it, they said "no" because no budget has been assigned there. So it's unrealistic burden for most of us here.

• Anonymous
  April 17, 2014
  The comment has been removed

• Anonymous
  May 13, 2014
  Hello James, I thought you left the testing field, but you still have thoughts around testing. Thanks for your post. One thing I like to call out here is about software distribution mechanism. For web application and web service application, the software company have total control over what customer gets and experience. Even for the Chrome browser, google updates the software for the user whenever they can. By having this control, the software company can utilize users as testers. Of course, the software company should have super detailed monitoring system, tracing every single activity of each user, rollback mechanism, different versioning(web service case) and etc. DevOps are developed in this areas because of this distribution control. I don't think mobile application will give the software company as much control as web application does. There will be opt-out options and providing update all the time will annoy users. If the mobile becomes the next big thing, then every update should be carefully tested since it will be hard to rollback. It will not be like delivering CD, but similar hassle will follow for update. Finally, I think it's totally up to companies to hire testers or not. Maybe testers will go away just like waterfall model disappears as agile model produces more and more success stories. Software industry will decide eventually whether testers are needed or not. I don't think a few individuals or companies can change it. What you say might work in Google or Microsoft, but I think the software industry is far bigger than Google and Microsoft.  

• Anonymous
  May 23, 2014
  The comment has been removed

• Anonymous
  July 19, 2014
  The comment has been removed

• Anonymous
  March 16, 2015
  Thank you for the article. I happened to work for a medical device company. Should we stick for waterfall? Is there a better way to recall our product and fixed the results for our customer? Is there a better way to utilize Agile method so that we can have a more balanced approach? Thanks again for your insight. I enjoyed most of your books and youtube videos.