3 illusions of Software Testing
Jerry Weinberg is my favorite author on the subject of Software Testing and I have posted about lessons from his Errors book, shared my thoughts on some types of Bugging. Lately I was very happy to finish yet another book by him called “Perfect Software and other illusions about Software Testing” which was on my reading list for long due to excellent reviews.
Now having spent almost 15 years on Software Testing domain and overall 20 years in the pursuit of Software Quality in general, my first impression was that the illusions Jerry will talk about will be one of those that I’m familiar with and/or have dealt with. But Jerry surprised me with his wealth of knowledge and interesting examples from his illustrious journey as a Software professional. I learned lot of lessons and sharing the top 3 with you.
- Pinpointing problems is not Testing
Some activities that I always thought to be Testing’s responsibility are actually in a grey area where these have to be shared between Testers and Programmers. Pinpointing a problem, after it is found, is one of them.
In one of the early team that I worked with required that when Testing reports a bug in the Tracking System, they should mention the first major version it was introduced. For example, if a Tester finds a bug in v 3.4, Tester would check if it existed in v 2.0 and v1.0 (if the answer to 2.0 is no). That is an activity that Jerry calls Pinpointing and suggests to differentiate it from Testing and Debugging.
Even in my current project, because we (the Testers) have the code and are able to debug an issue, our Data conversion issues are reported after pinpointing and debugging. Hence a Tester’s considerable time goes into locating the problem code after finding the problem. This is included in Testing whereas it needs some separate attention.
In Jerry’ words as he summarize this as a Common Mistake and suggests a solution:
Demanding that Testers locate every fault: This is totally a Developer’s job, because developers have the needed skills. Testers generally don’t have these skills, though at times, they may have useful hints.
- Providing Information is hard
Jerry talks in much detail Tester’s role as information provider. He suggests to use Satir interaction Model:
-> intake -> meaning -> significance -> response ->
There are separate chapters dedicated on each of the above and I’d encourage any Tester who is serious about their job to understand this model through reading the book. Different techniques have been discussed to improve information intake, how different people will mean different things from same observation, how to know which information is significant and finally how to make the best response accordingly.
- Product reviews are actually testing
In fact Jerry calls it “Testing without Machinery” and suggests that technical product reviews are actually a way to provide information and thus is Testing. He also lists some “instant Reviews” which is that if you try to review and you hear excuses like below, you know that something is wrong:
- It’s too early to review it.
- It’s too small to bother with. What could possibly go wrong?
- We’ll let you know as soon as we are ready.
- We [tested][tried][demonstrated] it and it works okay.
It’s a treasure of information on those few pages which you can get instantly once you ask for a review. This has inspired me to write more about Peer Code Reviews so you should expect an article soon.
Let me end this with Jerry’s words:
If humans were prefect thinkers, we wouldn’t need to test our work. … But we are imperfect, irrational, value-driven, diverse human beings. Therefore, we test, and we test our testing.