Smoke Testing is boring, Help!!!
Ahmed is running the smoke test for 35th time in the last six months and feels like quitting testing. Smoke testing finishes in an hour or so, results posted and Ahmed returns to other testing tasks. Ahmed thinks that testing is not as boring as it was earlier in the day and keeps enjoying the job.
What is that makes Smoke testing so boring?
Boredom is defined in many ways and usually refers to a state when you have a situation which doesn’t satisfy your mental needs of that time. There are two parts of it: a) Situation and b) the thoughts in the mind.
If you are bored like my 5 year daughter who after playing her favorite game on iPAD for about an hour, doing cycling for a while, eating the food of her choice, comes to her mother and says “I’m feeling bored, can you play with me?” In this case, this is clearly the state of the mind that needs to be controlled. The 5 year old kid can’t do it but we as mature professionals should be able to as some parts of our jobs will always be boring.
But wait, is the only advice I have for you is “change yourself as you can’t change things around you”? I know you hear this a lot in your life and feel sick of such advisers as they seem to not understand the situation you are in. Ahh, situation, yes that situation is also part of the equation. Let’s see if we can do something about it?
The smoke testing process is like this: download the build, install / setup, run the same tests you ran a few days ago, record results in an ancient software called Excel, share results via email.
Automate the setup parts
By the time as tester, Ahmed reaches to the interesting part of running tests and observing the results, he is already bored by setting things up. So the plan is to bring a fresh testing mind at the start of running tests while everything before it is done by the miracles of automation.
While there remains many question on test automation that can add value, no one questions the powers of automation when it comes to doing stuff that doesn’t involve intelligence. And set up is exactly that.
In our team, for example, we have a Python script that keeps sniffing the FTP site where builds are posted. As soon as a build arrives, it downloads it, unzips it and places at our test machine along with needed dataset. In our case, we don’t need specific testing environment, but I have seen other teams do the additional part to kick off a Virtual Machine and install the build there. Or you can have a QA version of your web application where it goes. The choice is yours, but this effort on automation will certainly pay you off. Give this a try.
Test the delta, not the entire stuff
Given the agilish nature of projects these days, the builds come too often. The more you have to do a thing in same way, the more boring it becomes.
The trick here is to be aware of the changeset that you have in the latest build compared to last build to define the scope of your smoke test. Rather than running all tests with same focus, run the ones that might be affected by recent changes first with full attention. Then rest can be let go.
If you don’t know how to get those changes, get in touch with Programmers in your team. In our case, the build is generated from a server and we could extract the “build description” for any two builds and then compare them.
By adapting this technique, the attention of tester will improve in significant way along with the overall understanding of what is changing and what parts of the system are moved by those changes.
Report results in charming way
I know that testers do lot of effort in designing and executing the tests, but when it comes to showing off their findings to the world, they end up with some dull sheets or large list of numbers. For example, this is how we used to present our Smoke testing activity at the end of sprint:
“Smoke Test executed 6 times for each build.”
Now that sounds like boring to the reader/listener also and Project Managers say to themselves “Thanks God, we have testers who do these dirty jobs for us”.
We converted this boring way into a live page on our SharePoint internal site, where results are updated each time a smoke test is executed. This is how it looks and is being liked by testers and non-testers alike.
Another idea that I heard from another tester was to use the smiley face for a Passing result and a sad one for Failing. You can try that or try any other powerful way to show results to make your reports management friendly.
Have you felt that smoke testing is boring and did you try something else than the above?