Time Wasted in Testing is Contextual

This is a guest post by Huma Hamid *

In a recent blog post by Majd, where he mentioned the time spent on non-testing activities to be time wasted, caused some stir within the blog followers. This lead him to write another post about the same topic, but later post looked at the non-testing activities as “supporting activities”.

hsbc-goodbad-spring07

(picture from http://agrayburdett.wordpress.com . This picture was part of HSBC‘s ads on perspectives)

After going through his posts and the response he received from the knowledgeable readers, I found this whole discussion very much contextual. I am still wondering if there exists an absolute list of activities, where we can draw a line between valuable or wasted testing activities? In different teams and different test cultures, testing activities are perceived very differently. It heavily depends on the organizational culture, test culture and most importantly it depends on the value a test team adds to the overall product development life cycle.

Having first-hand experience of working with a very fast paced test team (in an Agile based development environment), and later becoming part of a test team following a traditional software development model, heavily dependent on a through test documentation, I strongly feel that the concept of time wasted during testing really depends on the environment in which testing is being done.

In the former case, it seems like that the main focus of a test team essentially revolves around making sure that an application is quickly and smartly tested, activities like creating virtual machines, generating test data, writing and maintaining test plans or even reporting and verifying defects (as Majd has also mentioned) seem to take a heavy toll on the perceived value of testing. In an environment like this, fastest and correct delivery of information and shortest feedback loop plays an important role in a team’s success.  Test teams take a great pride in sharing the numbers like “how quickly their smoke test plan was executed and how briskly a build’s health status report was shared with the stakeholders”. In such a team, the term sooner the better holds a real value. Keeping that in view, the context in which Majd has pointed out the “non-testing” activities could be considered as a time wasted in performing the overall testing job, because it apparently has no perceived value in meeting the “need for speed”.

On the other hand, a traditional and documentation based development environment focuses more on the accuracy, completeness and coverage of the test specification to ensure that the requirements are correctly perceived and met, hence in this context the discussion about non-testing activities takes a completely different meaning. Writing better documents, (sometimes perfect documents) can take a test team’s significant time and effort. It seems like that such a test team is more likely to spend time on the activities like writing and maintaining documents, scheduling document reviews and waiting for moderation and approval. Test build arrival can take months; hence, there is no sense of urgency in terms of updating the stakeholders about a build’s health status, until and unless a complete test round is finished (this also depends on how many testing layers a team has in place to ensure a short feedback loop). The completeness and accuracy of testing is heavily dependent on perfect test documents. In such a scenario, it is impossible not to spend time on writing good documents. Whenever, documentation is by-passed due to short of time, the test team cannot proceed further without a heavy churn and eventually suffers from a sense of guilt for not writing perfect test specifications which could ensure a full and accurate test coverage. So, for such a team documentation might come first and testing comes later.

Having said that, this reminds of an experiment conducted at my previous company, where a university research student was conducting an academic experiment on the efficiency of exploratory vs. traditional testing approaches.  Our test teams were asked to become part of his experiments, where the whole test group was divided into two teams. First test team was asked to run tests after writing test specifications using traditional testing approach, whereas the second test team was expected to perform exploratory testing by writing quick and brief test sessions. I was randomly put into the team, which was expected to perform the experiment using traditional testing approach i.e. writing detailed test cases before performing actual testing of the application. Since, I have always counted myself a true Agile tester spirit, hence becoming part of a traditional test team was disappointing for me, because I felt that my time will be wasted in writing tests and I won’t be able to find more bugs quickly like the other team who didn’t have to write detailed test documentation. As I expected, I spent most of my time struggling with writing test specifications, which in my eyes was an effort of lesser value and least significance. But now, as I work in a team where definition of valuable testing activities starts with good test documents, I feel less bad about writing test specifications. Hence, in my view this discussion is contextual. It really depends on the environment you are working in and the perceived value of the activities a tester is performing. As a reader, you may disagree to that!

Just a reminder to myself, I still am a true Agile tester spirit. When I cannot find a way without writing documents, I try finding ways doing that quickly, smartly and efficiently. For me, value of speed and accuracy still matters! – 🙂

(* Huma is one of the few people who forced me to start out a blog on software testing. I had the pleasure of working with her in a team where we had lengthy discussion on the role of testing, being Agile, how to force the whole team to think about quality etc. etc.. Thanks to Huma for sharing her thoughts on this topic)

16 responses to “Time Wasted in Testing is Contextual”

  1. Smita says :

    Good thoughts Huma.

    Like

  2. Ather Imran says :

    Very insightful Huma and many thanks to Majd for having you share your thoughts. You articulated it very well.

    I, in general, agree with what you say. Many complex situations and discussions are contextual. An absolute or arbitrary judgement hurts i.e. one taken without having a context in mind. Hence, you are spot on that we need to contextualize and evaluate rather than make a black and white judgement.

    What I will add is this: whenever there is a contextual situation and judgement, there are still some higher or fundamental principles that need to be identified. It is important to raise our levels of abstraction and evaluation a degree higher and see if there are some non-contextual principles that hold. You and Majd are the expert testers, so I would not venture into giving you those principles. But I am just laying out a general framework of thinking.

    Second thought I want to add is that whatever context you are in today may change or even become irrelevant tomorrow. So there is always a need to “inspect and adapt” checkpoints in any situation – to see that what you defined as your contexts are still relevant or not. Sometimes they have shifted but we are stuck with the old way of doing things, the only justifying argument being that “this is how it has always been done here”.

    Ather Imran
    http://thinkingspirits.com

    Like

    • majd says :

      Thanks Ather for your valuable thoughts. That comment reminds me one of our University professor who’d start answering every question with “it depends”. So I agree that context is important, but some principles are also very important. I hope some of this discussion will force the testers to re-think towards their approach on “where the testing time goes”. Thanks again.

      Like

    • Huma Hamid says :

      Thank you for sharing your thoughts, Ather. I am in 100% agreement to what you have said. There is no denying the fact that no matter what the context is, there are always better ways of doing certain things, based on the fundamental principles. While writing this, I realized that discussing those points would lead to a different discussion. In this post, I tried my best not to evaluate and prove one way of doing things better than the other on the basis of my judgement, rather I wanted to highlight only the perceived value of time wasted in different test teams and cultures. For me it was very interesting to learn that with time and situations, our reactions towards certain ways of doing things may also change, based on what brings value to our teams. So, you are absolutely right and I think this calls for another blog post. 🙂

      Like

  3. Muhammad Jamil Akhtar says :

    Thoughtful and very well written Huma. I am predicting a new blogger 🙂

    Thanks Ather for your points. These are very valid and powerful points.

    Like

  4. Sohail Sarwar says :

    Good one Huma and Thanks Majd for making it possible, the sharing of philosophical thoughts from old fellows of ours.
    Picture above has placed the discussion in a “nutshell”; fully explaining the “context”
    Moving from “Agile Testing” environment to document driven “Traditional Testing” can be very tiring.. 🙂 .. esp if your first job involved AGILE.
    The context may be defined by organizational culture/environment/norms… It definitely needs to be updated to grow->adapt and hence survive at individual level as well as at level of organization. (Unfortunately, very few follow the same cycle of adaptation)… Still a long way to realize importance of context.. 🙂 I can relate this mindset to Majd’s story where people are not ready to give up usual path (https://knowledgetester.org/2013/08/15/take-off-your-khopay)..:)

    Like

    • majd says :

      Thanks Sohail and keep visiting. May be some day, you have a slot to share your thoughts. End of the day, it is the collective wisdom that matters, not the individual one.

      Like

    • Huma Hamid says :

      Thanks very much Sohail. I agree that at the end of the we all need to learn and adapt to different working cultures and environments, by keeping contexts in view.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s