The magical “Phakkis” and Software Testing

If you have ever travelled in a bus in rural parts of Pakistan, you’ll recognize this very well. Yes, those buses that are labelled as “Non-stop” because they stop a lot. And on each stop, many vendors move in the bus who sell all kind of food, household items and even medicines. Some of them leave the bus when it embarks for the next stop whereas the ones who sell really deep solutions like medicines, they usually prefer to stay in the bus and do their marketing until the next stop comes. The most common of them sell “Phakki”, “Munjon” etc. Though I can take any of the examples but I’d prefer the “Phakki” because it has a personal connection.

These vendors are highly skilled marketing people. First, they won’t start directly from what they sell rather they’d start the talk by praying for everyone’s successful journey and may start with a generic statement that encompasses all the passengers like “all my brothers and sisters; the elderly ones and the kids etc.” Then they explain at length the typical and common symptoms of the illnesses that people would have. To be specific, in case of a “Phakki” seller, he’d ask if you ever feel heaviness after eating hard, or do you have gas trouble or do you find it hard to ease you during various trips to the washrooms. They would pick problems that are common and remember that “the problems that are hard to go away”. After this five to ten minutes talk that would grab the attention of all the passengers, they’d then tell you that they a magical solution to all the problems mentioned before which is a “Phakki”. (For those who don’t know what it is, it is a well grinded mixture of many herbs and it is usually taken by putting it onto the palm of your hand and then tossed into the mouth and then water is gulped to take it inside. It is also called a “Chooran”)


(The photo is taken from here. No, I’m not endorsing their product, I just have a policy to add link to the original photo)

As a customer, you are set to believe that the magical “Phakki” can solve all the digestion problems of all the people in all the situations. That’s what they do it deliberately by making the message generic and that’s where if we ever buy a “Phakki” should be aware of. As I mentioned the personal connection and here it is: I do use “Phakki” (though I don’t buy from those bus vendors but prefer to get it from source), but I know that it cannot solve all my problems. So I use it to solve a “typical problem in a typical situation” and it perfectly works there. For other digestion problems, I take other medicines and also take a good control of what I eat when I have that problem.

Sorry for the long description of a thing that you may not like but I hope you know where I am taking this.

The many tools vendors especially those who sell automation tools would do the same marketing trick to you. After all humans are humans. Whether they are trying to cure digestion systems or information systems, they take easy paths. If you take a look at marketing techniques of those tools, they’ll be strikingly similar to the “Phakki” sellers i.e. they’d take the problems that are very common, the problems that are hard to go away like making a poor quality product a high quality one. They also often don’t provide the situations where not to use their tools and the associated hazards. They just tell you that you use our tool and “all the problems of all the teams in all the organizations in all situations will go away”.

Rather than taking the extreme path to simply stay away from such tools, I suggest you the same approach I mentioned above. Use that tool to solve “typical problem in typical situations in typical teams”. For other problems, use other tools or techniques. Keep in mind that “using a tool is not a strategy rather the strategy can have one or more tools”.

Same caution has to be applied when “processes” are sold to you that if you follow a certain standard like ISO or CMMI, define a super process that will solve all your problems. In this case many of the authors of those standards or processes do know that their solution will not solve “all the problems of all the people in all the situations” but their “product” is soon picked by marketers who are the trainers and implementers of those processes. They start making claims which are not always true. When you go for adapting a certain process in your culture, you should know that it will solve “some of the problems in some of the situations” and is not a magical “Phakki”.

As Jerry Weinberg writes in his book “Errors: Bugs, Boos-Boos, Blunders”:

There’s never going to be one best way to test and fix software, no matter what some marketing organization tries to sell you. Ultimately, you have to develop your own process, to create the unique service you can offer to your employer or sell to your clients.

Lets use the “Phakkis” aka “tools” and “processes” in the above fashion to live a healthy life and deliver healthy solutions.

Testers, mind your business

Testers have many jobs at hand. They need to run as many as possible tests to find out useful information about the health of solution to help make decision makers take decisions. Anyone who has experience in testing knows that the real job is not to “run” tests but “design” tests. Similarly the quantity of tests that were executed doesn’t matter if they are not hitting the right target. That is why number of failing vs. passing test cases, number of new defects reported are a poor way to represent quality and hence a poor feed of information to the decision makers.

What matters most is perhaps what the test does and why it is important? In one of my previous posts, I suggested that testers become gray box testers to get some ideas on coming up with some serious tests and hence bugs. Today I want to bring the focus on the business side of the things.

Testing in a way serves a bridge between Business and Technology. You can say that this is true for the entire “engineering” team but perhaps testers are the main guys responsible for this.


(Picture from with edits by me)

Given that the domain you serve will vary from organization to organization and from team to team, it is tough to lay down specific guidelines so as how you can understand the domain. A software serving to help design Nuclear Plants may require it’s testers to learn about all the standards of materials, quality and health/safety. Whereas a software serving the education sector may require it’s testers to understand how brains are educated and what quality education means and what are the experiments done in this area so far.

Therefore I’ll not mention the ‘how’ part of the problem rather focus on ‘why’ it is important for testers to learn more about the domain as the “Golden Circle” suggests that ‘why’ is more important than ‘what’ and ‘how’.

  1. You talk the language of your users

Very early in my career, I was visiting a client with a team to understand the requirements of a solution that we were building for their pharmaceutical setup. Client was represented by 3-4 managers and they were throwing a lot of terms. I don’t remember the exact term of one of the reports that they wanted but one of our team members said the name of the report wrong. Manager politely corrected. When our team member made the same mistake in the conversation, the client said:

“How come you make a solution for us, when you don’t even name the things we need?”

I learned a lesson that day and always prefer to use the terminology that our users would use. That makes all the internal communication that you do as tester become more “user focused” and you know “what user wants will always be fixed”.

  1. You augment the team with “Business facing tests”

Keeping the Agile Quadrants in your mind, you know that your team specially the Programmers are writing most of the technology facing tests, so if you dedicate your efforts towards the tests that mimic the business cases that’s like a perfect combination.

  1. You know what drives what

Pieces when combined become the solution. So as you move your “vision of the tree” to the “vision of the forest”, you start gaining more understanding of why a particular piece is needed to solve a particular problem. This helps you write tests that are like end-to-end or workflow tests (the tests that achieve a particular user workflow). Often such tests results in bugs that surely increase your reputation as a tester who knows how the system works.

How much of your time as tester goes into understanding the domain? And do you list more benefits of this investment?

Meetup “Testing is a fantastic career”

Learning happens in many ways and perhaps the most powerful way is to share your thoughts with like-minded people. That’s exactly what last weekend witnessed when a meetup took place in Islamabad where we broke participants in teams and gave them topics to discuss. The findings were then presented to all the audience and the awesome testers added their opinions. The result was just as fantastic as the title of the meetup was. One sign of it being that when it was announced from stage: “the time for discussion is over, so please conclude and get ready to present”, no body listened to the announcement and they kept talking in their tables.

It is difficult for me to say but the idea of such kind of setup was not mine:) It was Taimur Sarwar’s brain child who coined this at our last meetup. As our previous efforts were to gather more and more testers in different cities, we thought it’s about time to bring them closer. S&P Global Pakistan was generous enough to offer their space to conduct the event. The settings were perfect, the food was delicious and the souvenirs given to each participant was cherry on the top.

To have this session in an effective way, there were a few VP (Planning & Coordination) positions which was happily accepted by Amir Shahzad, Sadaf Minhas, Dr. Zohiab Iqbal, Dr. Uzair Khan, Taimur and myself. At least I can tell people that I was once a VP in my life, though Dr. Zohaib being President of Pakistan Software Testing Board didn’t like the demoted title. But who cares.

This team met few weeks before the event and did what their title suggested. The plan was like this:

Bring in about 30 testers mostly in the range of 0-3 years of experience, break them in few teams, provide them topics, provide them a Facilitator who is someone with experience, they discuss the topic and present to the larger group. That’s it! and the result was amazing.

So here came the Facilitators with whom we had another planning meeting which in itself was kind of a mini-meetup where ~10 top brains of testing from the town pondered together how to best conduct the sessions. Along with VPs, these were Ahmed Mugheera, Haider Farooq, Kashif Butt, Hanan Atif, Hina Zafar and Farhana Adeel.

Enough for the preparations details, let’s move to the event day. As testers arrived, they were allotted tables and thus Facilitators with topics. Before the brain storming could begin, a keynote was delivered by Afif Zaidi, who is Director IT at S&P Global and started his career as a Software Tester. Afif recapped his illustrious career in few minutes and then shared his vision of testing. His description of Testing as a bridge between Business, Technology and Project Management was impressive and the tone was set for the participants to get knowledgeable about all of them.


(More pictures are on our facebook page)

There were two topics given to each table with one looking towards the testing career and other meant to increase the technical knowledge of the table. The full list of topics is below:

Career and Skills topics:

  1. What is Quality in general life? How do you differentiate between Quality Assurance and Quality Control?
  2. Importance of Communication & Collaboration and when to seek help vs. do yourself.
  3. Tester – Programmer relationship
  4. I’m bad at coding, can I still be a Tester? I’m not a developer, do I still have a career path?
  5. Role of tester in Agile.

Testing Techniques and Concepts topics:

  1. Exploratory Testing, Session based testing
  2. Bug life cycle
  3. Effective bug reporting
  4. How automation fits in? What to automate and what to not? What is possible vs. practical?
  5. How can I improve domain knowledge?

As you can imagine that each of the above topics is a whole day thing, but teams did an effective job to discuss them in ~45 minutes and then present in a few minutes at stage which followed by comments and questions from the audience. There were so much new ideas that flowed in and so many new viewpoints thrown in that I can think I can write blog posts for the whole next year if I keep covering them:)

Just to give you an idea of the proceedings, I’d like to share two takeaways. While discussing domain knowledge, the hall kind of agreed that Testers should do that in depth. In short, if you are serving a financial industry you need to learn different type of things than you’d learn if you are serving the health industry. Many testers don’t realize that their job is not to test the code, rather their job is to test the solution.

Another one was related to effective bug reporting where some believed that testers should write a super duper bug report that everyone could understand and is flawless. But reality is far away from this and a bug report initiates conversation within the team about an issue which is a potential threat to the release. Many testers don’t realize that bug report is not their accomplishment report, rather it is the starting point of evaluating the risks.

Other than the above, there was a table of senior testers who discussed and then presented their thoughts on Testing Strategy and Planning with some insights into estimation and prioritization of testing activities.

The closing keynote was delivered by Badar Khan who is Managing Director of NMX Global Software Islamabad center and he also started his career as a tester. Badar shared his achievements in a humble way and recapped his almost two decade’s journey as a tester. His advice was simple: you should always be truthful. Yes, as a tester don’t hide what you know and share it. He also shared a nice recent example where a higher up Manager could be wrong and send you in wrong direction; so what you should do in that case.

As you can imagine from this summary that it was hell of knowledge to digest in a day and we were lucky to have some brilliant testers present there who exactly did that. And more importantly they built new connections which will help them in their journey into the testing career.

I know we couldn’t invite all of testers in our network to the event due to the limited capacity. But don’t worry, we have plans to do more in coming quarters along with a national level testing conference towards end of 2016. Insha Allah (God willing).

How do you see such type of meetup compared to the one we had earlier? How we should do our next sessions?

Why you should be Gray Box Tester

One of the standard distinction between testing types is that they are either Black box or White box. When you ask “What do you know about testing?”, this is one of the first sentence you hear from newbies into the testing world. They are not wrong because they tend to believe that world is simple and is binary. Alas, the real world is lot more complex and is fuzzy.

That’s why we need more “Gray box” testers or as the punch line of my blog says “Software Tester equipped with Knowledge”. But wait what is the distinction.

I really don’t like the term Black vs. White box testing at the first place as an Opaque vs. Transparent box testing would explain it better. But we can’t do much about what the industry talks. (Think about how linguistics complain about the original word but the one that public speaks matters the most).

So here is a borrowed comparison from the web which explains that Black box is when you don’t know the internal details, White is when you know all of them and Gray is when you know some of them. Again this is not the complete picture, but should work for our purpose.


(This is taken from here: )

To give you an example, think about the ignition system of your vehicle. For a Driver (read Black box tester), when the key is turned, or the button is pressed, the engine should start. That’s it. A white box tester knows that depending upon what type of engine is being used, the ignition system gets current from the battery, pulls in the fuel and gathers the outside air into the engine and then combustion happens by the spark provided by the current. This in turns moves the pistons which move the crankshaft which moves the transmission line going to the gear system. That’s too much detail that might be useful for a mechanic who fixes the car but can blow off a driver’s mind, that’s why we need Gray box testers.

Taking this example further, the Gray box tester should at least know the building blocks of the ignition system, so that in case of a problem it can be figured out if the battery is weak or fuel is not being pumped etc. That’s what drivers with some mechanic like skills. These are the Gray box testers.

If you have survived this example, let me take back to the software world.

Given that the job of a Software developer is to build a technical solution to solve a business problem, then a Tester’s job is to make sure the solution works. That’s where Tester should know the business well and they should know the technology well.

There will be many benefits when you’ll learn the internal details as a Gray box tester but I can guarantee these three gains for sure:

  1. You talk the language of the town

Have you ever been to a place where you don’t know the language of the town? You find it very hard to get directions or order meals or in short explain what you want. The same thing happens if you cannot talk technical to the team who mainly talks in terms of technology. For example reporting a bug that says “Export fails while processing XYZ type of stuff as the libraries to support this is missing” is way better than saying that “Export fails while processing ABC file”.

  1. You know what changes what

In these fast paced development cycles, builds to test come too fast. And we don’t have time to test it all again and again. No tester can survive doing the whole regression testing all the time and the only solution is to test the delta. You can only determine what areas to test for a given change, if you know the internal details of how things are linked together.

  1. You can write specific intriguing tests

As they say that “The Devil is in the details”. Anybody can write tests that checks the surface area of the product but the real tests that add value to the team and the consumer (read technology and business) are the ones who dig deep into the solution. These well-crafted tests can only be written and executed by a Tester who knows that area well and kind of “lives there”.

If you like the idea, the first thing might be drawing a block diagram / architectural diagram of the project / product you are working on and show it to your Programmer friends.

Are you ready to start this journey?

Smoke Testing is boring, Help!!!

Ahmed is running the smoke test for 35th time in the last six months and feels like quitting testing. Smoke testing finishes in an hour or so, results posted and Ahmed returns to other testing tasks. Ahmed thinks that testing is not as boring as it was earlier in the day and keeps enjoying the job.

What is that makes Smoke testing so boring?

Boredom is defined in many ways and usually refers to a state when you have a situation which doesn’t satisfy your mental needs of that time. There are two parts of it: a) Situation and b) the thoughts in the mind.

If you are bored like my 5 year daughter who after playing her favorite game on iPAD for about an hour, doing cycling for a while, eating the food of her choice, comes to her mother and says “I’m feeling bored, can you play with me?” In this case, this is clearly the state of the mind that needs to be controlled. The 5 year old kid can’t do it but we as mature professionals should be able to as some parts of our jobs will always be boring.

But wait, is the only advice I have for you is “change yourself as you can’t change things around you”? I know you hear this a lot in your life and feel sick of such advisers as they seem to not understand the situation you are in. Ahh, situation, yes that situation is also part of the equation. Let’s see if we can do something about it?

The smoke testing process is like this: download the build, install / setup, run the same tests you ran a few days ago, record results in an ancient software called Excel, share results via email.

Automate the setup parts

By the time as tester, Ahmed reaches to the interesting part of running tests and observing the results, he is already bored by setting things up. So the plan is to bring a fresh testing mind at the start of running tests while everything before it is done by the miracles of automation.

While there remains many question on test automation that can add value, no one questions the powers of automation when it comes to doing stuff that doesn’t involve intelligence. And set up is exactly that.

In our team, for example, we have a Python script that keeps sniffing the FTP site where builds are posted. As soon as a build arrives, it downloads it, unzips it and places at our test machine along with needed dataset. In our case, we don’t need specific testing environment, but I have seen other teams do the additional part to kick off a Virtual Machine and install the build there. Or you can have a QA version of your web application where it goes. The choice is yours, but this effort on automation will certainly pay you off. Give this a try.

Test the delta, not the entire stuff

Given the agilish nature of projects these days, the builds come too often. The more you have to do a thing in same way, the more boring it becomes.

The trick here is to be aware of the changeset that you have in the latest build compared to last build to define the scope of your smoke test. Rather than running all tests with same focus, run the ones that might be affected by recent changes first with full attention. Then rest can be let go.

If you don’t know how to get those changes, get in touch with Programmers in your team. In our case, the build is generated from a server and we could extract the “build description” for any two builds and then compare them.

By adapting this technique, the attention of tester will improve in significant way along with the overall understanding of what is changing and what parts of the system are moved by those changes.

Report results in charming way

I know that testers do lot of effort in designing and executing the tests, but when it comes to showing off their findings to the world, they end up with some dull sheets or large list of numbers. For example, this is how we used to present our Smoke testing activity at the end of sprint:

“Smoke Test executed 6 times for each build.”

Now that sounds like boring to the reader/listener also and Project Managers say to themselves “Thanks God, we have testers who do these dirty jobs for us”.

We converted this boring way into a live page on our SharePoint internal site, where results are updated each time a smoke test is executed. This is how it looks and is being liked by testers and non-testers alike.


Another idea that I heard from another tester was to use the smiley face for a Passing result and a sad one for Failing. You can try that or try any other powerful way to show results to make your reports management friendly.

Have you felt that smoke testing is boring and did you try something else than the above?

Do you love your testing job?

(This is a guest post from Sohail Sarwar, one of the cannons in the Knowledge Tester’s arsenal. He is a regular contributor, so doesn’t need much of an introduction. His previous posts are here and here.)

Situation 1: A young QA engineer joins bit senior QA engineers on lunch table.

Senior: So where you have graduated from?

Junior: Xyz University.

Senior: That is one of the finest universities; what happened to you that you have chosen domain of QA? Something wrong with your development skills?

Junior: No, I have been good at programming. In fact, I won coding competitions.

Senior: Hmmmmmmm.

As this discussion was concluding, I could see number of questions on face of “baby-tester” entailing in a discomfort regarding his choice of professional career.


(image credit: )

Situation 2: An experienced tester requests a meeting with his Team lead and enters the room.

Team lead:  Yes, what is agenda or reason to call this meeting?

Tester: I want to switch the domain of work to development or I quit?

Team lead: But you are doing a good job and there are opportunities for you to grow.

Tester: I don’t have growth like a development guy. Plus, I foresee myself de-valued with added number of experience years. Also, there are more Dev jobs than testing jobs.

Manager, with perplexed face, gave strong arguments to advocate domain of testing but “Tester’s firmness” in his disbelief remained evident from his face.

In software organizations, these are the situations that leads/managers have to deal quite often. What are the factors spawning such situations?

  • Lack of training for opting proper career path in academia with respect to interests/ability. Moreover, wrong myths prevail in academia that software industry is all about software development.
  • Lack of guidance from senior resources in organizational hierarchy about importance of testing and subsequent growth in coming years for new testers.
  • Lack of promoting the culture “You are doing a great job” instead, it’s “You are doing an inferior job”.
  • Lack of appreciation and giving less visibility to contribution of testing resources in success of projects at organization level.
  • Lack of understanding that a resource with inferior capabilities may be suitable for role of testing.
  • Most common and dumbest myth of all times, “Anyone can Test”.

This list of “Lack of’s” can go on and on but bottom line is, no matter how skilled a testers is, one is found deprived of its’ testing role, insecure and dissatisfied of what one is doing. The Tester might not say but there is a pinching question in it’s sub-conscious, “Am I really doing a job that is worthwhile”.

Solutions to these problems may not come in effect overnight and may not comprehend in these lines; but they surely are concealed in changing “mindsets” to value the sheer need and effectiveness of testing (and skilled testers).

Some possible solutions:

  • A person going for a job of tester should know that one wants a tester’s job and also why opted this career path. Moreover, students be enlightened of career paths other than software development in academia.
  • Software community should know that testing is no more a role of “value addition”. It is the role that refers to “value creation”. This value creation should be highlighted by senior resources in industry as well as academia.
  • Number of testing jobs may not be used to measure superiority of massive Dev jobs over any other discipline of software life cycle.
  • Software tester’s job can only be done by a software tester. There is no replacement for a learned, skilled and dynamic software tester.
  • Software testers should remain focused for technical growth and adapt to technological trends for keeping up their learning. There will be growth, security and confidence with inner peace and happiness.:)

In a nutshell, “Love your job”. Reap whatever you want from this career.

Please share your thoughts if you have anything to add as issue with respective solution from your experience.

Tester Meetup reaches Lahore

The Tester Meetups that have been a regular feature for the Isloo testers met the vibrant Lahori testers community over the last weekend. And results were as great as you can expect.

To gather a crowd of over 100 testers representing 30+ companies on a Saturday early morning in a city where dining out late is not just a habit but a ritual was the first sign of success. Then crisp presentations and well engaged discussions during the 3 hour long session made everyone believe that we’ve got it right. And when the event was over, it was not actually over as lots of chats were flowing to discuss many ideas like doing training, workshops, more meetups. The testing community is definitely on a mission in Pakistan.


(more photos of the event are on facebook)

The meetup which was a co-production of Pakistan Software Testing Board , Kualitatem and Information Technology University was held at the beautiful Arfa Software Technology Park . The session kickstarted with Dr. Zohaib Iqbal, President of PSTB, to introduce the current state of testing community in Pakistan and the value that such meetups can add. He recapped the journey so far and the plans ahead and asked for as many as volunteers as possible. His theme of “collaborate to achieve greatness” was well liked by the audience.

The first technical talk was by Rizwan Jaffri who is an experienced tester and has been actively attempting to engage the local community. Rizwan’s main point was that testing processes can co-exist in a SCRUM (or other Agile) setups and he spoke of his own experience how Test case Reviews and Test Practice reviews help improve the test planning and execution in current and future sprints. He was generous enough to share some templates that he has found useful and offered to contact him if you wish to use them.

Then Humaira Anwar took the stage who was one of the driving forces to bring the meetup to Lahore after she attended the Islamabad’s January edition. Humaira talked about the basic Security testing that a functional tester can do apart from the usual responsibilities. She shared the OWASP set of vulnerabilities and some common ways to take care of those issues. She also made a point by saying that organizations can’t afford dedicated Security Testers, so each tester should do minimal testing to make sure that the solutions their company is offering are secure.

The third talk was included in the event on popular demand. It was a replica of Dr. Uzair Khan’s talk at the Islamabad event where he explained how he and his team has adapted Model Based Testing to automate Game testing. The fun part was when he demo’ed an open source version of Mario and ran it in a simulated way to find bugs. Audience asked some intriguing questions which the humble Dr. gave quite satisfactory answers.

Before the tea break, the partners of the event shared their vision. Khurram Mir who is co-founder of Kualitatem, the biggest IT company in Pakistan to focus on Testing services, shared the story of success and how competing in the modern world requires raising personal standards of professionalism and self-learning. Dr. Adnan Noor Mian who is HoD of CS department at ITU did a quick tour of ITU offerings and extended his offer to do any such events in future as well.

The group break up for a delicious tea which was more of an excuse to indulge in even juicier discussions that happened in the gallery next to the seminar room. Lots of new connections were seen to be made there.

All testers assembled again to have a panel discussion on “Automation, Why and What to automate”. First the panelist shared their version of the story and then the audience poured in lots of questions. The panel included likes of Sadia Malik who is head of QA services at Kualitatem, Kamran Khan who has a decade plus of automation experience, Dr. Shaukat Ali who is a Ph.D. scholar at Simula Research lab, Norway, Muhammad Rafiq and Arif Masood (famous for his testerlogic blog) who are automation practitioners. Questions were a mixed bag, where people asked that they have certain years of experience and can they start automation now, if a code base is years old – whether you should automate that or not, what are the criteria of selection for automation and so on.

Given the lack of choices the organizers had, I was asked to close the session. After thanking all the participants who sacrificed their Saturday morning to their passion for the testing profession, I requested the audience to collaborate more and try the things they learned today at home. “Do try this at home” was what my message was.

Certificates were distributed among the presenters, panelists and organizers with a special thanks to Dr. Khurram Bhatti of ITU who took all the pain to arrange all the stuff which ensured the seamless execution of the event.

And your guess is right that this event is start of the story, not the end. We’ll be soon back in Lahore to have more sessions like above and now eyeing to do so in Karachi too. Our testers friends in Islamabad, don’t worry, we have some plans for you too:)


Get every new post delivered to your Inbox.

Join 1,542 other followers