Lahore being the cultural capital of Pakistan is best known for it’s festivity. There is a saying in Punjabi that goes like this “sutt din tay utth mailay, kum karaan main kairay waylay” which roughly translates to “There are 7 days in a week and there are 8 Festivals. How can I find any time to do my job?” But on last weekend, we intermixed the technical testing profession topics with the easy going table team discussions so that people can have fun while learning.
The idea was earlier testified few months ago in a meetup that happened in Islamabad and we thought to use the conventional wisdom to repeat things with some variation to newer places in another time. This time around we got great partnership of a technology giant from Lahore Confiz Ltd. who helped us connect so many Knowledge Testers through the independent platform of Pakistan Software Testing Board.
The event took place at a private hall which was specially arranged by the active Confiz testing team lead by Rizwan Jafri. The location provided the conducive environment for the discussion where ten round tables were setup each hosting 5-6 testers along with a Discussion Moderator who helped initiate, expand and summarize the discussion topic for that particular table. The members on each team were selected from our previous Lahore tester meetup participants such that we had testers with experience ranging from <1 years to 14+ years representing over 25 organizations to provide enough diverse opinions.
The planning for the event actually started many weeks before the actual meetup and I’m very much thankful to our tester friends in Lahore: Rizwan Jafri, Kamran Khan, Humaira Anwar, Muhammad Ayyaz and Qaiser Munir who meticulously designed the meetup. The evergreen guidance of Dr. Zohaib Iqbal and Dr. Uzair Khan was always there and Taimur Sarwar being the champion advocator of testing discussion added value by not only in the planning phase but also traveled from Islamabad to join the session with his team members.
The discussion topics were a bit similar to the one we had in Islamabad edition and bit new as introduced by our Discussion Moderators who were all from the Lahore testing community. I’m thankful to Ayyaz and Qaiser who took the additional role of moderation and they were joined by a very talented league of testers that included: Maissom Abbasi, Najabat Ali Khan, Muhammad Ahsan, Sadaf Khan, Ali Khalid, Waqar Ahmad, Maria Mukhtar and Zeeshan Dilawar.
(more photos coming soon on our facebook page)
Given the time constraint, the teams were given 35 minutes of discussion time after an opening note by me in which I talked about our philosophy behind holding such meetups. Emphasizing the necessity to push testers to meet new testers, I explained the notion of Echo Chambers and shared some ideas on how to come out of them. I also talked about how just listening to talks won’t mature your thought process unless you express yourself in a safe environment like the one provided by this meetup. To my surprise, my above murmurings about meetups was introduced as a Keynote.
For all of my readers who were not present at the event, below were the ten topics which were then presented by each team:
- Becoming a full stack Quality Engineer
- Tester-Programmer relationship
- Role of tester in User Acceptance Testing
- How important is Communication and Collaboration for Testers
- I’m bad at coding, can I still be a Tester?
- Performance Testing – what is involved when it is well-done
- What to and what not to automate
- The role of Tester in Agile teams
- Considerations in developing an Automation Framework
- Should QA Engineer be a Specialist or Generalist?
Each team summed up their discussion through a short presentation of <5 minutes on the stage. The beauty of the plan was that this job was done by the junior most tester on that table which helped our newer lot gain knowledge and confidence side by side. The audience then was invited to throw in questions on the topic or add the comments. The real takeaways were here and I’m sure everyone had a full list of notes from this learning experience to give them new perspectives. To give you an idea of the proceedings, let me share couple of examples.
On the topic of whether a Tester be good at coding, the discussion quickly turned into the balance of Domain knowledge and Technical knowledge that a Tester needs. The comments went in favor of Testers who don’t understand coding but strong at the domain knowledge are better vs. comments going in favor of Testers who know the layers of technical implementation being good. Eventually the hall kind of agreed that we need a mix of both.
Similarly during the discussion on what and what not to automate, some people believed that we need a cautious approach and can only automate within some constraints. But folks with practical knowledge explained that the decision should not be based on how much you know about automation or how much you can achieve, rather it should be based on how much automation is needed by the project you are executing.
These discussions were well summed up by closing note by Omair Sajid, Senior Director Engineering at Confiz who very much liked the idea of testers getting together and explore their profession. He talked through his experience that finding a real Tester has been a challenge and was happy to see that community is raising the standard through such events which will produce quality testers. He also shared his vision of automation and suggested that today’s modern solution developed at high pace for demanding situations cannot succeed until test automation is not applied.
As the certificates for Discussion Moderators and Organizers were being awarded, Dr. Zohaib took the opportunity and shared our future plans to keep this momentum going. It included a definitive plan of a meetup in Karachi in October along with another being planned in Lahore in a few month in collaboration with Systems Ltd. He also outlined the rough plans that we have to hold a national level software testing conference to happen towards end of 2016 which will be a yearly event and will move from one city to another.
Oh I forgot to mention that Lahore along with it’s festivity is known for the food and it was well proved by delicious and mouthwatering eateries that were offered with tea at the end. As the participants enjoyed the tasteful food, the environment resulted in lot of new connections being made on the fly and lots of ideas on how to work together to further build the amazing testing community that we have.
What ideas do you have for our future meetups?
My Mama used to say when I was a kid that “Don’t worry. Your mistake is not a mistake, if you learn from it”
In his book “Errors: bugs, boo-boos and blunders”, Jerry Weinberg talks the subject of errors at length and it was a delightful reading. As a software practitioner, we all see errors all the time. But if one wants to take the errors seriously, this book is a must read. To boost your appetite, let me share a few lessons that I learned from the book:
Errors are not a Moral thing
Yes, when we get to know that we were responsible for an error, we feel guilty. Some of it comes from our school time when we were told that making error is a moral thing and the one who does is a bad person. Unfortunately many Managers live in the school teacher mentality where they want to embarrass team members publicly when they make mistakes. Jerry emphasizes this and suggests to get out of this mind block to accept that all of us make errors. This should not result in doing nothing by saying “hey, what we can do as errors are a natural thing” rather this should result in focusing on what we can do to limit the error and how they can be stopped in future. In Jerry’s words:
To err is human; to combat error is software engineering.
Errors have a link with the management
When we study errors and causes of errors, we often focus on the Person who made the error. If we go beyond, we look at what was fed to that Person and how it was done. In terminology, it can be translated as how the requirements were defined, how the Programmer understood the requirements and how the Programmer implemented the requirements. All too often, we forget that the environment in which all these things happen and how it contributes to the errors made. And good management is all about create environment to reduce such errors.
Jerry has listed eight Fs of software failure and how management can work on them. But he also said right there:
But good management is so boring!
Errors will always exist
You might be thinking that you are a software engineering champion and you are a management champion, so you know the best of software processes that you’ll apply in best of your management style such that errors will never occur. That’s the pitfall for each Perfectionist and Jerry suggests that we need to live with the errors. He lists 8 Laws of Errors Defense to help us out and one of them is:
One thing worse than error is an error farm
As a tester, you might have seen many error farms where errors are produced in abundance. When we report errors, we should be looking for error farms and reporting them too.
How do you see errors? What lessons you have learned from them?
If you have ever travelled in a bus in rural parts of Pakistan, you’ll recognize this very well. Yes, those buses that are labelled as “Non-stop” because they stop a lot. And on each stop, many vendors move in the bus who sell all kind of food, household items and even medicines. Some of them leave the bus when it embarks for the next stop whereas the ones who sell really deep solutions like medicines, they usually prefer to stay in the bus and do their marketing until the next stop comes. The most common of them sell “Phakki”, “Munjon” etc. Though I can take any of the examples but I’d prefer the “Phakki” because it has a personal connection.
These vendors are highly skilled marketing people. First, they won’t start directly from what they sell rather they’d start the talk by praying for everyone’s successful journey and may start with a generic statement that encompasses all the passengers like “all my brothers and sisters; the elderly ones and the kids etc.” Then they explain at length the typical and common symptoms of the illnesses that people would have. To be specific, in case of a “Phakki” seller, he’d ask if you ever feel heaviness after eating hard, or do you have gas trouble or do you find it hard to ease you during various trips to the washrooms. They would pick problems that are common and remember that “the problems that are hard to go away”. After this five to ten minutes talk that would grab the attention of all the passengers, they’d then tell you that they a magical solution to all the problems mentioned before which is a “Phakki”. (For those who don’t know what it is, it is a well grinded mixture of many herbs and it is usually taken by putting it onto the palm of your hand and then tossed into the mouth and then water is gulped to take it inside. It is also called a “Chooran”)
(The photo is taken from here. No, I’m not endorsing their product, I just have a policy to add link to the original photo)
As a customer, you are set to believe that the magical “Phakki” can solve all the digestion problems of all the people in all the situations. That’s what they do it deliberately by making the message generic and that’s where if we ever buy a “Phakki” should be aware of. As I mentioned the personal connection and here it is: I do use “Phakki” (though I don’t buy from those bus vendors but prefer to get it from source), but I know that it cannot solve all my problems. So I use it to solve a “typical problem in a typical situation” and it perfectly works there. For other digestion problems, I take other medicines and also take a good control of what I eat when I have that problem.
Sorry for the long description of a thing that you may not like but I hope you know where I am taking this.
The many tools vendors especially those who sell automation tools would do the same marketing trick to you. After all humans are humans. Whether they are trying to cure digestion systems or information systems, they take easy paths. If you take a look at marketing techniques of those tools, they’ll be strikingly similar to the “Phakki” sellers i.e. they’d take the problems that are very common, the problems that are hard to go away like making a poor quality product a high quality one. They also often don’t provide the situations where not to use their tools and the associated hazards. They just tell you that you use our tool and “all the problems of all the teams in all the organizations in all situations will go away”.
Rather than taking the extreme path to simply stay away from such tools, I suggest you the same approach I mentioned above. Use that tool to solve “typical problem in typical situations in typical teams”. For other problems, use other tools or techniques. Keep in mind that “using a tool is not a strategy rather the strategy can have one or more tools”.
Same caution has to be applied when “processes” are sold to you that if you follow a certain standard like ISO or CMMI, define a super process that will solve all your problems. In this case many of the authors of those standards or processes do know that their solution will not solve “all the problems of all the people in all the situations” but their “product” is soon picked by marketers who are the trainers and implementers of those processes. They start making claims which are not always true. When you go for adapting a certain process in your culture, you should know that it will solve “some of the problems in some of the situations” and is not a magical “Phakki”.
As Jerry Weinberg writes in his book “Errors: Bugs, Boos-Boos, Blunders”:
There’s never going to be one best way to test and fix software, no matter what some marketing organization tries to sell you. Ultimately, you have to develop your own process, to create the unique service you can offer to your employer or sell to your clients.
Lets use the “Phakkis” aka “tools” and “processes” in the above fashion to live a healthy life and deliver healthy solutions.
Testers have many jobs at hand. They need to run as many as possible tests to find out useful information about the health of solution to help make decision makers take decisions. Anyone who has experience in testing knows that the real job is not to “run” tests but “design” tests. Similarly the quantity of tests that were executed doesn’t matter if they are not hitting the right target. That is why number of failing vs. passing test cases, number of new defects reported are a poor way to represent quality and hence a poor feed of information to the decision makers.
What matters most is perhaps what the test does and why it is important? In one of my previous posts, I suggested that testers become gray box testers to get some ideas on coming up with some serious tests and hence bugs. Today I want to bring the focus on the business side of the things.
Testing in a way serves a bridge between Business and Technology. You can say that this is true for the entire “engineering” team but perhaps testers are the main guys responsible for this.
(Picture from http://www.sutterstock.com with edits by me)
Given that the domain you serve will vary from organization to organization and from team to team, it is tough to lay down specific guidelines so as how you can understand the domain. A software serving to help design Nuclear Plants may require it’s testers to learn about all the standards of materials, quality and health/safety. Whereas a software serving the education sector may require it’s testers to understand how brains are educated and what quality education means and what are the experiments done in this area so far.
Therefore I’ll not mention the ‘how’ part of the problem rather focus on ‘why’ it is important for testers to learn more about the domain as the “Golden Circle” suggests that ‘why’ is more important than ‘what’ and ‘how’.
- You talk the language of your users
Very early in my career, I was visiting a client with a team to understand the requirements of a solution that we were building for their pharmaceutical setup. Client was represented by 3-4 managers and they were throwing a lot of terms. I don’t remember the exact term of one of the reports that they wanted but one of our team members said the name of the report wrong. Manager politely corrected. When our team member made the same mistake in the conversation, the client said:
“How come you make a solution for us, when you don’t even name the things we need?”
I learned a lesson that day and always prefer to use the terminology that our users would use. That makes all the internal communication that you do as tester become more “user focused” and you know “what user wants will always be fixed”.
- You augment the team with “Business facing tests”
Keeping the Agile Quadrants in your mind, you know that your team specially the Programmers are writing most of the technology facing tests, so if you dedicate your efforts towards the tests that mimic the business cases that’s like a perfect combination.
- You know what drives what
Pieces when combined become the solution. So as you move your “vision of the tree” to the “vision of the forest”, you start gaining more understanding of why a particular piece is needed to solve a particular problem. This helps you write tests that are like end-to-end or workflow tests (the tests that achieve a particular user workflow). Often such tests results in bugs that surely increase your reputation as a tester who knows how the system works.
How much of your time as tester goes into understanding the domain? And do you list more benefits of this investment?
Learning happens in many ways and perhaps the most powerful way is to share your thoughts with like-minded people. That’s exactly what last weekend witnessed when a meetup took place in Islamabad where we broke participants in teams and gave them topics to discuss. The findings were then presented to all the audience and the awesome testers added their opinions. The result was just as fantastic as the title of the meetup was. One sign of it being that when it was announced from stage: “the time for discussion is over, so please conclude and get ready to present”, no body listened to the announcement and they kept talking in their tables.
It is difficult for me to say but the idea of such kind of setup was not mine🙂 It was Taimur Sarwar’s brain child who coined this at our last meetup. As our previous efforts were to gather more and more testers in different cities, we thought it’s about time to bring them closer. S&P Global Pakistan was generous enough to offer their space to conduct the event. The settings were perfect, the food was delicious and the souvenirs given to each participant was cherry on the top.
To have this session in an effective way, there were a few VP (Planning & Coordination) positions which was happily accepted by Amir Shahzad, Sadaf Minhas, Dr. Zohiab Iqbal, Dr. Uzair Khan, Taimur and myself. At least I can tell people that I was once a VP in my life, though Dr. Zohaib being President of Pakistan Software Testing Board didn’t like the demoted title. But who cares.
This team met few weeks before the event and did what their title suggested. The plan was like this:
Bring in about 30 testers mostly in the range of 0-3 years of experience, break them in few teams, provide them topics, provide them a Facilitator who is someone with experience, they discuss the topic and present to the larger group. That’s it! and the result was amazing.
So here came the Facilitators with whom we had another planning meeting which in itself was kind of a mini-meetup where ~10 top brains of testing from the town pondered together how to best conduct the sessions. Along with VPs, these were Ahmed Mugheera, Haider Farooq, Kashif Butt, Hanan Atif, Hina Zafar and Farhana Adeel.
Enough for the preparations details, let’s move to the event day. As testers arrived, they were allotted tables and thus Facilitators with topics. Before the brain storming could begin, a keynote was delivered by Afif Zaidi, who is Director IT at S&P Global and started his career as a Software Tester. Afif recapped his illustrious career in few minutes and then shared his vision of testing. His description of Testing as a bridge between Business, Technology and Project Management was impressive and the tone was set for the participants to get knowledgeable about all of them.
There were two topics given to each table with one looking towards the testing career and other meant to increase the technical knowledge of the table. The full list of topics is below:
Career and Skills topics:
- What is Quality in general life? How do you differentiate between Quality Assurance and Quality Control?
- Importance of Communication & Collaboration and when to seek help vs. do yourself.
- Tester – Programmer relationship
- I’m bad at coding, can I still be a Tester? I’m not a developer, do I still have a career path?
- Role of tester in Agile.
Testing Techniques and Concepts topics:
- Exploratory Testing, Session based testing
- Bug life cycle
- Effective bug reporting
- How automation fits in? What to automate and what to not? What is possible vs. practical?
- How can I improve domain knowledge?
As you can imagine that each of the above topics is a whole day thing, but teams did an effective job to discuss them in ~45 minutes and then present in a few minutes at stage which followed by comments and questions from the audience. There were so much new ideas that flowed in and so many new viewpoints thrown in that I can think I can write blog posts for the whole next year if I keep covering them🙂
Just to give you an idea of the proceedings, I’d like to share two takeaways. While discussing domain knowledge, the hall kind of agreed that Testers should do that in depth. In short, if you are serving a financial industry you need to learn different type of things than you’d learn if you are serving the health industry. Many testers don’t realize that their job is not to test the code, rather their job is to test the solution.
Another one was related to effective bug reporting where some believed that testers should write a super duper bug report that everyone could understand and is flawless. But reality is far away from this and a bug report initiates conversation within the team about an issue which is a potential threat to the release. Many testers don’t realize that bug report is not their accomplishment report, rather it is the starting point of evaluating the risks.
Other than the above, there was a table of senior testers who discussed and then presented their thoughts on Testing Strategy and Planning with some insights into estimation and prioritization of testing activities.
The closing keynote was delivered by Badar Khan who is Managing Director of NMX Global Software Islamabad center and he also started his career as a tester. Badar shared his achievements in a humble way and recapped his almost two decade’s journey as a tester. His advice was simple: you should always be truthful. Yes, as a tester don’t hide what you know and share it. He also shared a nice recent example where a higher up Manager could be wrong and send you in wrong direction; so what you should do in that case.
As you can imagine from this summary that it was hell of knowledge to digest in a day and we were lucky to have some brilliant testers present there who exactly did that. And more importantly they built new connections which will help them in their journey into the testing career.
I know we couldn’t invite all of testers in our network to the event due to the limited capacity. But don’t worry, we have plans to do more in coming quarters along with a national level testing conference towards end of 2016. Insha Allah (God willing).
How do you see such type of meetup compared to the one we had earlier? How we should do our next sessions?
One of the standard distinction between testing types is that they are either Black box or White box. When you ask “What do you know about testing?”, this is one of the first sentence you hear from newbies into the testing world. They are not wrong because they tend to believe that world is simple and is binary. Alas, the real world is lot more complex and is fuzzy.
That’s why we need more “Gray box” testers or as the punch line of my blog says “Software Tester equipped with Knowledge”. But wait what is the distinction.
I really don’t like the term Black vs. White box testing at the first place as an Opaque vs. Transparent box testing would explain it better. But we can’t do much about what the industry talks. (Think about how linguistics complain about the original word but the one that public speaks matters the most).
So here is a borrowed comparison from the web which explains that Black box is when you don’t know the internal details, White is when you know all of them and Gray is when you know some of them. Again this is not the complete picture, but should work for our purpose.
To give you an example, think about the ignition system of your vehicle. For a Driver (read Black box tester), when the key is turned, or the button is pressed, the engine should start. That’s it. A white box tester knows that depending upon what type of engine is being used, the ignition system gets current from the battery, pulls in the fuel and gathers the outside air into the engine and then combustion happens by the spark provided by the current. This in turns moves the pistons which move the crankshaft which moves the transmission line going to the gear system. That’s too much detail that might be useful for a mechanic who fixes the car but can blow off a driver’s mind, that’s why we need Gray box testers.
Taking this example further, the Gray box tester should at least know the building blocks of the ignition system, so that in case of a problem it can be figured out if the battery is weak or fuel is not being pumped etc. That’s what drivers with some mechanic like skills. These are the Gray box testers.
If you have survived this example, let me take back to the software world.
Given that the job of a Software developer is to build a technical solution to solve a business problem, then a Tester’s job is to make sure the solution works. That’s where Tester should know the business well and they should know the technology well.
There will be many benefits when you’ll learn the internal details as a Gray box tester but I can guarantee these three gains for sure:
- You talk the language of the town
Have you ever been to a place where you don’t know the language of the town? You find it very hard to get directions or order meals or in short explain what you want. The same thing happens if you cannot talk technical to the team who mainly talks in terms of technology. For example reporting a bug that says “Export fails while processing XYZ type of stuff as the libraries to support this is missing” is way better than saying that “Export fails while processing ABC file”.
- You know what changes what
In these fast paced development cycles, builds to test come too fast. And we don’t have time to test it all again and again. No tester can survive doing the whole regression testing all the time and the only solution is to test the delta. You can only determine what areas to test for a given change, if you know the internal details of how things are linked together.
- You can write specific intriguing tests
As they say that “The Devil is in the details”. Anybody can write tests that checks the surface area of the product but the real tests that add value to the team and the consumer (read technology and business) are the ones who dig deep into the solution. These well-crafted tests can only be written and executed by a Tester who knows that area well and kind of “lives there”.
If you like the idea, the first thing might be drawing a block diagram / architectural diagram of the project / product you are working on and show it to your Programmer friends.
Are you ready to start this journey?
Ahmed is running the smoke test for 35th time in the last six months and feels like quitting testing. Smoke testing finishes in an hour or so, results posted and Ahmed returns to other testing tasks. Ahmed thinks that testing is not as boring as it was earlier in the day and keeps enjoying the job.
What is that makes Smoke testing so boring?
Boredom is defined in many ways and usually refers to a state when you have a situation which doesn’t satisfy your mental needs of that time. There are two parts of it: a) Situation and b) the thoughts in the mind.
If you are bored like my 5 year daughter who after playing her favorite game on iPAD for about an hour, doing cycling for a while, eating the food of her choice, comes to her mother and says “I’m feeling bored, can you play with me?” In this case, this is clearly the state of the mind that needs to be controlled. The 5 year old kid can’t do it but we as mature professionals should be able to as some parts of our jobs will always be boring.
But wait, is the only advice I have for you is “change yourself as you can’t change things around you”? I know you hear this a lot in your life and feel sick of such advisers as they seem to not understand the situation you are in. Ahh, situation, yes that situation is also part of the equation. Let’s see if we can do something about it?
The smoke testing process is like this: download the build, install / setup, run the same tests you ran a few days ago, record results in an ancient software called Excel, share results via email.
Automate the setup parts
By the time as tester, Ahmed reaches to the interesting part of running tests and observing the results, he is already bored by setting things up. So the plan is to bring a fresh testing mind at the start of running tests while everything before it is done by the miracles of automation.
While there remains many question on test automation that can add value, no one questions the powers of automation when it comes to doing stuff that doesn’t involve intelligence. And set up is exactly that.
In our team, for example, we have a Python script that keeps sniffing the FTP site where builds are posted. As soon as a build arrives, it downloads it, unzips it and places at our test machine along with needed dataset. In our case, we don’t need specific testing environment, but I have seen other teams do the additional part to kick off a Virtual Machine and install the build there. Or you can have a QA version of your web application where it goes. The choice is yours, but this effort on automation will certainly pay you off. Give this a try.
Test the delta, not the entire stuff
Given the agilish nature of projects these days, the builds come too often. The more you have to do a thing in same way, the more boring it becomes.
The trick here is to be aware of the changeset that you have in the latest build compared to last build to define the scope of your smoke test. Rather than running all tests with same focus, run the ones that might be affected by recent changes first with full attention. Then rest can be let go.
If you don’t know how to get those changes, get in touch with Programmers in your team. In our case, the build is generated from a server and we could extract the “build description” for any two builds and then compare them.
By adapting this technique, the attention of tester will improve in significant way along with the overall understanding of what is changing and what parts of the system are moved by those changes.
Report results in charming way
I know that testers do lot of effort in designing and executing the tests, but when it comes to showing off their findings to the world, they end up with some dull sheets or large list of numbers. For example, this is how we used to present our Smoke testing activity at the end of sprint:
“Smoke Test executed 6 times for each build.”
Now that sounds like boring to the reader/listener also and Project Managers say to themselves “Thanks God, we have testers who do these dirty jobs for us”.
We converted this boring way into a live page on our SharePoint internal site, where results are updated each time a smoke test is executed. This is how it looks and is being liked by testers and non-testers alike.
Another idea that I heard from another tester was to use the smiley face for a Passing result and a sad one for Failing. You can try that or try any other powerful way to show results to make your reports management friendly.
Have you felt that smoke testing is boring and did you try something else than the above?