“All testers are special but some are more special than others”. To borrow a line from Animal Farm with some variation.
In last month’s testers meetup, one of the discussion points was “Should testers be Generalist or Specialist?” As the discussion was happening, I decided to write a blog post on this subject as it is close to my heart. It took me almost a month to do that which is bad given the fast paced world we live in but not as bad if compared with the whole span of life.
I thought I’ll start with an overview of what we mean by Specialist and Generalist etc. but my draft was in progress when I saw this excellent post at Testastic blog which I think lays a good foundation for the readers. Also as I was researching this topic, I found this great webinar by Derk-Jan de Grood and Jan Jaap Cannegieter done for EuroSTAR which introduces T and Pi shaped very well. My advice would be to leave my blog here and first go to the above links.
Thanks to the sincere readers of the blogs for consulting the references and welcome back. To all other casual readerrs, T shaped professional is a person who has a depth of knowledge on a Specialty but has enough breadth of knowledge on the General subject. A typical example that I give is from medical profession i.e. an ENT specialist while recommending a medicine needs to be aware if the patient has heart disease or is pregnant to avoid any side effects. A Pi shaped professional is then the one who has two special legs meaning they are specialists on two subjects. See the picture below that I copied from the webinar slides that I mentioned above.
(Slide from webinar: https://www.youtube.com/watch?v=XzA-FvRG0DE )
With that background, let’s move to the topic now.
First and foremost, a Tester is already a T shaped professional. If not, tester should be. Testers are members of the team who should have enough knowledge of different functions of the teams like Development, Deployment, Build process, Documentation, Release management etc. yet their specialty lies in Testing. The teams are aware of that and they usually respect that by leaving the testing job to the Testing Specialists.
Some people gets confused with the cross-functional aspect of Agile teams and say that all members test so how come a Tester is special. But good teams know that having a Specialist tester help. In one of the teams that I joined last year, the project lead said to me:
Majd, I don’t want you to spend most of the time writing tests. I’d like to spend most of your time to see how we are doing on the testing front.
So if you are a Tester, you are already a Specialist on testing and you should rather be a T shaped professional by learning the other functions of the team.
The question then comes to developing the other specialty or second leg of a Pi shaped testers. In the webinar mentioned above, when the presenters did few rounds of workshops with testers they found out that the second leg can be Programming, Management, Automation etc. All of those are good but in my opinion for a Tester, knowledge of other fields like Programming or Management actually makes them a better T shaped Tester by expanding breadth of knowledge. A Pi shaped tester’s other leg should also be in testing.
I don’t want to force my opinion and you are free to pick second legs other than a Testing area. But if you take my advice, I’d suggest the other legs could be Performance testing, Security Testing, Usability Testing, Automated Testing etc. By that I mean that you should be subject matter specialist on the subject and not just a Generalist on all. If you want to get started, here is a piece from tester friend Rizwan Jafri that came out this week about automation.
Are you a T shaped or Pi shaped tester? If not, at what stage you are. And if yes, what are the second leg options you’d like to share with us.
Lahore being the cultural capital of Pakistan is best known for it’s festivity. There is a saying in Punjabi that goes like this “sutt din tay utth mailay, kum karaan main kairay waylay” which roughly translates to “There are 7 days in a week and there are 8 Festivals. How can I find any time to do my job?” But on last weekend, we intermixed the technical testing profession topics with the easy going table team discussions so that people can have fun while learning.
The idea was earlier testified few months ago in a meetup that happened in Islamabad and we thought to use the conventional wisdom to repeat things with some variation to newer places in another time. This time around we got great partnership of a technology giant from Lahore Confiz Ltd. who helped us connect so many Knowledge Testers through the independent platform of Pakistan Software Testing Board.
The event took place at a private hall which was specially arranged by the active Confiz testing team lead by Rizwan Jafri. The location provided the conducive environment for the discussion where ten round tables were setup each hosting 5-6 testers along with a Discussion Moderator who helped initiate, expand and summarize the discussion topic for that particular table. The members on each team were selected from our previous Lahore tester meetup participants such that we had testers with experience ranging from <1 years to 14+ years representing over 25 organizations to provide enough diverse opinions.
The planning for the event actually started many weeks before the actual meetup and I’m very much thankful to our tester friends in Lahore: Rizwan Jafri, Kamran Khan, Humaira Anwar, Muhammad Ayyaz and Qaiser Munir who meticulously designed the meetup. The evergreen guidance of Dr. Zohaib Iqbal and Dr. Uzair Khan was always there and Taimur Sarwar being the champion advocator of testing discussion added value by not only in the planning phase but also traveled from Islamabad to join the session with his team members.
The discussion topics were a bit similar to the one we had in Islamabad edition and bit new as introduced by our Discussion Moderators who were all from the Lahore testing community. I’m thankful to Ayyaz and Qaiser who took the additional role of moderation and they were joined by a very talented league of testers that included: Maissom Abbasi, Najabat Ali Khan, Muhammad Ahsan, Sadaf Khan, Ali Khalid, Waqar Ahmad, Maria Mukhtar and Zeeshan Dilawar.
(more photos coming soon on our facebook page)
Given the time constraint, the teams were given 35 minutes of discussion time after an opening note by me in which I talked about our philosophy behind holding such meetups. Emphasizing the necessity to push testers to meet new testers, I explained the notion of Echo Chambers and shared some ideas on how to come out of them. I also talked about how just listening to talks won’t mature your thought process unless you express yourself in a safe environment like the one provided by this meetup. To my surprise, my above murmurings about meetups was introduced as a Keynote.
For all of my readers who were not present at the event, below were the ten topics which were then presented by each team:
- Becoming a full stack Quality Engineer
- Tester-Programmer relationship
- Role of tester in User Acceptance Testing
- How important is Communication and Collaboration for Testers
- I’m bad at coding, can I still be a Tester?
- Performance Testing – what is involved when it is well-done
- What to and what not to automate
- The role of Tester in Agile teams
- Considerations in developing an Automation Framework
- Should QA Engineer be a Specialist or Generalist?
Each team summed up their discussion through a short presentation of <5 minutes on the stage. The beauty of the plan was that this job was done by the junior most tester on that table which helped our newer lot gain knowledge and confidence side by side. The audience then was invited to throw in questions on the topic or add the comments. The real takeaways were here and I’m sure everyone had a full list of notes from this learning experience to give them new perspectives. To give you an idea of the proceedings, let me share couple of examples.
On the topic of whether a Tester be good at coding, the discussion quickly turned into the balance of Domain knowledge and Technical knowledge that a Tester needs. The comments went in favor of Testers who don’t understand coding but strong at the domain knowledge are better vs. comments going in favor of Testers who know the layers of technical implementation being good. Eventually the hall kind of agreed that we need a mix of both.
Similarly during the discussion on what and what not to automate, some people believed that we need a cautious approach and can only automate within some constraints. But folks with practical knowledge explained that the decision should not be based on how much you know about automation or how much you can achieve, rather it should be based on how much automation is needed by the project you are executing.
These discussions were well summed up by closing note by Omair Sajid, Senior Director Engineering at Confiz who very much liked the idea of testers getting together and explore their profession. He talked through his experience that finding a real Tester has been a challenge and was happy to see that community is raising the standard through such events which will produce quality testers. He also shared his vision of automation and suggested that today’s modern solution developed at high pace for demanding situations cannot succeed until test automation is not applied.
As the certificates for Discussion Moderators and Organizers were being awarded, Dr. Zohaib took the opportunity and shared our future plans to keep this momentum going. It included a definitive plan of a meetup in Karachi in October along with another being planned in Lahore in a few month in collaboration with Systems Ltd. He also outlined the rough plans that we have to hold a national level software testing conference to happen towards end of 2016 which will be a yearly event and will move from one city to another.
Oh I forgot to mention that Lahore along with it’s festivity is known for the food and it was well proved by delicious and mouthwatering eateries that were offered with tea at the end. As the participants enjoyed the tasteful food, the environment resulted in lot of new connections being made on the fly and lots of ideas on how to work together to further build the amazing testing community that we have.
What ideas do you have for our future meetups?
My Mama used to say when I was a kid that “Don’t worry. Your mistake is not a mistake, if you learn from it”
In his book “Errors: bugs, boo-boos and blunders”, Jerry Weinberg talks the subject of errors at length and it was a delightful reading. As a software practitioner, we all see errors all the time. But if one wants to take the errors seriously, this book is a must read. To boost your appetite, let me share a few lessons that I learned from the book:
Errors are not a Moral thing
Yes, when we get to know that we were responsible for an error, we feel guilty. Some of it comes from our school time when we were told that making error is a moral thing and the one who does is a bad person. Unfortunately many Managers live in the school teacher mentality where they want to embarrass team members publicly when they make mistakes. Jerry emphasizes this and suggests to get out of this mind block to accept that all of us make errors. This should not result in doing nothing by saying “hey, what we can do as errors are a natural thing” rather this should result in focusing on what we can do to limit the error and how they can be stopped in future. In Jerry’s words:
To err is human; to combat error is software engineering.
Errors have a link with the management
When we study errors and causes of errors, we often focus on the Person who made the error. If we go beyond, we look at what was fed to that Person and how it was done. In terminology, it can be translated as how the requirements were defined, how the Programmer understood the requirements and how the Programmer implemented the requirements. All too often, we forget that the environment in which all these things happen and how it contributes to the errors made. And good management is all about create environment to reduce such errors.
Jerry has listed eight Fs of software failure and how management can work on them. But he also said right there:
But good management is so boring!
Errors will always exist
You might be thinking that you are a software engineering champion and you are a management champion, so you know the best of software processes that you’ll apply in best of your management style such that errors will never occur. That’s the pitfall for each Perfectionist and Jerry suggests that we need to live with the errors. He lists 8 Laws of Errors Defense to help us out and one of them is:
One thing worse than error is an error farm
As a tester, you might have seen many error farms where errors are produced in abundance. When we report errors, we should be looking for error farms and reporting them too.
How do you see errors? What lessons you have learned from them?
If you have ever travelled in a bus in rural parts of Pakistan, you’ll recognize this very well. Yes, those buses that are labelled as “Non-stop” because they stop a lot. And on each stop, many vendors move in the bus who sell all kind of food, household items and even medicines. Some of them leave the bus when it embarks for the next stop whereas the ones who sell really deep solutions like medicines, they usually prefer to stay in the bus and do their marketing until the next stop comes. The most common of them sell “Phakki”, “Munjon” etc. Though I can take any of the examples but I’d prefer the “Phakki” because it has a personal connection.
These vendors are highly skilled marketing people. First, they won’t start directly from what they sell rather they’d start the talk by praying for everyone’s successful journey and may start with a generic statement that encompasses all the passengers like “all my brothers and sisters; the elderly ones and the kids etc.” Then they explain at length the typical and common symptoms of the illnesses that people would have. To be specific, in case of a “Phakki” seller, he’d ask if you ever feel heaviness after eating hard, or do you have gas trouble or do you find it hard to ease you during various trips to the washrooms. They would pick problems that are common and remember that “the problems that are hard to go away”. After this five to ten minutes talk that would grab the attention of all the passengers, they’d then tell you that they a magical solution to all the problems mentioned before which is a “Phakki”. (For those who don’t know what it is, it is a well grinded mixture of many herbs and it is usually taken by putting it onto the palm of your hand and then tossed into the mouth and then water is gulped to take it inside. It is also called a “Chooran”)
(The photo is taken from here. No, I’m not endorsing their product, I just have a policy to add link to the original photo)
As a customer, you are set to believe that the magical “Phakki” can solve all the digestion problems of all the people in all the situations. That’s what they do it deliberately by making the message generic and that’s where if we ever buy a “Phakki” should be aware of. As I mentioned the personal connection and here it is: I do use “Phakki” (though I don’t buy from those bus vendors but prefer to get it from source), but I know that it cannot solve all my problems. So I use it to solve a “typical problem in a typical situation” and it perfectly works there. For other digestion problems, I take other medicines and also take a good control of what I eat when I have that problem.
Sorry for the long description of a thing that you may not like but I hope you know where I am taking this.
The many tools vendors especially those who sell automation tools would do the same marketing trick to you. After all humans are humans. Whether they are trying to cure digestion systems or information systems, they take easy paths. If you take a look at marketing techniques of those tools, they’ll be strikingly similar to the “Phakki” sellers i.e. they’d take the problems that are very common, the problems that are hard to go away like making a poor quality product a high quality one. They also often don’t provide the situations where not to use their tools and the associated hazards. They just tell you that you use our tool and “all the problems of all the teams in all the organizations in all situations will go away”.
Rather than taking the extreme path to simply stay away from such tools, I suggest you the same approach I mentioned above. Use that tool to solve “typical problem in typical situations in typical teams”. For other problems, use other tools or techniques. Keep in mind that “using a tool is not a strategy rather the strategy can have one or more tools”.
Same caution has to be applied when “processes” are sold to you that if you follow a certain standard like ISO or CMMI, define a super process that will solve all your problems. In this case many of the authors of those standards or processes do know that their solution will not solve “all the problems of all the people in all the situations” but their “product” is soon picked by marketers who are the trainers and implementers of those processes. They start making claims which are not always true. When you go for adapting a certain process in your culture, you should know that it will solve “some of the problems in some of the situations” and is not a magical “Phakki”.
As Jerry Weinberg writes in his book “Errors: Bugs, Boos-Boos, Blunders”:
There’s never going to be one best way to test and fix software, no matter what some marketing organization tries to sell you. Ultimately, you have to develop your own process, to create the unique service you can offer to your employer or sell to your clients.
Lets use the “Phakkis” aka “tools” and “processes” in the above fashion to live a healthy life and deliver healthy solutions.
Testers have many jobs at hand. They need to run as many as possible tests to find out useful information about the health of solution to help make decision makers take decisions. Anyone who has experience in testing knows that the real job is not to “run” tests but “design” tests. Similarly the quantity of tests that were executed doesn’t matter if they are not hitting the right target. That is why number of failing vs. passing test cases, number of new defects reported are a poor way to represent quality and hence a poor feed of information to the decision makers.
What matters most is perhaps what the test does and why it is important? In one of my previous posts, I suggested that testers become gray box testers to get some ideas on coming up with some serious tests and hence bugs. Today I want to bring the focus on the business side of the things.
Testing in a way serves a bridge between Business and Technology. You can say that this is true for the entire “engineering” team but perhaps testers are the main guys responsible for this.
(Picture from http://www.sutterstock.com with edits by me)
Given that the domain you serve will vary from organization to organization and from team to team, it is tough to lay down specific guidelines so as how you can understand the domain. A software serving to help design Nuclear Plants may require it’s testers to learn about all the standards of materials, quality and health/safety. Whereas a software serving the education sector may require it’s testers to understand how brains are educated and what quality education means and what are the experiments done in this area so far.
Therefore I’ll not mention the ‘how’ part of the problem rather focus on ‘why’ it is important for testers to learn more about the domain as the “Golden Circle” suggests that ‘why’ is more important than ‘what’ and ‘how’.
- You talk the language of your users
Very early in my career, I was visiting a client with a team to understand the requirements of a solution that we were building for their pharmaceutical setup. Client was represented by 3-4 managers and they were throwing a lot of terms. I don’t remember the exact term of one of the reports that they wanted but one of our team members said the name of the report wrong. Manager politely corrected. When our team member made the same mistake in the conversation, the client said:
“How come you make a solution for us, when you don’t even name the things we need?”
I learned a lesson that day and always prefer to use the terminology that our users would use. That makes all the internal communication that you do as tester become more “user focused” and you know “what user wants will always be fixed”.
- You augment the team with “Business facing tests”
Keeping the Agile Quadrants in your mind, you know that your team specially the Programmers are writing most of the technology facing tests, so if you dedicate your efforts towards the tests that mimic the business cases that’s like a perfect combination.
- You know what drives what
Pieces when combined become the solution. So as you move your “vision of the tree” to the “vision of the forest”, you start gaining more understanding of why a particular piece is needed to solve a particular problem. This helps you write tests that are like end-to-end or workflow tests (the tests that achieve a particular user workflow). Often such tests results in bugs that surely increase your reputation as a tester who knows how the system works.
How much of your time as tester goes into understanding the domain? And do you list more benefits of this investment?
Learning happens in many ways and perhaps the most powerful way is to share your thoughts with like-minded people. That’s exactly what last weekend witnessed when a meetup took place in Islamabad where we broke participants in teams and gave them topics to discuss. The findings were then presented to all the audience and the awesome testers added their opinions. The result was just as fantastic as the title of the meetup was. One sign of it being that when it was announced from stage: “the time for discussion is over, so please conclude and get ready to present”, no body listened to the announcement and they kept talking in their tables.
It is difficult for me to say but the idea of such kind of setup was not mine🙂 It was Taimur Sarwar’s brain child who coined this at our last meetup. As our previous efforts were to gather more and more testers in different cities, we thought it’s about time to bring them closer. S&P Global Pakistan was generous enough to offer their space to conduct the event. The settings were perfect, the food was delicious and the souvenirs given to each participant was cherry on the top.
To have this session in an effective way, there were a few VP (Planning & Coordination) positions which was happily accepted by Amir Shahzad, Sadaf Minhas, Dr. Zohiab Iqbal, Dr. Uzair Khan, Taimur and myself. At least I can tell people that I was once a VP in my life, though Dr. Zohaib being President of Pakistan Software Testing Board didn’t like the demoted title. But who cares.
This team met few weeks before the event and did what their title suggested. The plan was like this:
Bring in about 30 testers mostly in the range of 0-3 years of experience, break them in few teams, provide them topics, provide them a Facilitator who is someone with experience, they discuss the topic and present to the larger group. That’s it! and the result was amazing.
So here came the Facilitators with whom we had another planning meeting which in itself was kind of a mini-meetup where ~10 top brains of testing from the town pondered together how to best conduct the sessions. Along with VPs, these were Ahmed Mugheera, Haider Farooq, Kashif Butt, Hanan Atif, Hina Zafar and Farhana Adeel.
Enough for the preparations details, let’s move to the event day. As testers arrived, they were allotted tables and thus Facilitators with topics. Before the brain storming could begin, a keynote was delivered by Afif Zaidi, who is Director IT at S&P Global and started his career as a Software Tester. Afif recapped his illustrious career in few minutes and then shared his vision of testing. His description of Testing as a bridge between Business, Technology and Project Management was impressive and the tone was set for the participants to get knowledgeable about all of them.
There were two topics given to each table with one looking towards the testing career and other meant to increase the technical knowledge of the table. The full list of topics is below:
Career and Skills topics:
- What is Quality in general life? How do you differentiate between Quality Assurance and Quality Control?
- Importance of Communication & Collaboration and when to seek help vs. do yourself.
- Tester – Programmer relationship
- I’m bad at coding, can I still be a Tester? I’m not a developer, do I still have a career path?
- Role of tester in Agile.
Testing Techniques and Concepts topics:
- Exploratory Testing, Session based testing
- Bug life cycle
- Effective bug reporting
- How automation fits in? What to automate and what to not? What is possible vs. practical?
- How can I improve domain knowledge?
As you can imagine that each of the above topics is a whole day thing, but teams did an effective job to discuss them in ~45 minutes and then present in a few minutes at stage which followed by comments and questions from the audience. There were so much new ideas that flowed in and so many new viewpoints thrown in that I can think I can write blog posts for the whole next year if I keep covering them🙂
Just to give you an idea of the proceedings, I’d like to share two takeaways. While discussing domain knowledge, the hall kind of agreed that Testers should do that in depth. In short, if you are serving a financial industry you need to learn different type of things than you’d learn if you are serving the health industry. Many testers don’t realize that their job is not to test the code, rather their job is to test the solution.
Another one was related to effective bug reporting where some believed that testers should write a super duper bug report that everyone could understand and is flawless. But reality is far away from this and a bug report initiates conversation within the team about an issue which is a potential threat to the release. Many testers don’t realize that bug report is not their accomplishment report, rather it is the starting point of evaluating the risks.
Other than the above, there was a table of senior testers who discussed and then presented their thoughts on Testing Strategy and Planning with some insights into estimation and prioritization of testing activities.
The closing keynote was delivered by Badar Khan who is Managing Director of NMX Global Software Islamabad center and he also started his career as a tester. Badar shared his achievements in a humble way and recapped his almost two decade’s journey as a tester. His advice was simple: you should always be truthful. Yes, as a tester don’t hide what you know and share it. He also shared a nice recent example where a higher up Manager could be wrong and send you in wrong direction; so what you should do in that case.
As you can imagine from this summary that it was hell of knowledge to digest in a day and we were lucky to have some brilliant testers present there who exactly did that. And more importantly they built new connections which will help them in their journey into the testing career.
I know we couldn’t invite all of testers in our network to the event due to the limited capacity. But don’t worry, we have plans to do more in coming quarters along with a national level testing conference towards end of 2016. Insha Allah (God willing).
How do you see such type of meetup compared to the one we had earlier? How we should do our next sessions?
One of the standard distinction between testing types is that they are either Black box or White box. When you ask “What do you know about testing?”, this is one of the first sentence you hear from newbies into the testing world. They are not wrong because they tend to believe that world is simple and is binary. Alas, the real world is lot more complex and is fuzzy.
That’s why we need more “Gray box” testers or as the punch line of my blog says “Software Tester equipped with Knowledge”. But wait what is the distinction.
I really don’t like the term Black vs. White box testing at the first place as an Opaque vs. Transparent box testing would explain it better. But we can’t do much about what the industry talks. (Think about how linguistics complain about the original word but the one that public speaks matters the most).
So here is a borrowed comparison from the web which explains that Black box is when you don’t know the internal details, White is when you know all of them and Gray is when you know some of them. Again this is not the complete picture, but should work for our purpose.
To give you an example, think about the ignition system of your vehicle. For a Driver (read Black box tester), when the key is turned, or the button is pressed, the engine should start. That’s it. A white box tester knows that depending upon what type of engine is being used, the ignition system gets current from the battery, pulls in the fuel and gathers the outside air into the engine and then combustion happens by the spark provided by the current. This in turns moves the pistons which move the crankshaft which moves the transmission line going to the gear system. That’s too much detail that might be useful for a mechanic who fixes the car but can blow off a driver’s mind, that’s why we need Gray box testers.
Taking this example further, the Gray box tester should at least know the building blocks of the ignition system, so that in case of a problem it can be figured out if the battery is weak or fuel is not being pumped etc. That’s what drivers with some mechanic like skills. These are the Gray box testers.
If you have survived this example, let me take back to the software world.
Given that the job of a Software developer is to build a technical solution to solve a business problem, then a Tester’s job is to make sure the solution works. That’s where Tester should know the business well and they should know the technology well.
There will be many benefits when you’ll learn the internal details as a Gray box tester but I can guarantee these three gains for sure:
- You talk the language of the town
Have you ever been to a place where you don’t know the language of the town? You find it very hard to get directions or order meals or in short explain what you want. The same thing happens if you cannot talk technical to the team who mainly talks in terms of technology. For example reporting a bug that says “Export fails while processing XYZ type of stuff as the libraries to support this is missing” is way better than saying that “Export fails while processing ABC file”.
- You know what changes what
In these fast paced development cycles, builds to test come too fast. And we don’t have time to test it all again and again. No tester can survive doing the whole regression testing all the time and the only solution is to test the delta. You can only determine what areas to test for a given change, if you know the internal details of how things are linked together.
- You can write specific intriguing tests
As they say that “The Devil is in the details”. Anybody can write tests that checks the surface area of the product but the real tests that add value to the team and the consumer (read technology and business) are the ones who dig deep into the solution. These well-crafted tests can only be written and executed by a Tester who knows that area well and kind of “lives there”.
If you like the idea, the first thing might be drawing a block diagram / architectural diagram of the project / product you are working on and show it to your Programmer friends.
Are you ready to start this journey?