There are two type of testers; those who read and those who don’t. If you are reading this blog, I’m happy that you belong to the former and you have a 100% chance to improve yourself as a Tester.
The problem with knowledge is that as you know more, you realize that you know less. That realization forces you to know more from people who might know that you don’t know. That knowledge can be either gained by meeting those persons and if you don’t have that luxury, you can always read what they’ve written.
Thus began my journey as a tester and it is going on by reading more books. That is why when in the recent TOP (Testers of Pakistan) Meetup, I suggested testers to read at least one book this year as one of the main strategies to improve as a tester. Since then I have received some questions on what books testers should be reading and here come some suggestions.
Essentially a tester should be reading three type of books: 1) Books about software testing, 2) Books about the technology/domain they are working and 3) Any book that help them flourish their soft skills. As you can imagine that second category is very generic and it totally depends on the context. The context also changes with a new job or new role. Similarly third category also varies person to person, so as which characteristics one wish to shine/develop. So my take today is primarily on what testing books a tester should be reading.
And I have an offer. If you are based in Pakistan and have read / will read any of the following book and do in-person / on-call conversation with me about them, KnowlegeTester will issue the following “Certified Book Reading Tester” certificate to you. Yes, I mean it to make it sound good for you and me.
With all this long background (“Tamheed e Toolani” in Urdu), here are top three books that I suggest:
When I started my first job as a Tester, my manager gave me this book in the first week and asked to read till chapter 7. So did I and learnt a great deal about basic software testing concepts. It also introduced me to Cem Kaner and his work which really helped me grow as a tester.
Then when I started to work with my manager in the US, I was recommended to read this book – one lesson a time. And ever since I keep referring to this book on different subjects and always find some wisdom. For example a lesson in early chapters that relates software testers as headlights of the car (and not the brakes) helped me lot with defining the scope of my work. It also introduced me to the context driven testing and works of James Bach and Michael Bolton which always give some new and refreshing ideas to the testing world.
This book was discovered by me as part of our transition to Agile world and the struggle we’re having. I was hooked by the terms “Technology facing tests” vs. “Business facing tests” and used Agile Testing Quadrants, Automation testing pyramid and other models given in this book a thousand time in my discussion with testers and engineering teams. Though there is a newer book on the subject that I’m slowly reading but the first one is that every tester needs if they are part of a team that is building software in today’s changing world.
Do you recommend other books to the amazing “ready to learn” testers?
Pakistan’s testing community is finally rising up for the mission to “promote software quality” and this was evident from the Testers of Pakistan (TOP) meetup that happened in great way on Saturday January 9th at Zigron campus, I-9, Islamabad. A joint venture of Knowledge Tester, Pakistan Software Testing Board (PSTB) and Zigron hosted over 100 passionate testers representing about 50 organizations with the session comprising of 5 captivating talks.
This journey of testers meeting and sharing knowledge started with a small edition back in June 2014 with a small gathering of testers. The next edition was even bigger and recent one proved that we have reached to a point where we can expand this idea to a national level testing conference at Pakistan. More details later on this.
The session started by opening comments by Kashif Mueen who is the CTO at Zigron. Kashif welcomed the audience and expressed his opinion on how people might believe testing a second grade job whereas his experience tells it otherwise. He mentioned his trip to the Boeing facility and how mission critical is testing of each component over there. He then related that to the software of today which are being used in every walk of life and how a failure would be catastrophic. Thus he concluded that testing is an important domain and testers need to raise the bar to cope with all those challenges.
After the opening remarks, the first talk was given by Hassan Farooq who is part of the fabulous Zigron testing team. Hassan talked about BDD using Cucumber and started with few definitions of TDD (Test Driven Development) and how BDD (Behavior Driven Development) is different. He then went on to show an example test for the login feature of a standard website written in Gherkin and implemented in JUnit and results appearing in Jenkins . The audience poured in lot of questions which Hassan answered with even more details.
Up next was a description of “Aspects of Software Quality” by Sadaf Munir Minhas who is Director Quality at Keep Truckin (a Google venture). Sadaf is a familiar face for being a regular attendee of the meetups and explained how Software Quality has to be ensured at three levels and how the responsibility is not just on the testers but improving processes is part of the game. She touched upon ISO, CMMI and other standards that can help achieve those objectives.
The next talk was perhaps the most interesting of the day about “Automating Game Testing” by Dr. Uzair Khan who is a Model Based Testing researcher at QUEST lab and is also Director of PSTB. Dr. Uzair first explained how a game can be modeled into a state machine using UML and then how this can be flattened to give test cases and oracles. These test cases are then can be coded to play the game automatically and see if it behaves in required fashion. He also demoed a Java version of Mario, ran a sample test and proved how this idea is worth further investigation.
In between these talks was a filler by myself (which was added on popular demand :) ). My main thing was to appreciate the audience who joined us on a cold Saturday morning and then recapping the journey so far, I laid down the following three initiatives for year 2016:
- A national level Testing Conference somewhere in last quarter with a hope to invite even international speakers.
- Hosting such meetups at Lahore and Karachi. (It looks former will happen soon as we had official presence of Lahore based testing service provider company Kualitatem and few others who travelled ~300 KM to attend the session).
- Split the meetups at Islamabad to focus on certain areas e.g. Performance, Security etc.
I also suggested testers to read at least one technical book in 2016 and try something new to make this year, the best year in their life as a Tester.
The normal operation resumed after my talk and stage was taken over by Saad Khalid who is CEO of Mercurial Mind. Saad explained the latest trends in Mobile Applications and the IoT (Internet of Things) architecture. Using an example of an apparently simple looking app that he is developing, he went into details so as how complex the overall system is and how difficult it is to test it. He suggested that testers should have the vision of Softare Architect if they wish to be become the awesome tester everyone loves.
The fifth and final talk of the day was done by the VizTeck team represented by Junaid and Nauman. Those gentelemen talked about Automated Testing using node.js and showed implementation of the concepts using Mocha and Chai. They also cautioned that tools and frameworks like these make the job of writing tests easy, but that should not take focus on writing the “easy but unwanted” testes rather the intent is to make “needed” tests easy to be written.
Wrap up session was conducted by Dr. Zohaib Iqbal who also talked about the initiative of PSTB and being President what is the mission in his eyes. He encouraged more and more partners to join in to help produce sessions like these, and in general take the Pakistan’s testing community to the next level.
The lengthy but entertaining talks ended with distribution of shields for the Speakers and certificates for the Organizers. All then enjoyed lot of chit chat over a cup of tea which resulted in new connections established / old connections revised / ways to work together.
I must thank Amir Shahzad and all his Zigron testing team for taking all the pain to make the event a great success that it was. All the arrangements were seamless as if they are done by professional organizers.
More photos are here. Slides will be soon shared on our pages and also at the TOP pages.
This covers details of my talk at last month’s Agile Conference Pakistan 2015. In the previous post, the breadth of topics was covered and now I plan to cover one topic in depth and that is “Agile Testing”.
The talk started with the notion that “Agile is not the reason to transform the industry” rather that “Changing world has forced industry to adapt Agile”. I took my famous example from the game of cricket so as how the game has evolved from Test Cricket to One Day Internationals and Twenty20s. No doubt we are living in the time of T20s and we need to have a testing strategy can flexibly handle all changes happening internally and externally.
Then I touched the Agile Manifesto’s statement of “Responding to Change over Following a plan” as my main mission of the day. To add some fun, I copied text of this statement from Agile Manifesto page in Urdu, Arabic and Persian which resulted in a big laugh and was a perfect way to gain audience’s attention before the serious stuff began.
The three ideas that I presented as Agile Testing foundation where 1) Agile Testing Quadrants 2) Testing is a team responsibility and 3) Test automation pyramid. All of the concepts were based on the excellent Agile Testing books of Lisa Crispin and Janet Gregory and I advised the audience to read those books if you want to seriously learn Agile Testing.
In the Agile Testing Quadrants discussion, I used the following version of Quadrants and rather than putting focus on what should be in which Quadrant, I emphasized the point that this is a great way to gauge if your testing strategy is spread out. Such that if you see one of the Quadrant missing, that needs attention. But if you see more activity in a Quadrant compared to other, that might be okay depending upon what your project/product is.
I also gave away some exercise to be filled later which you can find Agile Testing and You.
Next topic was making Testing a responsibility of each team members. After explaining Three Amigos concept with some real life example situations, I presented following model to see how much of the testing is done by “Testers”, “Programmers” or “Client/Business People”. Again the idea is that if everyone chips in, that is the best formation to have.
Finally I explained Mike Cohn’s test automation pyramid to make the point that foundation of the pyramid should be at the Unit level not at the GUI level as most of the test automation implementer might think. Also the middle layer of testing the application at API layer is very important. If you are testing an application that cannot be tested at that level, remember:
A poor test-ability reflects a poor usability
To sum up the stuff, I used Growing Agile’s Testing manifesto and pin pointed the areas that were covered in the talk. At the end I encouraged all participant to practice the skills learned, adapt them to their needs and feel free to discuss with me if they face issues.
The whole slide deck along with other presentation at the Conference are shared here. To get a feeling of how the session went, you can check photos here and a video of the session will be soon posted on the Conference page.
Let’s apply the stuff we learned at the 2nd edition of the conference and gear up for next year’s event!
Yeah, you read the title right. It was a conference held in Islamabad – the capital of Pakistan on November 14th and it was the second edition. I feel really sorry that I missed the first one but am very happy that I became part this time and they have me forever now. It was a very well-orchestrated event which was attended by about 150 Professionals and representatives from the Academia. Below is a brief summary of the sessions and I do plan to right in detail about some of these topics.
The keynote speech was delivered by my friend and mentor Ather Imran on “Agile Culture”. Having worked in Software Industry for so long and currently being CEO of Sybrid who offers a variety of IT services, Ather was well aware of what an organizational culture is. He explained Agile transformation using the golden circle of Why, How and What. The pick of the talk was when he talked about Schneider culture model and suggested to evaluate your organization to see which culture they belong. Agile cultures are heavy on Collaboration and Cultivation, have some interest in Commitment and don’t like the Control one. Ather then also talked at length that after assessing your organizational culture, how you can mold it to Agile culture and what you need to do to achieve that.
Adeel Ali who is based in Texas, US and has his setup in Bahawalpur, Pakistan named ClickChain, talked about “Technical Excellence” in light of one of the Agile Principles. Being an Agile Excellence Coach, Adeel knew exactly why most of the Agile projects fail and he emphasized the need on improve coding quality and training the developers to accomplish that. He quoted that if you apply SCRUM as a management practice but you still have a messy code base, it will take you nowhere as issues will remain the same.
Next up was Suhail Iqbal who has done all the certifications in the world on Project Management and has a deep understanding of different processes and techniques. Suhail introduced the Lean principles with examples from Construction Industry and it’s similarities and differences with Agile. He also talked about how the Agile concepts are being applied to non-conventional sectors (by which he meant non-IT as Agile originated in the Software world). He also introduced LeAgile which is an effort to combine the both worlds.
The last session before the lunch was by me which was an introduction to Agile Testing. After establishing the fact that changes happen too often now a days, I suggested that having more tests is the only way that you can make changes at a high pace. I introduced the Agile Testing Quadrants, the concept of Three Amigs and emphasized that “Testing is role where every team member tests”. I also touched upon Automaton Pyramid and the importance of having a heavy base on Unit Tests.
After a delicious lunch which was actually an excuse to socialize and meet so many like-minded people, the session restarted with a talk on Emotional Intelligence by Mohsin Lodhi. Mohsin is a leadership trainer with experience much more than the average age of the audience that was attending the session. He in his trademark style explained the importance of Emotional Intelligence and how according to modern researches it is proven that EQ plays more role in the success than in the IQ. Mohsin introduced some personality prototypes and audience most liked Tina the Time bomb who’d explode any time. He then talked about how Self Awareness is the key to Social Awareness and Relationship Management and how can we achieve that. Mastering these will help a lot on Agile world where you have to work in teams and do a lot of communication and collaboration.
Barkan Saeed who is CEO of Vizteck and was one of the brains behind this conference took over the stage with a panel discussion on challenges of Agile adaption in Pakistan. Owais Anjum who is CEO of eMumba was representing OPEN Islamabad was joined with Kashif Mueen of Zigron and Nauman Faridi to talk about variety of topics including the challenge of Self-Organizing teams given that the general belief is that native culture is against that notion. The panel also highlighted the current boom in IT and how organizations in Pakistan can benefit from it.
As I mentioned that I do plan to write more about the above topics and maybe will start with a detailed post on my talk. Stay tuned and also keep an eye on Conference page for official coverage of event with photos and videos.
The success of the conference was largely due to the tireless efforts of the organizing committee with Naveed Ramzan being a notable mention. I am convinced that more success will follow.
Do register yourself with PADS (Pakistan Agile Development Society) so that you get in touch for any future events. Thanks and bye for now.
Knowledge Tester turned 3 last week!
This has been a very rewarding journey. As I talked about in the second and first year, this has been another year of success where reach of the blog has now reached 100+ countries with 1100+ individual followers and per post views have almost doubled this year. I can share more success stories along these lines but thought to also touch upon the challenges in the journey.
(The image is taken from: http://www.shapecollage.com/blog/shape-collage-3rd-anniversary-giveaway-loupe-updates-and-more )
When you start a mission, your enthusiasm is high in the start and that carries you for some time. But as the time passes you re-evaluate your priorities and at times struggle to continue your passion. This was a challenging year as a blogger for me and let me discuss the top most concern.
New stuff to write is difficult to find. When I started the blog, I had many topics on which I wrote my posts. The list started to dry out in the third year. And with so many other bloggers talking about the happenings in the testing world, writing about something new or write something new about an existing stuff became tough.
The thing that rescued me was that as you get a followers community of your blog, you hear interesting comments on and off the blog posts. I think about half of my posts in the third year came from those discussions. They were generated from the Activities or the follow up queries from those events.
So my plan is to focus more on such events in the next year which will mobilize more the Pakistan testing community and that will automatically become the supply line for ideas on my blog.
But as I mentioned above, more and more visitors are coming to the blog and highest referrer is now Google with my blog post on “Exploratory Testing vs. ad-hoc testing” going viral in the mid year.
Thanks are due to all the readers of the blog. A special thanks to Ruma Dak who wrote a guest post for my blog on feedback and allowed me to share a post about ANDing your life on her blog. Connections like these would never have been made without starting a blog.
With this, let us start another exciting year of blogging. What topics you like the most or want me to talk about?
When we are kids, we like to believe that world is a simple place.
Our teacher in playgroup or prep or nursery classes would tell us that “Sun rises from the East and sets in the West”. We like this simple concept because it is in accordance with our daily observation and more importantly it is an easy concept to digest for a kid of just a few years age.
As we grow, we are told that Sun is not rotating but it is our Earth that rotates which gives us an effect of Sunrise and Sunset. As we get into higher classes, we are told that Earth also rotates around it’s own orbit and Sun has many other planets. We are then told that Solar system is just one of the so many Galaxies in the space. We are told that we are living in an ever expanding Universe and we are told of something known as Black Holes and theories of Big Bang. We are told that we are not certain if we are expanding or moving to a certain point or we are just hanging around. This very complex nature of the Universe is simplified for us as kids to believe that world is easy: “Sun rises from the East and sets in the West”.
Similarly when we start our professional study or professional career, we like to believe that world is a simple place. There are some rules that if we follow, we can be successful in our business. If we add these magical lines in our CVs, we’ll get our dream jobs. If we network in these best ways, we can excel in anything we want to achieve. We tend to believe that there are some “best practices” that if we get to know and master them, we’ll be the champions in what we do.
This is as simplified version of the world as simplifying the Universe to Sun rises and Sun sets.
The only best practice is that there is no best practice
This Dilbert strip was taken from here: http://theleanthinker.com/2013/12/31/the-problem-with-best-practices/
What we forget when we are in search of best practices is that world is a complex place. Each individual has it’s own strengths and weaknesses, each of us operate in a different region of the world in contrasting cultures, each of us have different values and different meanings of the outcome. What works for me may not work for you. What we need to do is to understand the situation, apply some practices, evaluate the results and keep tweaking them.
You may be tricked to say that see here is the best practice:
Understand the situation, apply some practices, evaluate the results and keep tweaking them
If you are thinking in this way, this is due to the reason that we don’t want to leave our childhood and we love stories and we love simplified versions of the world. The above is not a best practice.
What are your thoughts on best practices?
Having no tests means you have a challenge with the Quality of your solution. Having thousands of tests means you have challenges of test management.
Yes, like testers and development teams, tests have to be managed.
So often we find claims on different tools’ websites that “we have thousands of tests running all the time to ensure quality” but let me today take you inside the life of that world.
As I talked about this problem of putting some tests to “Ignored list” to make the build passing all the time in my post about Continuous Integration, when tests reach a big number the number of Ignored tests also increase and a campaign has to be launched regularly to clear them. The tests in this list come due to three reasons:
- Tests are failing because test code is bad
- Tests are failing because production code is bad
- Tests are fine; they just take too long or have other dependency (data/hardware) that prevent them to be run on build servers.
For the bad test code, a public shaming (a.k.a. peer pressure) email that list down authors with biggest number of ignored tests help. Keep on sending those emails as the some people are more “shame-proof” than others.
For the bad production code, make sure the issue is reported in bug tracking system and elevate it’s status. Do mention to the team that if we fix them, we get two benefits: a) Bug count reduces and b) test count increases.
For the third category, some strategy is needed to run those tests after the build on regular basis. For example, we ignore all Performance tests from the build as they take too long. Then we run them each week and compare them with benchmarks to see if results are good. Bad results are then shared with the team.
Tests that you think run but they don’t
The machinery to include all tests in your repository is a piece of code and it can have bugs too. When you first write and commit your tests, you assume that they run each time yet they don’t.
In one of the audits that I did on my project recently revealed that about 10% of the tests were “not in the system”. The build files had some commented portion that happened due to some reason and then stayed there. There were some assumptions about the folder and files which were not being honored by all tests after some changes. Bringing them back in the system gave us some additional coverage right away.
What do tests do?
The quality of tests remains questionable as unless you see what each test does, you can’t say how good your tests are. But there is always the first step of doing code coverage.
Having thousands of tests doesn’t mean that you have good coverage and all methods are being tested. In the audit that I mentioned above, we found out that our overall LOC (lines of code) coverage was still 50%. Another thing that we found was that:
Module A has 500+ tests with LOC coverage 70%
Module B has 500+ tests with LOC coverage 40%
So just looking at number of tests, you can’t guess the coverage. A note here is that coverage is also not a perfect measure of quality but as I mentioned it is a good start until you can review each test.
Do you have thousands of tests in your project? What kind of challenges you have to deal with?