The best answers to “what is the job of testing?” always have a key component suggesting that it is “to provide information”.
Testers job is to provide information for the product under test
They provide information about product under test, to expose risks for the team.
Now if you have been provided information any time in your recent past, you can easily recall that some of it was valuable and some of it was not. Even some information which was provided to you was false, misleading, hiding the facts or inflicted with similar diseases. So how do we, the testers, can provide information which is always always valuable to the person who is receiving information?
I believe the answer is in two parts: a) How do we collect information? and b) How do we process and present the information?
May be I am bit vague above, so let me spend some time to explain myself.
To provide information, you need to have all the information available to you collected from different sources like test results, design documents, release schedules etc. And then you need to process this information for the given need like the occasion, the person or timing of the information being provided. Like providing specific filters to view information from a certain angle.
Let’s dig a bit deeper.
How do we collect information?
Some testers are overly relying on only the results of testing. They don’t like to discuss with Development about how those tests are designed or how the results are interpreted rather they believe that results of testing are the only source of truth.
Then there are others who love documents. All of the information they have is primarily coming from a document e.g. Design document, Project Plan, Release Plan, or some sort of change tracking system like TFS. For them, those documents are the only source of truth.
And there are the third group who like verbal communication. They discuss each and every type of test in daily SCRUM calls and get feedback on Iteration Review results. They discuss release plans with the management rather than looking at what the document suggested at the start. They are of the view that people involved in the team are the only source of truth.
Not many people know that if all the information you have is coming from a single source, you don’t have a complete view. Consider political arguments that happen in your work environment between people who watch/follow a particular news channel.
So if as a Tester, you want to improve your skill as a better information provider, the first step is to widen your input sources. Test Results, Documents, People all matter. And even there are other sources which you know better than me.
How do we process and present information?
So when you are asked “How was your weekend?”, your answer largely depends on who is asking and in what circumstances. Your answer will vary in details and emotions for your jogging partner compared to your colleague; your best friend compared to someone you are meeting for the first time at a business meeting.
Similarly when you are asked “How is the testing going?” or “Are we release ready?”, if you are giving the same answer to every person in every possible settings, it shows that you are not filtering the information you are having and passing it on as it is. Apply different views to the same information given the context and your answers will contain valuable information for the receivers.
What tips you have to improve information gathering and processing steps?
KnowledgeTester turned 5 few days ago!
And as I discussed the journey in first, second, third and fourth year, my first happiness is to see that it is still going. I am, just like many of you, type of a person who “has a habit of starting projects and not completing them”. Though the project of blogging never finishes but continuing it for 5 years certainly can be used as a yardstick to call it mature.
(the original image is here: http://learnchineseinyunnan.com/news/news-wire/2015/08/20/2015-the-fifth-anniversary-of-thanks-giving/ )
The biggest achievement of fifth year is not a personal one but a collective one. And I sincerely hope you know that collective success has more savor than a personal one. And that was the first ever testing conference in Pakistan that we saw happening in April 2017 in a great way. My report on it “Pakistan Software Quality Conference 2017” has been the most viewed post of the year and I’m so happy to have achieved this through all friends specially the support of Pakistan Software Testing Board.
Interestingly that was my top most priority when I recapped fourth year. Now you can easily guess that this year’s priority is to make PSQC a regular thing in our Industry and we are working for PSQC’18 which will happen in February 2018 in Lahore Insha Allah (God Willing). Though we went into a momentary post conference enjoyment mode, where we made little progress in recent months but we are now getting back with two meetups coming soon in the remaining months of 2017.
On the viewership increase, I am seeing a 20% increase in per post views which may not seem great compared to previous years. But you know as the baseline increases, it becomes hard to keep increasing at same pace e.g. think how salary raise in % is tougher as your salary gets higher. But net effect is that views per post are almost at 1,000 now (967 to be precise) in last year. Yes, I can’t believe that this viewership is for the stuff that I’m writing but it’s true 😊
During the last two years or so, due to the community building activities KnowledgeTester was low on providing training to Testers. The good news is that I’m setting this as my second goal for the sixth year i.e. to hold Training along with continuing community activities. Wish me luck.
That’s about it for now and I’m thankful to each of you who contributed towards this success. And I’m sure you’ll be with me in next years of this journey.
Do you have any suggestions on what kind of training we should be offering this year?
Every plan is perfect until the execution happens. As a losing team’s coach once said: “We had a perfect plan on the paper. The only problem is that the match was played on the ground and not on the paper”.
So should we stop striving for better planning? Of course not. Rather we should learn from each execution so that our next plan is better than the previous ones. Taking a hint from this thinking, let me share 3 tips based upon execution failures to improve your test planning.
I have selected the tips based upon 3 layers of what I’m calling Agile test planning. Such that as a Tester, you are required to do “Release Planning” or asked to do an “Iteration Planning” or asked to do an “Individual Tester Planning”.
Let’s take an example of “Release Planning” first.
Many years ago my Employer decided to start offering solutions in the Mobile world. Before that we were only in the Desktop world. Our first solution was for Android and then we worked on a Platform that can produce Apps in Android, iOS or Windows. A year or so after this has happened, I was asked to do some test planning for a release meant for a Client in China. I ‘assumed’ that they’ll need mobile solution as it was our top priority then. So my plan started with Android and then iOS testing on Simulators and Devices.
The plan looked great. Our first iteration started and by that time, I had no interaction with the Client. Client started reporting a lot of bugs and all of them were reported in Windows Desktop environment. I learned that this Client is only interested in good old Windows Desktop solution and doesn’t care about mobile.
The lesson which I learned is my tip # 1 for this blog:
Always ask question rather than assuming things
Let’s now move to “Iteration Planning”.
In a project that I was part as the lead tester, the Product Owner complained that the number of new bugs reported towards the end of Iteration is far greater than the ones reported at the start. My first response was that as more things get ready for testing towards the end, more testing is done and hence more defects were found.
Then I gave it a deep thought by looking at the history of tests we were executed. One pattern was there that certain test plans produced higher number of defects (following the heuristic that Software Bugs are like Real bugs) and the test execution order was left on the tester to pick. Tester knew that this test plan will consume lot of time and energy, so they left these to the end.
The rule that we made afterwards is my tip # 2 for this blog:
Prioritize test execution based upon which tests will fail first
This article can help you get started on Risk Based Testing.
Let’s look at the last one called “Individual tester planning”.
When I was young and raw, I had very ambitious plans for myself assuming that I’ll get all things in my favor. I planned it 8 hours a day with no distractions. Eventually I couldn’t always finish the stuff that I committed at the start of Iteration.
With age and lot of experiences, I have become more realistic about my estimates and always consider it as a 6 hours day. I also account for unknown distractions like network outage, random meeting popping up, sick days and what not. It’s not that I give a pessimistic by including all those factors but I do keep margin for some of them.
The lesson that I learned is my tip # 3 for this blog:
A plan that doesn’t incorporate change is not a plan
What tips you can offer from your execution failures to improve test planning?
Summers offer exploration options. And exploration is always good for Testers.
Couple of weeks ago, I spent few days in the picturesque valley of Neelum in Azad Jammu Kashmir. The main attraction was to enjoy the beautiful scenes and flowing waters across the way. But being Tester for life, I had a second mission: find some bugs.
We did not have to wait long.
At the entry point where Tourist information was recorded as the road touches the Line of Control (a loose border between Pakistan and India for the disputed Kashmir region), we were given a brochure from the Tourist department. It was very helpful as it had distances, important phone numbers and places of interest. But my 12 yo daughter (who has accompanied me on previous bug hunting summer excursions) spotted an issue.
Sorry for this being in Urdu but it supposed to mean the “Meeting point of Neelum and Jehlum rivers”. The funny part was that by taking the ‘mmm’ sound off the second river, it meant “Meeting point of Neelum reiver and Stupidity”. Yes we stopped there to make that meet happen.
On our way beyond this meeting point was a long travel along the Neelum river. There is a lesson of Context in the name of the river. The same river is called “Kishan Ganga” by our Indian neighbors. The funny part is that at certain points, the River itself is the border. For example in the above picture, I’d be calling the river as “Neelum” and people across the river would be calling it “Kishan Ganga”. We can have a long discussion on what is the “correct” name of the river but reality is that it is one river with two names. How familiar is this situation when we have a word fight on certain terminology and that’s why understanding each other’s point of view matters a lot. For example in building a good relationship between Developers and Testers.
English being 3rd language in our region with every valley/region having it’s own language and then Urdu as national language, there are so many spelling mistakes to be caught in the journey. For example a hotel had numbered it’s multi-story building as following:
As you can imagine that with so much greenery out there, the possibility of finding the real bugs was very high. Some of them were unique in their textures, some were disguised and some were shouting out loud to be caught:
And yes your guess is right. Where there are bugs, there are bug hunters. Some of them were also ready for a photo-shoot:
There were many other spelling mistake bugs which I caught but I’m leaving them as they were level 0 or low severity bugs 😊. But there was one thing for which I couldn’t find a bug: the beautiful landscape that had all in one picture like the one below from Arrang Kel:
And oh yeah, before I go I’d like to mention a life philosophy lesson that I found on one of Rickshaws in Muzaffarbad which read:
Reading “Saans hay to Chance hay” meaning that if you are breathing, you have a chance to do something. We get bogged down by stressing office environment, project fatigues, overwork but remember how worthy it is still being alive. Because
Saans hay to Chance hay
Have you explored any new area in your Summers? What did you find worthy of sharing?
Consider the automatic machines you have at home as an example. You have a Washing machine to do a laundry on any day that you want. And then you have Dish washing machine that can clean your crockery at your will.
Aha… someone would say, here is some common stuff. You see both of them are machines that do “washing”, so why not we generalize it and make it one machine that will be able to “wash” either clothes or plates. They both use water and some sort of “detergent” to cleanse the items we put in it; they both use rinsing as a way to take off all the dirt etc.; and they both use air to dry the items once the washing is done.
If someone approaches you with the above plan, you should immediately recognize that you are being sold a “Phakki”. This can be an excellent plot of any piece of fiction from cartoons to movies but it has nothing to do with the real world. So if someone sells you a Generalized Test Automation solution, your response should be similar to the above plan.
(Charlie Chaplin in “Modern Times” movie. Full clip is here)
The common stuff in the case of washing machine can be components that you have in both machines, like an electric motor, water drainage system, air blowers and other similar pieces. But the function that each machine performs stays unique and thus requires a unique design and implementation.
So when you approach automation of your testing activities, you can have some common components that you use in different solutions, but each solution will have it’s unique needs. For example, over last couple of years we have written many scripts to automate the following testing activities:
- Smoke Testing
- Performance Testing
- Data Conversion Testing
- Compatibility Testing
- Cross Platform Testing
- Memory Leak Testing
All these scripts are specialized as they target one type of testing at a time. The common pieces are just the sorts of “Get the build to test”, “Gather the results” and “Notify about results”. The actual code that automates Performance Testing is a different beast than the one we have for Compatibility Testing.
We do have a lean type of Test Automation Framework. For other cases, there can be a framework that takes a General test automation tool as a starting point and delivers a specific set of specialized implementation for your product. In one of the talks at our regular meetups, my tester friend Adeel Shoukat gave an automation tip that was roughly “You’ll need to write lot of your own test routines using tools as you cannot use the tools right off the shelf to test your application”.
If someone tells you that here is a Generic Test Automation suite that will solve all your testing problems, you are being goofed.
Stay away from those pocket pickers.
How has been your journey towards automation? Have you ever seen or implemented a Generalized automation solution by any chance?
With the DevOps movement and the notion of packaging your Software as a Service (SaaS), many traditional concepts are being shaken up. Whether you like it or not but the truth is that in today’s ever changing and globalized market, the old ways to develop software are not working.
Look at facebook for example. Have you ever seen a notice like this?
“Services at facebook will remain closed from this date to that date (or for these hours) for maintenance purpose. You’ll not be able to access your accounts during this time. We regret the inconvenience caused by this.”
Obviously not because that is so much 1990’s way of upgrading an Internet based service. Facebook instead follows the mechanism of “changing the wheels while car is moving” rather I’d say “changing the engine while the jet is flying”. Yes, facebook has a policy of deploying new changes daily since long and you don’t even know when it was updated with newer features. All this has only been possible through lots of changes in the way software is developed, deployed and maintained.
One of the key features in such situations is to shift Testing to the Left. There are other significant pieces too which I’ll try to talk in some future posts but let’s now focus on Shift Left of Testing.
These pages will help you get the concept in a much better way and I’d recommend reading them. If you are one who like to get someone read articles for you and give you a nice summary, I’m here to help you with this: “Traditionally Testing is done after a feature is developed. Shift Left refers to shifting testing to left (think of classical waterfall model where things moved from left to right in phases), such that it becomes an integral part of Developing the feature”.
(The image is taken from here.)
This concept is not now. Agile Testing introduced the principle of Testing being done by all team members with Power of Three or Three Amigos. Lot of talk has been on testing early in the process for long. But in all such suggestions, the onus of testing remains on the Tester in the team and others come in as helpers. Like when Testing team says that we need to Test early (kind of let’s shift testing to the left), what they usually mean is that a Tester should become part of the design discussions and should start writing/executing tests as early as possible. But here
We want Testing to happen early regardless of who does it.
It’s kind of funny that for years we were told that Developers (though I prefer the term Programmer but writing it Developer to refer to that point of view) are superior than Testers and Testing itself is a B grade job that should go to such workers. Now all that inferior work is suddenly becoming such an important task that it will be done by the Developer.
What does it mean for you if you are a Programmer?
A senior developer will understand that this job is to provide solutions to problems, not write code.
So become a Senior Developer all of sudden by realizing this. Write tests and run them even before you commit changes to ensure that solution is being written, not the code.
What does it mean for you if you are a Tester?
You need to let go of the mentality that “Testing is something that Testers do”. Rather you should help the team do testing by improving the Testing machinery or the tooling needed by the team. Spend more time in helping Programmers write more tests. Make systems that help run Continuous Testing as part of each step as Dan Ashby mentions that we test even more in DevOps. Remember that Rob Lambert shared a lesson during his company’s transition to weekly releases from yearly releases:
Testing as an activity, has to become central to the team.
In our team, we have started this shift recently and I hope to write more on the lessons learned in coming weeks and months. Wish us luck!
Have your team joined Shift Lift movement yet? What are your observations so far?
One of the sources of knowledge is to know the sources of knowledge. Each society has sub-societies in it and if you are not there, you’d think that this never happens in our city. This exactly is the case for me but I was lucky to become part of Pakistan Agile Development Society last year through the 2nd edition of Agile Conference Pakistan. That allowed me to learn more throughout the year and the best part came in the last weekend when along with a wonderful team of organizers, speakers, sponsors, volunteers and audience I enjoyed the Agile Conference Pakistan 2016 which was attended by ~250 Agile practitioners.
Below is a very very short summary on what happened there and more will be shared on Society’s facebook page and website. The first set of photos is out and soon the slides and recordings will also be shared.
Society’s chairman Suhail Iqbal said in opening note that it is a very happy trend where more and more people are joining the Agile conference. He also mentioned that though Agile movement arrived a little late in the Pakistan IT industry but we can work together to make it happen in a great way.
The first talk was “Agile Transformation vs. Agile Adaption” by Syed Ahmed who is CEO of DPL-IT and ex-chairman of P@SHA. Syed started the talk with difference between Process-centric companies and People-centric companies and rejected the notion that one process can change the world rather it is the human potential that changes the world. With some examples he explained that the yardstick of whether a company is process or people-centric is the answer to this question: “Do you allow people to make mistakes?” His view was that Agile Adoption is like doing the rituals whereas a Transformation will come when you really have belief in something. After establishing these facts, he presented the case study of his own company DPL-IT where they have gone through an Agile Transformation. He shared details on how they are promoting a “Rebel ethos” culture and how they have a “Sir-carr” which fines you for a certain money if you call people as “Sir” rather than with their names. He also suggested a flipped hierarchy as below (this is drawn by me and not the actual slide):
In the last few slides, he had some lessons learned which included a “Let Go” thinking in top management’s mind, making things transparent and having a belief that the results will start coming though there will be some rough patches.
The next talk was about some suggested recommendations on Managing Agile Transformation by Javad Ahmed who is COO of a US based company Oracular (Smart-IS in Pakistan). Javad gave his viewpoint on what Agile is, why we need it, and how he has implemented it in his own scope. Given his consultancy background, through various examples he shared the practical implications of doing an organization wide Agile transformation. He had lot of good advice to offer for bringing in the “business perspective” in making all decisions and below are some quotes that I noted:
- Nothing can be done without Top management’s commitment and they’ll only buy-in if they see business value.
- Agile is focused on “Right Results” and an “On-target” outcome.
- Agile is an ideology.
- Agile facilitates prioritization.
- Do not short-change staffing e.g. asking a Scrum Master to play as Product Owner too.
- Counselling and Coaching is very important in the Agile journey.
- Too many times the documentation is like a solved Pakistan Studies papers that happen in our education system where marks are given based on how lengthy and colorful is the answer.
Then Afif Zaid and Mujeeb Zahoor shared the story of Agile success at S&P Global’s Pakistan office. Mujeeb walked through a brief history of the overall organization and the stuff that happens in Islamabad. Afif then provided inside scoop of how they worked through this Agile Transformation phase and how they reached to the milestone of having 57 successful SCRUM teams across the globe. His tips for others to do the same were passing on the vision, help team on setting the priorities and making teams self-organized and empowered. One of the many experiments they are doing is playing a “Shark Tank” to allow everyone to throw ideas. Mujeeb then concluded the talk by sharing the business success that Agility has brought to them.
OPEN Islamabad’s President and CEO of Sybrid, Ather Imran was the first speaker after lunch to talk about “Agile Culture and Knowledge Organizations in Pakistan. Ather referred to Peter Drucker’s work on what a Knowledge Worker is and what is a Knowledge Organization. Then after briefly going through the Agile principles, he concluded that:
Knowledge Organization = Agile
He then brought in some interesting thoughts on the Power Psychology and how we tend to be followers by default. Through examples including the South Korean Airline experience from Malcolm Gladwell’s book Outlier, Ather made it clear that our culture is where people defer to the Power and want to be obedient to the authority. He then shared a few tips to break that with consistent efforts which included 10 Rules that he explained one by one. For example he wants to “Delegate to the level of Anarchy” because if you are delegating and are comfortable with that, there is still control in your hands. So getting uncomfortable with delegation is the key.
The next talk was by the leadership training maestro Mohsin Lodhi who talked about “Leadership in the Digital World”. Mohsin started the talk with establishing the fact that how technology is eating many jobs and many industries are now squeezed to hand held devices. Through his typical style of backing statements with numbers, he shared many new researches that enforces the reality that we are living in a fast paced and non-linear changing world. He connected the dots that Results are very important but one should never forget that the results come through processes and processes are due to a culture and creating a culture is the job of the leader. (The below is my notes and not actual slide)
He then through some moving and dancing slides, shared the lessons from the famous social experiment done with 5 monkeys and suggested not to react to the shower of culture around us by saying that “that’s how it has always been”. At the end he also explained the Change Equation and how leaders should work on the buy-in and show through their actions if they are serious about bringing any change.
An Agile Coach at VizTek and an Agile lover Nabeel Ansar then talked about “Avoiding common mistakes in Agile Transformations”. Nabeel started the talk with when and how Agile fails and emphasized on the conditions in which you apply Agile to be more responsible for the failure. He gave an interesting example of driving a Ferrari on the IJP road and then blaming the road for the poor performance of the car. Through his personal experience of Agile Transformation in few organizations, he recommended to first “show something by doing it” and then ask for others to help. He also shared 8 Agile Transformation Patterns which included the likes of “Start small” and “Seed and Grow”. The point that I most liked in the talk was Nabeel’s insistence on improving the Engineering practices because a fast delivery is only possible if you have the automation, testing and other build processes geared towards such delivery.
A short closing note was delivered by Farid Zafar, Associate Professor at Bahria University Islamabad thanking the audience for sparing their precious weekend day for the love of Agile.
Believe me my notepad had much more than the above and I’m sure that was the case with all the participants. Let’s keep this journey towards excellence together and I hope to share some after thoughts soon on some of the subjects listed here!