Archive | Agile Testing RSS for this section

Tester in the valley

Summers offer exploration options. And exploration is always good for Testers.

Couple of weeks ago, I spent few days in the picturesque valley of Neelum in Azad Jammu Kashmir. The main attraction was to enjoy the beautiful scenes and flowing waters across the way. But being Tester for life, I had a second mission: find some bugs.

We did not have to wait long.

At the entry point where Tourist information was recorded as the road touches the Line of Control (a loose border between Pakistan and India for the disputed Kashmir region), we were given a brochure from the Tourist department. It was very helpful as it had distances, important phone numbers and places of interest. But my 12 yo daughter (who has accompanied me on previous bug hunting summer excursions) spotted an issue.

Sorry for this being in Urdu but it supposed to mean the “Meeting point of Neelum and Jehlum rivers”. The funny part was that by taking the ‘mmm’ sound off the second river, it meant “Meeting point of Neelum reiver and Stupidity”. Yes we stopped there to make that meet happen.

On our way beyond this meeting point was a long travel along the Neelum river. There is a lesson of Context in the name of the river. The same river is called “Kishan Ganga” by our Indian neighbors. The funny part is that at certain points, the River itself is the border. For example in the above picture, I’d be calling the river as “Neelum” and people across the river would be calling it “Kishan Ganga”. We can have a long discussion on what is the “correct” name of the river but reality is that it is one river with two names. How familiar is this situation when we have a word fight on certain terminology and that’s why understanding each other’s point of view matters a lot. For example in building a good relationship between Developers and Testers.

English being 3rd language in our region with every valley/region having it’s own language and then Urdu as national language, there are so many spelling mistakes to be caught in the journey. For example a hotel had numbered it’s multi-story building as following:

As you can imagine that with so much greenery out there, the possibility of finding the real bugs was very high. Some of them were unique in their textures, some were disguised and some were shouting out loud to be caught:

And yes your guess is right. Where there are bugs, there are bug hunters. Some of them were also ready for a photo-shoot:

There were many other spelling mistake bugs which I caught but I’m leaving them as they were level 0 or low severity bugs 😊. But there was one thing for which I couldn’t find a bug: the beautiful landscape that had all in one picture like the one below from Arrang Kel:

And oh yeah, before I go I’d like to mention a life philosophy lesson that I found on one of Rickshaws in Muzaffarbad which read:

Reading “Saans hay to Chance hay” meaning that if you are breathing, you have a chance to do something. We get bogged down by stressing office environment, project fatigues, overwork but remember how worthy it is still being alive. Because

Saans hay to Chance hay

Have you explored any new area in your Summers? What did you find worthy of sharing?

Automation cannot be Generalized

Consider the automatic machines you have at home as an example. You have a Washing machine to do a laundry on any day that you want. And then you have Dish washing machine that can clean your crockery at your will.

Aha… someone would say, here is some common stuff. You see both of them are machines that do “washing”, so why not we generalize it and make it one machine that will be able to “wash” either clothes or plates. They both use water and some sort of “detergent” to cleanse the items we put in it; they both use rinsing as a way to take off all the dirt etc.; and they both use air to dry the items once the washing is done.

If someone approaches you with the above plan, you should immediately recognize that you are being sold a “Phakki”. This can be an excellent plot of any piece of fiction from cartoons to movies but it has nothing to do with the real world. So if someone sells you a Generalized Test Automation solution, your response should be similar to the above plan.

(Charlie Chaplin in “Modern Times” movie. Full clip is here)

The common stuff in the case of washing machine can be components that you have in both machines, like an electric motor, water drainage system, air blowers and other similar pieces. But the function that each machine performs stays unique and thus requires a unique design and implementation.

So when you approach automation of your testing activities, you can have some common components that you use in different solutions, but each solution will have it’s unique needs. For example, over last couple of years we have written many scripts to automate the following testing activities:

  • Smoke Testing
  • Performance Testing
  • Data Conversion Testing
  • Compatibility Testing
  • Cross Platform Testing
  • Memory Leak Testing

All these scripts are specialized as they target one type of testing at a time. The common pieces are just the sorts of “Get the build to test”, “Gather the results” and “Notify about results”. The actual code that automates Performance Testing is a different beast than the one we have for Compatibility Testing.

We do have a lean type of Test Automation Framework. For other cases, there can be a framework that takes a General test automation tool as a starting point and delivers a specific set of specialized implementation for your product. In one of the talks at our regular meetups, my tester friend Adeel Shoukat gave an automation tip that was roughly “You’ll need to write lot of your own test routines using tools as you cannot use the tools right off the shelf to test your application”.


If someone tells you that here is a Generic Test Automation suite that will solve all your testing problems, you are being goofed.

Stay away from those pocket pickers.

How has been your journey towards automation? Have you ever seen or implemented a Generalized automation solution by any chance?

Testing is shifting Left

With the DevOps movement and the notion of packaging your Software as a Service (SaaS), many traditional concepts are being shaken up. Whether you like it or not but the truth is that in today’s ever changing and globalized market, the old ways to develop software are not working.

Look at facebook for example. Have you ever seen a notice like this?

“Services at facebook will remain closed from this date to that date (or for these hours) for maintenance purpose. You’ll not be able to access your accounts during this time. We regret the inconvenience caused by this.”

Obviously not because that is so much 1990’s way of upgrading an Internet based service. Facebook instead follows the mechanism of “changing the wheels while car is moving” rather I’d say “changing the engine while the jet is flying”. Yes, facebook has a policy of deploying new changes daily since long and you don’t even know when it was updated with newer features. All this has only been possible through lots of changes in the way software is developed, deployed and maintained.

One of the key features in such situations is to shift Testing to the Left. There are other significant pieces too which I’ll try to talk in some future posts but let’s now focus on Shift Left of Testing.

These pages will help you get the concept in a much better way and I’d recommend reading them. If you are one who like to get someone read articles for you and give you a nice summary, I’m here to help you with this: “Traditionally Testing is done after a feature is developed. Shift Left refers to shifting testing to left (think of classical waterfall model where things moved from left to right in phases), such that it becomes an integral part of Developing the feature”.

(The image is taken from here.)

This concept is not now. Agile Testing introduced the principle of Testing being done by all team members with Power of Three or Three Amigos. Lot of talk has been on testing early in the process for long. But in all such suggestions, the onus of testing remains on the Tester in the team and others come in as helpers. Like when Testing team says that we need to Test early (kind of let’s shift testing to the left), what they usually mean is that a Tester should become part of the design discussions and should start writing/executing tests as early as possible. But here

We want Testing to happen early regardless of who does it.

It’s kind of funny that for years we were told that Developers (though I prefer the term Programmer but writing it Developer to refer to that point of view) are superior than Testers and Testing itself is a B grade job that should go to such workers. Now all that inferior work is suddenly becoming such an important task that it will be done by the Developer.

What does it mean for you if you are a Programmer?

Yes, you got it right. As a Programmer, you should be writing tests as you write the feature. As Matt Brigs states in this post:

A senior developer will understand that this job is to provide solutions to problems, not write code.

So become a Senior Developer all of sudden by realizing this. Write tests and run them even before you commit changes to ensure that solution is being written, not the code.

What does it mean for you if you are a Tester?

You need to let go of the mentality that “Testing is something that Testers do”. Rather you should help the team do testing by improving the Testing machinery or the tooling needed by the team. Spend more time in helping Programmers write more tests. Make systems that help run Continuous Testing as part of each step as Dan Ashby mentions that we test even more in DevOps. Remember that Rob Lambert  shared a lesson during his company’s transition to  weekly releases from yearly releases:

Testing as an activity, has to become central to the team.

In our team, we have started this shift recently and I hope to write more on the lessons learned in coming weeks and months. Wish us luck!

Have your team joined Shift Lift movement yet? What are your observations so far?

Agile Conference Pakistan 2016 – Takeaways

One of the sources of knowledge is to know the sources of knowledge. Each society has sub-societies in it and if you are not there, you’d think that this never happens in our city. This exactly is the case for me but I was lucky to become part of Pakistan Agile Development Society last year through the 2nd edition of Agile Conference Pakistan. That allowed me to learn more throughout the year and the best part came in the last weekend when along with a wonderful team of organizers, speakers, sponsors, volunteers and audience I enjoyed the Agile Conference Pakistan 2016  which was attended by ~250 Agile practitioners.

Below is a very very short summary on what happened there and more will be shared on Society’s facebook page and website. The first set of photos is out and soon the slides and recordings will also be shared.

Society’s chairman Suhail Iqbal said in opening note that it is a very happy trend where more and more people are joining the Agile conference. He also mentioned that though Agile movement arrived a little late in the Pakistan IT industry but we can work together to make it happen in a great way.


The first talk was “Agile Transformation vs. Agile Adaption” by Syed Ahmed who is CEO of DPL-IT and ex-chairman of P@SHA.  Syed started the talk with difference between Process-centric companies and People-centric companies and rejected the notion that one process can change the world rather it is the human potential that changes the world. With some examples he explained that the yardstick of whether a company is process or people-centric is the answer to this question: “Do you allow people to make mistakes?” His view was that Agile Adoption is like doing the rituals whereas a Transformation will come when you really have belief in something. After establishing these facts, he presented the case study of his own company DPL-IT where they have gone through an Agile Transformation. He shared details on how they are promoting a “Rebel ethos” culture and how they have a “Sir-carr” which fines you for a certain money if you call people as “Sir” rather than with their names. He also suggested a flipped hierarchy as below (this is drawn by me and not the actual slide):


In the last few slides, he had some lessons learned which included a “Let Go” thinking in top management’s mind, making things transparent and having a belief that the results will start coming though there will be some rough patches.

The next talk was about some suggested recommendations on Managing Agile Transformation by Javad Ahmed who is COO of a US based company Oracular (Smart-IS in Pakistan). Javad gave his viewpoint on what Agile is, why we need it, and how he has implemented it in his own scope. Given his consultancy background, through various examples he shared the practical implications of doing an organization wide Agile transformation. He had lot of good advice to offer for bringing in the “business perspective” in making all decisions and below are some quotes that I noted:

  • Nothing can be done without Top management’s commitment and they’ll only buy-in if they see business value.
  • Agile is focused on “Right Results” and an “On-target” outcome.
  • Agile is an ideology.
  • Agile facilitates prioritization.
  • Do not short-change staffing e.g. asking a Scrum Master to play as Product Owner too.
  • Counselling and Coaching is very important in the Agile journey.
  • Too many times the documentation is like a solved Pakistan Studies papers that happen in our education system where marks are given based on how lengthy and colorful is the answer.

Then Afif Zaid and Mujeeb Zahoor shared the story of Agile success at S&P Global’s Pakistan office. Mujeeb walked through a brief history of the overall organization and the stuff that happens in Islamabad. Afif then provided inside scoop of how they worked through this Agile Transformation phase and how they reached to the milestone of having 57 successful SCRUM teams across the globe. His tips for others to do the same were passing on the vision, help team on setting the priorities and making teams self-organized and empowered. One of the many experiments they are doing is playing a “Shark Tank” to allow everyone to throw ideas. Mujeeb then concluded the talk by sharing the business success that Agility has brought to them.

OPEN Islamabad’s President and CEO of Sybrid, Ather Imran was the first speaker after lunch to talk about “Agile Culture and Knowledge Organizations in Pakistan. Ather referred to Peter Drucker’s work on what a Knowledge Worker is and what is a Knowledge Organization. Then after briefly going through the Agile principles, he concluded that:

Knowledge Organization  = Agile

He then brought in some interesting thoughts on the Power Psychology and how we tend to be followers by default. Through examples including the South Korean Airline experience from Malcolm Gladwell’s book Outlier, Ather made it clear that our culture is where people defer to the Power and want to be obedient to the authority. He then shared a few tips to break that with consistent efforts which included 10 Rules that he explained one by one. For example he wants to “Delegate to the level of Anarchy” because if you are delegating and are comfortable with that, there is still control in your hands. So getting uncomfortable with delegation is the key.

The next talk was by the leadership training maestro Mohsin Lodhi who talked about “Leadership in the Digital World”. Mohsin started the talk with establishing the fact that how technology is eating many jobs and many industries are now squeezed to hand held devices. Through his typical style of backing statements with numbers, he shared many new researches that enforces the reality that we are living in a fast paced and non-linear changing world. He connected the dots that Results are very important but one should never forget that the results come through processes and processes are due to a culture and creating a culture is the job of the leader. (The below is my notes and not actual slide)


He then through some moving and dancing slides, shared the lessons from the famous social experiment done with 5 monkeys  and suggested not to react to the shower of culture around us by saying that “that’s how it has always been”. At the end he also explained the Change Equation and how leaders should work on the buy-in and show through their actions if they are serious about bringing any change.

An Agile Coach at VizTek and an Agile lover Nabeel Ansar then talked about “Avoiding common mistakes in Agile Transformations”. Nabeel started the talk with when and how Agile fails and emphasized on the conditions in which you apply Agile to be more responsible for the failure. He gave an interesting example of driving a Ferrari on the IJP road and then blaming the road for the poor performance of the car. Through his personal experience of Agile Transformation in few organizations, he recommended to first “show something by doing it” and then ask for others to help. He also shared 8 Agile Transformation Patterns which included the likes of “Start small” and “Seed and Grow”. The point that I most liked in the talk was Nabeel’s insistence on improving the Engineering practices because a fast delivery is only possible if you have the automation, testing and other build processes geared towards such delivery.

A short closing note was delivered by Farid Zafar, Associate Professor at Bahria University Islamabad thanking the audience for sparing their precious weekend day for the love of Agile.

Believe me my notepad had much more than the above and I’m sure that was the case with all the participants. Let’s keep this journey towards excellence together and I hope to share some after thoughts soon on some of the subjects listed here!

Generalist, Specialist, T and Pi shaped Testers

“All testers are special but some are more special than others”. To borrow a line from Animal Farm with some variation.

In last month’s testers meetup, one of the discussion points was “Should testers be Generalist or Specialist?” As the discussion was happening, I decided to write a blog post on this subject as it is close to my heart. It took me almost a month to do that which is bad given the fast paced world we live in but not as bad if compared with the whole span of life.

I thought I’ll start with an overview of what we mean by Specialist and Generalist etc. but my draft was in progress when I saw this excellent post at Testastic blog which I think lays a good foundation for the readers. Also as I was researching this topic, I found this great webinar by Derk-Jan de Grood and Jan Jaap Cannegieter done for EuroSTAR which introduces T and Pi shaped very well. My advice would be to leave my blog here and first go to the above links.

Thanks to the sincere readers of the blogs for consulting the references and welcome back. To all other casual readerrs, T shaped professional is a person who has a depth of knowledge on a Specialty but has enough breadth of knowledge on the General subject. A typical example that I give is from medical profession i.e. an ENT specialist while recommending a medicine needs to be aware if the patient has heart disease or is pregnant to avoid any side effects. A Pi shaped professional is then the one who has two special legs meaning they are specialists on two subjects. See the picture below that I copied from the webinar slides that I mentioned above.


(Slide from webinar: )

With that background, let’s move to the topic now.

First and foremost, a Tester is already a T shaped professional. If not, tester should be. Testers are members of the team who should have enough knowledge of different functions of the teams like Development, Deployment, Build process, Documentation, Release management etc. yet their specialty lies in Testing. The teams are aware of that and they usually respect that by leaving the testing job to the Testing Specialists.

Some people gets confused with the cross-functional aspect of Agile teams and say that all members test so how come a Tester is special. But good teams know that having a Specialist tester help. In one of the teams that I joined last year, the project lead said to me:

Majd, I don’t want you to spend most of the time writing tests. I’d like to spend most of your time to see how we are doing on the testing front.

So if you are a Tester, you are already a Specialist on testing and you should rather be a T shaped professional by learning the other functions of the team.

The question then comes to developing the other specialty or second leg of a Pi shaped testers. In the webinar mentioned above, when the presenters did few rounds of workshops with testers they found out that the second leg can be Programming, Management, Automation etc. All of those are good but in my opinion for a Tester, knowledge of other fields like Programming or Management actually makes them a better T shaped Tester by expanding breadth of knowledge. A Pi shaped tester’s other leg should also be in testing.

I don’t want to force my opinion and you are free to pick second legs other than a Testing area. But if you take my advice, I’d suggest the other legs could be Performance testing, Security Testing, Usability Testing, Automated Testing etc. By that I mean that you should be subject matter specialist on the subject and not just a Generalist on all. If you want to get started, here is a piece from tester friend Rizwan Jafri that came out this week about automation.

Are you a T shaped or Pi shaped tester? If not, at what stage you are. And if yes, what are the second leg options you’d like to share with us.

Learning from Errors

My Mama used to say when I was a kid that “Don’t worry. Your mistake is not a mistake, if you learn from it”

In his book “Errors: bugs, boo-boos and blunders”, Jerry Weinberg talks the subject of errors at length and it was a delightful reading. As a software practitioner, we all see errors all the time. But if one wants to take the errors seriously, this book is a must read. To boost your appetite, let me share a few lessons that I learned from the book:

Errors are not a Moral thing

Yes, when we get to know that we were responsible for an error, we feel guilty. Some of it comes from our school time when we were told that making error is a moral thing and the one who does is a bad person. Unfortunately many Managers live in the school teacher mentality where they want to embarrass team members publicly when they make mistakes. Jerry emphasizes this and suggests to get out of this mind block to accept that all of us make errors. This should not result in doing nothing by saying “hey, what we can do as errors are a natural thing” rather this should result in focusing on what we can do to limit the error and how they can be stopped in future. In Jerry’s words:

To err is human; to combat error is software engineering.

Errors have a link with the management

When we study errors and causes of errors, we often focus on the Person who made the error. If we go beyond, we look at what was fed to that Person and how it was done. In terminology, it can be translated as how the requirements were defined, how the Programmer understood the requirements and how the Programmer implemented the requirements. All too often, we forget that the environment in which all these things happen and how it contributes to the errors made. And good management is all about create environment to reduce such errors.

Jerry has listed eight Fs of software failure and how management can work on them. But he also said right there:

But good management is so boring!

Errors will always exist

You might be thinking that you are a software engineering champion and you are a management champion, so you know the best of software processes that you’ll apply in best of your management style such that errors will never occur. That’s the pitfall for each Perfectionist and Jerry suggests that we need to live with the errors. He lists 8 Laws of Errors Defense to help us out and one of them is:

One thing worse than error is an error farm

As a tester, you might have seen many error farms where errors are produced in abundance. When we report errors, we should be looking for error farms and reporting them too.

How do you see errors? What lessons you have learned from them?

Testers, mind your business

Testers have many jobs at hand. They need to run as many as possible tests to find out useful information about the health of solution to help make decision makers take decisions. Anyone who has experience in testing knows that the real job is not to “run” tests but “design” tests. Similarly the quantity of tests that were executed doesn’t matter if they are not hitting the right target. That is why number of failing vs. passing test cases, number of new defects reported are a poor way to represent quality and hence a poor feed of information to the decision makers.

What matters most is perhaps what the test does and why it is important? In one of my previous posts, I suggested that testers become gray box testers to get some ideas on coming up with some serious tests and hence bugs. Today I want to bring the focus on the business side of the things.

Testing in a way serves a bridge between Business and Technology. You can say that this is true for the entire “engineering” team but perhaps testers are the main guys responsible for this.


(Picture from with edits by me)

Given that the domain you serve will vary from organization to organization and from team to team, it is tough to lay down specific guidelines so as how you can understand the domain. A software serving to help design Nuclear Plants may require it’s testers to learn about all the standards of materials, quality and health/safety. Whereas a software serving the education sector may require it’s testers to understand how brains are educated and what quality education means and what are the experiments done in this area so far.

Therefore I’ll not mention the ‘how’ part of the problem rather focus on ‘why’ it is important for testers to learn more about the domain as the “Golden Circle” suggests that ‘why’ is more important than ‘what’ and ‘how’.

  1. You talk the language of your users

Very early in my career, I was visiting a client with a team to understand the requirements of a solution that we were building for their pharmaceutical setup. Client was represented by 3-4 managers and they were throwing a lot of terms. I don’t remember the exact term of one of the reports that they wanted but one of our team members said the name of the report wrong. Manager politely corrected. When our team member made the same mistake in the conversation, the client said:

“How come you make a solution for us, when you don’t even name the things we need?”

I learned a lesson that day and always prefer to use the terminology that our users would use. That makes all the internal communication that you do as tester become more “user focused” and you know “what user wants will always be fixed”.

  1. You augment the team with “Business facing tests”

Keeping the Agile Quadrants in your mind, you know that your team specially the Programmers are writing most of the technology facing tests, so if you dedicate your efforts towards the tests that mimic the business cases that’s like a perfect combination.

  1. You know what drives what

Pieces when combined become the solution. So as you move your “vision of the tree” to the “vision of the forest”, you start gaining more understanding of why a particular piece is needed to solve a particular problem. This helps you write tests that are like end-to-end or workflow tests (the tests that achieve a particular user workflow). Often such tests results in bugs that surely increase your reputation as a tester who knows how the system works.

How much of your time as tester goes into understanding the domain? And do you list more benefits of this investment?