Careers in Software Testing

Last week, I was invited by a group of “soon to be graduated” students of BS in Computer Sciences at FAST Islamabad campus. One of their subjects require them to host a talk by someone from Industry to complete the course. From the list of topics, I chose to speak about “Careers in Software Testing”.

Since on one of my previous tours, I had talked about taking a career in Software Testing, this time I focused more on what happens if you take that route. The main points that were laid down to the audience were:

  • Software Testing belongs to the Software Development as a whole. So careers in testing are similar to careers in Software Development.
  • The first few years of this career is establishing yourself as a technical resource.
  • Once you gain that position, you have two options. One is to become a people’s manager and start rising on the management ladders. The other is to remain a technical hub of knowledge and become the person who defines the technical path of the company.
  • The technical paths in Software Testing are various. The latest hot ones are Performance testers, Security or Penetration Testers and Automation Engineers.

career meter

Picture taken from

  • If the above looks too complicated or too much effort, try starting your own business and you’ll be CEO on the first day.
  • Presented Microsoft as a case study which offers jobs in Testing.
  • Covered how testing in Pakistan is growing and growing and growing.

The slides are shared here if you want to take a closer look into the above contents.

Did I miss something? What else we should tell graduating students about career paths in Software Testing?

Testing effort vs. coding effort

Usually people draw comparisons between testing effort and coding effort (which some also call Development effort whereas Development is the whole process of implementing requirements which includes testing). To use this as a trick question, I tried asking this in one of the recent interview session:

“For a given feature/user story, Programmer estimates that it will take 3 weeks to write code. How much time you think, you’ll need to test this feature?”

I got answers like:

  • At least one week of time. This is based on my experience.
  • I’ll ask for half the time as it takes at least 50% of the time to test.
  • To be on the safe side, I’ll ask for 60% time as my experience says it usually takes 50% time and then we may encounter more problems in a given feature.

Clearly the tester is thinking of testing effort is directly proportional to the coding effort. Even some people also have 10% unit testing tax formula or some suggesting law of diminishing returns on testing.

For me testing effort is not proportional of the coding effort and the answer is “it depends:). Let me explain with two experiences.


Few years ago, with introduction of Windows 64bit versions, one of the teams I know took the approach to let their applications run as 32bit applications in 64bit environment. The did minimal changes to announce that their software “supports” 64bit and then asked the testing team to confirm each and every features “works” as expected. Easily in this case the testing effort was way more than the coding effort given that the application was a huge one.

In another project, one of the teams was having performance issues in their system. They decided to flip the design scheme to take it from ‘attaching B to A’ to ‘create A and link with B’ and it took them couple of months effort to change all the code to get in line with this new design philosophy. Once done, testing team was asked to do performance benchmarking. As the UI layer never changed, the set of tests were executed without any issues and new numbers were reported in a few days. In this case testing effort was almost not notable as compared to the coding effort.

So when you are asked next time, “how much time it will take to test?” , you should first understand what the problem is you are asked to test and then negotiate a good number.

Do you ever developed a formula for testing efforts? And do you still want to use it?

The real success story of Panama Canal

It was 1881 when the French undertook the project of Panama canal with the ambition to join the Pacific and Atlantic oceans. The idea was there ever since this part of land was discovered by Vasco Núñez de Balboa (against whom name is the Panamian currency called Balboas) and even in 16th century, the Spanish wanted to construct a canal but they did not have the technical expertise needed for this at that time. To take part of this strategic path, a rail track was constructed by the Chinese in 1860s where ships were off loaded on one part of ~70KM land and ported through rail to the other side where another ship would take the load and keep moving.

The French seemed well prepared. Ferdinand de Lesseps who was the chief engineer of the famous Suez canal along with his team brought the most experienced workforce in this domain. The plan was prepared with all the geographic considerations and work started at a real good pace. But in 9 years of construction, the project was still any thing but on target with about 20,000 workers losing their lives. No not in the blasting or other construction related injuries or the venomous snakes hidden in the rainforest, but due to the yellow fever and malaria. Given the loss of workers, the morale of the rest of the workforce was really down. There were other factors also and the project was abandoned and it appeared that such a canal is not possible.

Panama Canal Extension 3

(Photo by me. Panama Canal expansion project. New set of locks on the Atlantic side)

Then the Americans took over this project in 1903 and used some part of the work done by the French earlier. They knew that the project didn’t complete due to the loss of workers rather than any technical grounds, so they first thought to solve that problem. There was a sanitation office setup with this special mission to eradicate yellow fever, malaria and similar diseases in this region. There was investment in extensive sanitation projects, including city water systems, fumigation of buildings, spraying of insect-breeding areas with oil and larvicide, installation of mosquito netting and window screens, and elimination of stagnant water. This did not completely eliminate the malaria like diseases but helped a lot in controlling it which resulted in the canal completed almost on scheduled and starting operating in 1914. It is now 101 years that canal is still operating and now an expansion of canal is underway that will double the passage of ships in a day which is estimated to open in 2016.

The above is a summary of the story of Panama Canal that I heard during my recent visit there. We enjoyed the beautiful beaches and diverse wildlife in the rainforest but the most talking goes around the Panama Canal (including the famous Palindrome “A man, a plan, a canal, Panama”). And using the ‘over-simplification logic’, I concluded that success of building the canal was not coming with a beautifully engineered design or well orchestrated work. Rather it was about supporting and protecting the workforce from diseases like Malaria so that they can focus on the work.

And then I looked around the modern day workplaces and the diseases that make them toxic. There are Malaria like attacks from distrust, dishonesty, lack of communication and not appreciating the lives of talented workers.

The workplaces that work on removing such diseases first rather than solving the technical problems, produces wonders like Panama Canal. Are you willing to do that?

Thoughts on Performance Testing

Every now and then, I’m been involved in discussion with fellow testers on the topic of Performance Testing. I realized that I never covered this on the blog. So here you go.

Note: There are many types of Performance Testing and below notes are related to checking time based performance of systems. Though these apply to other types also, but it’s always good to have a context.

Performance Tests don’t pass or fail

In a way that the result of a test is always a binary: either they pass or fail. Whereas Performance tests give you a number and only you have to decide if that number is reasonable enough to call that a Passing test. For example, you observe that the load time of your website’s home page on IE 10 is 4 seconds. Now is it good or bad performance is always a subjective discussion and unless you set a benchmark, the number looks very bad, bad, ok, good, very good etc.

There can be multiple benchmarks

So as mentioned above, the real question is that against what number we are comparing our test results. Those are called benchmarks.

The benchmarks can be a variety of things including:

  • Your competitor’s similar solution
  • Your last commercial release that is with the client
  • Your weekly/daily alpha release
  • A number that comes out from the person who is responsible for release.
  • A number that comes out of some Usability lab conducted with users.

It is also possible to record actual number and then represent it with compared with one or more or all benchmarks.


(the original photo is here:

Play around only one variable

Be mindful that performance measurements need to be done in ‘controlled environment’. If you are comparing a number coming from Machine A with Machine B, that is not a comparison as more than variables at play. So rest of the stuff will stay the same serving as baseline and two readings will be taken by changing one variable.

Again looking at the above example of load time of website, you’ll have to check it on the same machine with same memory resources, same set of other processes loaded and then compare the time. So if you are doing a comparison with benchmark, on same machine repeat the test with one or more benchmark. Now you have results that you can compare.

Take multiple measurements

Having one reading can be misleading and may not present the true picture. Depending on the task being measured, take multiple measurements like 5 or 10 or more and then take an average to get the real value.

Tools will help you

There are many tools that will help you measure time or memory. They also help you orchestrate your tests one after another and then provide you with set of numbers. Once you know what kind of testing you want to do, then picking a tool can be easy. For example to check for load time of website, this tool will be handy for Firefox.

Performance tests are in the 4th Quadrant of Agile Testing and are always accelerated by tools. But they are designed by knowledge testers like you.

Did I miss any thing on the general overview of performance testing? What things you’d add to this list?

Testers, stop saying these 3 things

No one can look inside you to find out how good you are at something. Everyone judges you from what you say and what you do and that is why if you want to build your repute as a Knowledge Tester, you should stop saying these 3 things.

  1. Where are the Specs

When I started learning the craft of software development in late 90’s I liked the quotes that said that “You can only walk on water if it is frozen. And you can only work on requirements if they are frozen”. (I couldn’t find the origin of this; can any one help?)

Fast forward many years and the constant change has made it impossible to freeze the requirements. Our users, their requirement, their priorities, their environments, the technology changes at such a fast pace that freezing them is against the laws of Physics.

So when you start on a new project or feature as a tester, never ask ‘where are the Specs’. Because chances are that either they don’t exist or they are outdated/ out of sync from how the software currently behaves.

I know what you are thinking now that how we can move forward if we don’t know what the software is supposed to do. The answer to that is “find out”. Yes, we can find them out through exploration and invoking Three Amigos conversations with Programmers and Customers.

And if someday you accidentally find the Specs, add them to your list of resources that reveal the ‘truth of expected behavior’. Don’t take them as the absolute resource for this purpose.


(the photo is taken from this article: )

  1. How should I test this?

We often get into this trap that as we engage other members of the team in testing, we ask their advice on how the systems should be tested. There are two problems associated with that kind of statement.

First, people can start doubting your skills by thinking that ‘we hired these guys to test and they ask us how to test. If we knew that, we could have tested this at our own’. The second problem is that you start following the thought process that produced the solution rather than thinking of new ways to look at the problem. Also at times, Programmers give their undone projects to testing team as soon as they are asked how to add more tests.

The right question to ask is: ‘What should I test?’ and then keep on extracting information, absorb that information and then come up with a plan so as how you’ll test it. There is no harm in saying that ‘this is how I plan to test, what thoughts do you have’ as in that case you’ll get answers driven by your test strategy and not following what others suggested you.

  1. I need more time to test

This is a killer statement and if you ask Project Managers about what they hate to hear from testers, this statement will be on the top of the list.

The thing is that testing never ends and it is impossible to find all the time to run all possible scenarios to find all the problems. In fact as you spend more time in testing, you figure out that you need more time for it. See how similar is this to spending time on vacation: you always need more time.

But the fact is that we don’t have time.

So, rather than looking at the equation from testing perspective, we need to look from the time perspective. Set clear rules that if you are given one month to test, this is what you’ll do. If you are given one week to test, this is what you’ll do. And if you are given just one hour, this is what you’ll test. On one of the release decisions that I’m part of, we are down to only 24 hours throughput time on testing. You tell us today you want to release it, within 24 hours you know ‘yes’ or ‘no’.

You might be thinking that I’m promoting automation but in fact I’m not. It is true that quick feedback from testing can be fast tracked through automation but defining clear priorities on what will be tested is what will get you there.

Do you also think that we should stop saying these things as Testers? If no, please let me know why. If yes, what else we should stop saying

State of Unit Testing in Pakistan – Survey results

The results from the survey are in and they are very encouraging.

It suggests that around 60% of the teams have Unit Testing practice in one way or other with around 20% say that they love Unit Testing. By the way, I really liked the Pie chart become in a nice shape and appears to be outcome of a testing report!


As always, the results are not complete until we analyze from where the data came. So here are the notes:

  • There were 40 respondents, so this is in no way true reflection of all Industry. I wish I could get more coverage as I did for the Tester Income. But you know people are more interested in that subject than Unit Testing.
  • Another quick insight into results is that overall survey hits are about double. So half the audience decided not to answer the survey. Either they didn’t want to look bad by saying “No, one does that” or they trusted the other replies.
  • And the result came from all places. 47% were Testers, 32% were Programmers and rest were all scattered. They featured CEOs, CTOs and Business people.
  • There were just a couple replies from outside Pakistan which were not included but I really appreciate the response.

So my friends, as I mentioned that our IT industry is maturing and let’s put our part in making Pakistan a place that produces Quality Software

What is your take away from this survey? And what other topics you would like our blog to survey?


State of Unit Testing in Pakistan – Survey

Pakistan IT industry is on the rise. Some are making way as a brand of their own where as many young ones are emerging. Even the theme of OPEN Islamabad’s Annual event for 2014 was ‘Celebrating Entrepreneurs”.

One sign of a maturing IT industry is that it becomes Quality conscious. Not in the terms of having a Quality Policy or having catchy phrases as company’s motto but following certain practices at the heart of it’s business. The heart of software producers is to “write code” and it only becomes better by taking lot of steps towards excellence. Writing unit tests is one of them.

That is the reason that in recent conversations with my fellows in Islamabad, I hear a lot about their journey towards adapting unit testing. But at times your echo chambers can lead you to a world based on illusions and not on reality. The best way to know the reality is to ask around and that is what Knowledge Tester is doing through this survey just like we found the reality on Tester’s income in Pakistan.

Would you mind filling in this 2 minute survey on the state of Unit Testing in Pakistan:

You can even fill it even if you are from elsewhere by mentioning your country name. And stay tuned for the results that follow in two week’s time.

And can you help me spread this word? Thanks in advance.


Get every new post delivered to your Inbox.

Join 990 other followers