Unit testing, what?

On my mission to promote quality, I’m visiting one place after another and talking to one group of people to another. As I talk about Software Quality as a shared responsibility, the concept of “Three Amigos”, Agile testing quadrants, and more; one thing is common: “No one wants to get out of the comfort zone”.

For example, when I suggest Testers to learn the technology they are testing and write some code to automate repetitive testing and testing process, they shy away from writing code. Similarly when I ask Programmers to take some responsibility and start writing Unit testing and try approaches like TDD, the expressions on their faces are not very encouraging. There are many discussions out there and I’m adding some concerns that I recently heard and my attempts to answer them:

“I’m on a project that is in production for multiple years. The code base has matured enough through rounds of testing that adding unit tests won’t add value”

The decision to write unit tests is not based on how many years the code base has been in production but on the basis of how many years the code base will stay in production. If you write some tests today for the bugs found so far in the system, the next deliveries will remain safe from those bugs. Remember the time, when you changed code to fix a bug and another bug poked it’s nose from some other place. If you have tests for them, you have that “safety net” that allows to do modifications to the code with more liberty.

“Well, I’ve tried TDD on our newest project but it seems to be a Productivity killer. I’m afraid that TDD doesn’t work well on green field projects”

In fact Test Driven Development is more suited for green field projects.

My experience being a Tester, the receiving end of Quality is that:

The bugs found in projects with TDD/Unit testing is about half compared to Projects with no Unit tests.

The reason you are seeing some productivity loss is that you are seeing the investment in writing tests as a repetitive investment where as it is a diminishing investment. As the time passes by and you run your set of tests again and again, the time needed to maintain existing tests/ add new tests will be lesser. And more importantly the “unknown” time that will be added to the project when testing team finds bugs (not covered in unit tests) is actually being planned into your work automatically.

Let’s take the following as example:

Project A incorporates TDD practice and this is how the time is spent in Coding vs. Testing+Debugging where blue is coding and yellow is the other part. Where as Project B doesn’t incorporate TDD and this is how it’s time is spent over a period of four months. These are hypothetical times but from experience, I can tell that the Yellow time remains mostly smaller in projects like Project A compared to the likes of Project B.

Unit Testing vs No Unit Testing

“So you are saying that as a Programmer, I write tests for my own code. So if I write tests to verify my understanding of the system, how those tests will add value?”

Right and you’ll be surprised of how your understanding starts improving as you write tests. Actually when you write tests, you more and more think like “how the user will use this feature?” rather than “how should I design the feature?” This brings in added checks and improved functionality.

What concerns do you hear and how do you answer them?

Tester Diaries – Applying SFDPOT Heuristic

(This is a guest post by Ridha*)

Welcome back to the Tester diaries. As you know that I am a student of BS in Software Engineering and learning software testing in interesting ways.

Now a days, I am working on my Final Year Project (FYP) and I wanted to design its test cases in parallel with development. Through the Knowledge Tester’s guidance, I came to know about SFDPOT and I was suggested to use this thought provoking heuristic for designing tests. As we all know heuristics not always find the best solution but these are some tricks to make our work easy.

SFDPOT letters stand for Structure, Function, Data, Platform, Operations and Time.  Read in detail about SFDPOT at: http://www.satisfice.com/articles/sfdpo.shtml and have a look at one example here: http://www.slideshare.net/karennjohnson/kn-johnson-2012-heuristics-mnemonics . (Note: the latest version of this heuristic is now SFDiPOT where i stands for Interface)

SFDPOT helped me to organize my understanding of testing my FYP in form of a mind map. My FYP name is CureJunction and I have plans to make it live one day at http://www.CureJunction.com . This will be a website that helps us find doctors with their specialty in our locality and then let us book online appointment with them. Many more functionalities will be provided at this website and I categorized them in the mind map that is shown below. It took me four sheets of paper and a multiple of four hours to come up with the final version :)


Do let me know how it is helpful for you? And how I can make it even better?

(Ridha Malik is becoming a Knowledge Tester while completing her BS in Software Engineering from University of Engineering & Technology, Taxila, Pakistan. As she explores the field, she is sharing her experiences as Tester Diaries and this is second edition of her thoughts. The first one was about Exploratory Testing.)

Recapping 2014

The WordPress.com stats helper monkeys prepared a 2014 annual report for this blog. I’m very thankful to all my visitors and dedicate this report to them!

Here’s an excerpt:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 14,000 times in 2014. If it were a concert at Sydney Opera House, it would take about 5 sold-out performances for that many people to see it.

Click here to see the complete report.

Testing Mobile Apps

Many predictions are going on so as what will be the top technology trends for 2015 , but surely one of the top trends of 2014 was (further) boom of Mobile Apps. We now hear Apple’s catchy phrase in almost every conversation “there is an App for that”. And one of the questions is how do we test these apps and I presented some thoughts about it a few weeks ago. Here is a summary.


(picture curtsey of HubSpot free images: http://offers.hubspot.com/free-stock-photos )

Given that the development life cycle of a mobile apps is any thing from 1 week to few months, it looks like that testing should be easy as there is less development effort. Also when we think that an App is limited in functionality compared to Applications, we somehow start believing that testing effort will also be limited as less features are there. In contrast, as elaborated by the matrix below taken from uTests’ excellent ebook on the subject, the complexity of the matrix is actually higher:

Problems like these brings in the value of testing the apps locally. Remember how Apple Maps App failed in areas where it didn’t work. And yesterday a friend of mine visiting from US showed that his weather app shows the location as Grand Trunk road while he was in a city on that few hundred miles long road.

The other point is Performance testing that either keeps a potential customer away from the app or forces them to reinstall app afterwards. The more important number is not the number of downloads but number of installed apps, so your apps should be sticky. Focused security testing will always be needed.

Since it was a short talk, I added some spice at the end by presenting some interesting ways to test which were taken from different sources but mainly from “Tap into mobile testing” book. These included trying things like Gesture Frenzy, or acting like a kid, or performing an All Hands testing day when all members of the team test the app. It is also very interesting to do Usability Testing through Personas  where you can classify your user base into different set of personalities and then pick some representatives to be acted while testing.

The whole presentation is here: Testing Mobile Apps

What tips do you have for mobile app testers?

Agile Testing and more

I was very happy to see ‘More Agile Testing’ book by the authors of ‘AgileTesting’ . Just finished ordering that and thought to share the learning experience with the first book.

When I got hold of the book, I was into my 3rd or 4th year of practicing SCRUM which is Agile and I had a firm belief that I am doing Agile Testing. But Lisa and Janet changed my mind by stating that “Testing on Agile Project is not Agile Testing”. Let me give an example that I came to explain this.

For example I am a Pakistani and embody all the cultural values that a usual Pakistani  would have. Now if a person starts living in Pakistan and spends a good number of years and then claims that she or he is Pakistani, would we believe? We’ll only believe if that person embodies all the values that would make a Pakistani, not simply by spending some good years here. Similarly Agile Testing is a mindset, a culture.


So that was my first lesson and then I spent good enough time on different ideas shared in the book. My best three are:

The Power of three (or Three Amigos) concept. In almost all of my talks with students or professionals, I mention this notion of collaborating with all. It was glad to see that in slides talking about the new book, it is mentioned that Testers can pair with Testers, coders, business people, analyst, customers and so on. It is kind of taking the collaboration to the next level.

The next concept was breaking testing efforts in concept of Agile Testing Quadrants. I found it useful that when I work on a project, I lay down this chart for me and see if we are covering all the risks. Michael Bolton has recently talked about improving this model and I’m sure Lisa and Janet would talk more about the testing quadrants in the new book.

The third and last takeaway from the book was the notion of embracing change. The Agile practices were not introduced for fun but they were introduced to be able to deliver quality in a changing world. More and more testers understand how to work in changing project needs, more and more they and the projects will be successful. Exploratory Testing helps a lot in situations like these.

So I am very excited to read the new book as it arrives and will talk more about it soon. Have you read the first book and what are your favorites in that?

PS: By the way, the complete list of Key Success Factors from the Agile Testing book are:

  • Look at the Big Picture
  • Use the Whole-team approach
  • Adapt an Agile Testing mindset
  • Collaborate with Customers
  • Automate Regression Testing
  • Provide and Obtain Feedback
  • Build a Foundation of Core Agile Practices:
    • Continuous Integration
    • Test Environments
    • Manage Technical Debt
    • Working Incremently
    • Coding and Testing are part of one Process
    • Synergy between Practices

API testing perspective

Being in the business of testing APIs for more than five years, I often get questions on how it is done and how it is different than other types of testing. Let me explain some basics in this post and some other day, we’ll talk in-depth.

The user in API world is Programmer. The first thing that makes API testing so special is that the consumers of API are 3rd party or the fellow programmers in your team or organization. Now testers are User’s agents and they should exactly do what a user would do. So the tester in API world would be a Programmer that writes tests, samples, widgets, apps, stuff using the delivered API.

That is why in some teams, one of the Programmers takes this role of tester. Or if you prefer to hire a tester (as we do here), you should look for people with strong programming skills who still want to be testers.

API testing perspective

It’s all automated, by default. Since API testing is not called “Automated API testing”, people may assume that part of it is manual and part automatic as happens in usual test projects. But nay, every thing is automatic by default and there are lots of tools to help you out. So the process of building the test code, running the test code and shipping a healthy build is and should be done automatically. Even best is to incorporate every thing through Continuous Integration. The Unit testing tools along with home grown test infrastructures are the hallmarks of an API testing project.

It has to be a joint venture. Though ideally Programmers, Designers, Architects, Business people should take interest in every testing project but what happens is that the ownership of a Testing project remains with the Testing team in most of the cases. It changes in the API testing world, where Programmers take a good part in designing and coding the test infrastructure and writing unit tests. The Testers then add value by writing more complex scenario tests.

Technology really matters. If you are testing a Desktop API or Web API or Mobile API, the scene changes altogether. Similarly your tools and approach towards API testing will be largely driven by Technology. For example, if GoogleTest is used to test C++ APIs, you might use NUnit for .Net APIs and so on. Normally, the language used to write the API is used to test the API again because the Programmers will use the same language e.g. C++ API will be used in C++ applications vs. REST API being used in Web applications.

Maintenance is the challenge as with all coding projects. Needless to say the biggest pain for any coding project is maintenance. People come and go and API features come and go. The API test code needs to be maintained all the time to reflect the current state of the underlying API it tests. Any techniques used to keep your code base maintainable are applicable for API testing projects.

Have you been part of API testing teams? And what are your main takeaways for others?

Two years of blogging

Knowledge Tester blog turned 2 last week!

As I wrote about my experience as a blogger in the first year, I mentioned that the momentum keeps on building. The second year experience is more rewarding and exciting than the first one. Let me share some of these moments.

First, if the first year was a year of contribution then the second year was certainly a year of collaboration. About one third of blog posts were either guest posts or they were about tester meet up or tester survey. One of the regular contributors Huma Hamid has now started her own blog and I’m so happy to have facilitated that process. Another regular contributor Arslan Ali along with other testing friends including Faiza Yousuf is conducting regular sessions on software testing via Outtabox. Through these blog pieces, I made many new testing friends which I could have never made without Knowledge Tester platform. Such is the power of having a platform.


(the original photo is at: http://www.teemagroup.com/newsletter/collaboration-technology-needs-collaborative-humans/ )

Second, the reach kept on expanding. I mentioned blog being visited from 53 countries last year and now this number has reached to 90+. Testing is really a global community. And the number of hits keep on increasing. Though last year it was on a scale of 1:8 which is difficult to maintain, yet there was almost a double increase in the traffic. More importantly, about one third of the traffic is coming from search engines which again means that blog is being hit by people who didn’t know me personally. Now I hope they do.

Third, I am now being approached for more coaching/training sessions than ever. One of the recent examples was a person visiting Pakistan from the UK and needing a quick start on software testing. Along with the other trainer Sohail Sarwar, we spent one fine evening at a coffee shop talking about software testing. That person is looking in a very good shape to become a successful tester. And another example is Skype coaching sessions for a Software Engineering student who is also sharing her experiences about this learning.

Finally, the best moment of the year was when my blog on software testing standards got featured on International Society for Software Testing site and I was so happy to see my name listed along the gurus of the field.

Before I start the third year, I have lot of thanks due to all of you: those who read it regularly and share their thoughts here, those who just read, those who came to the Knowledge Tester events, those who wanted to come, those who keep on supporting me on the mission of making this world a better place through Knowledge Testers.

I am reviewing the popular topics and would focus more on what matters more for you. Do you have any suggestions for me to what to write about?


Get every new post delivered to your Inbox.

Join 770 other followers