Why Standards don’t help Software world

One of the unfortunate paths of wisdom is that if you have solved a problem in a particular way, the next time you see that problem, you will always apply the same solution. Since Engineering, or even better word would be Manufacturing, world is existing for about two centuries now and we have solved many problems of that world, we tend to use the same approach towards the Software world. Though it keeps on failing.

The most famous example of this failing application is the notion of “working hours as a measure of productivity”. As in a factory, all other factors including Machine, Material, Method etc. usually remain the same, the focus is put on the Man. So if the Factory Worker A spends 10 hours and Factory Worker B spends 6 hours, A is more productive. Peter Drucker spend almost his entire life dispelling this concept for the development work and introduced the term of Knowledge Worker (this is also an inspiration for my blog’s name as Knowledge Tester). Such that for Knowledge Worker, the measure of productivity is not the hours worked but the work produced in those hours. Alas, we still find managers and leaders in software world, who still live in the Factory Worker mentality.

Now let’s move to the standards problem. In my earlier life I was a Mechanical Engineer and then I moved to software quality domain. When I was there, I absolutely loved the Standards and when I was first introduced to ISO 9000 (a standard on quality management), I immediately followed it. It was year 1996, when as part of my final year project, I undertook a study of ISO 9000 implementation in the Pakistan Industry along with my friend and we thought ISO 9000 will solve all the problems. The manufacturing world is a world of repetition, so if you set the pattern right, you’ll almost always get a consistent result.

But this does not improve quality, this just improves consistency.

Take this instance. During the visits to different factory locations where ISO 9000 was successfully applied, one of the managers told us about a non-conformance their auditor reported: (a non-conformance is an incident where a practice is not in accordance with the standard or the plan)

“In one of our steps, we rinse empty bottles before refilling them. When the auditor visited the site, the bottles were being rinsed with water and soap. The auditor reported a non-conformity because the procedure stated that bottles should be washed with water. Using soap was a non-conformance”.

ISO fails in software world

Okay, so when I moved to software quality profession my love for ISO 9000 was still there. I along with the same friend, successfully achieved ISO 9000 certification for the software house we were working for and thought of it is a big achievement. But as we were implementing it and started analyzing what we actually gained on quality improvement front, we quickly realized that ISO 9000 is meant for Manufacturing (the repeating) world. Where as in our circumstances, all of our versions were unique and even within versions, the copies with different clients were customized to meet their needs. We, in software world, live in an imaginative world and every software delivery is a step into unknown. Standards like ISO 9000 don’t help in such kind of knowledge work.

So now when I heard of ISO 29119 which is being proposed as one standard for Software Testing, I know that it will not work. Testing is a piece in the Software Development process that is equally creative and imaginative and works like these are not supposed to be managed through standards.

I have signed the petition to stop ISO 29119. Even if they don’t stop this standard, I’m of the view that this standard will not improve the industry as ISO 9000 didn’t improve the software quality.

What are your views on applying standards in the software world?


Tags: , ,

10 responses to “Why Standards don’t help Software world”

  1. Smita says :

    Informative and true..


  2. samadwaqar says :

    every standard or model has one very important clause or practice i.e. measurement & analysis. the purpose of this clause/practice is to identify statistically that whatever you have implemented is it giving you any improvement in the system. the purpose of the standard is not to just define process and start following it and be satisfied that we have a defined and uniform process for everything. What you have defined and are following, it has to be measured by identifying KPIs. may be you are following a procedure religiously and everyone is happy about it but if you dont measure you wont find out that if that process is worth it w.r.t. cost.
    Quality is defined as customer satisfaction but this definition is for the customer. For a businessman or a manager the definition of quality is customer satisfaction in an affordable cost.


    • majd says :

      Thanks Samad for your comments. I can understand your stance that if we measure and watch closely, we can always improve process. But again measurement is a concept borrowed from Manufacturing world like ‘1200 pieces per hour’ and if we can make it 1300 pieces per hour, we are improving. When we try to apply the measurement to a creative process like software development or software testing, the metrics don’t help us much and we are left to take qualitative measures.


  3. Salman says :

    I differ to disagree on this point. The standards provide you the guideline as to how to do things right, in the right way. There are times, when wrong things are done in right way or vice versa. The standards are just there as a guideline. While I amuse on the incident of Soap and Water washing you have provided, real auditors looks for connections between missing gaps. For example, Project schedule slippage, and status reporting. Every project, be it a manufacturing or a Software has a schedule. Manufacturing schedule might be in hard copy, while the software might be in Soft. The changes brought in schedule changes would be quick to perform in software rather than manufacturing.

    In short, every product based software company could go for ISO based standard or they could go for CMMI standards, which is specifically for Software industry, yes they have astounding performance results if they implement the standard correctly. The key is to implement it correctly.

    To put it in simple terms, every thing that we do in our life is made of procedures, ranging from operating a Cell phone to driving a car or parking it. If we follow a standard practice, we would have less stress and better productivity is always guaranteed.


    • majd says :

      Thanks Salman for your comment and I respect your viewpoint 🙂

      So my view of the things say that CMMI may look better than ISO but…. it will not help either. I spoke on ISO 9000 implementation from my experience, do you have applied CMMI and have seen a success? Sharing that result will really help.

      The main plea that I have is that making repetitive processes like parking a car, always help and they are even being automated. But if we try to streamline creative processes like software design, development or testing, we are making it counter-productive actually.


  4. rumadak says :

    Agree with what you have mentioned. Even if it is not about ISO, there is insane race about getting certified which actually doesn’t guarantee you will be a better tester.
    BTW, I signed the petition too!


    • majd says :

      Thanks Ruma for the comment and signing the petition 🙂

      Ahh… I agree to your point and am not favor of any certifications. I do plan to write about that some day so as why.


  5. Salman says :

    Thanks Majd for being Responsive. The basic idea is Automate things which require boring tasks like repetitive Testing Tasks. For example, regression activities could be automated. Creative activities are never automated, but they are perfected in the sense, they estimated well.

    The main objective is any certification, be that ISO, CMMi, CMM is to streamline your own development and lowering down your own costs. Be that in automotive way, or a repetitive documented way. Plus these days, many markets are asking the Software house, whether or not they are certified to some level, because every software is stretched to the point of clients dis satisfaction. Certifications play an important role for clients sake as well.

    As James Bach says, Exploratory testing can never be automated, same is the case with Development, but then they should at least be estimated well and that’s what the ISO Certification says too.

    You are right about being counter productive. It will be if it implemented in wrong way. If implemented well, then Certifications do actually help, and yes I have seen it action. Usually processes do slow down a task, but it makes it more effective and less error prone. Again, the key is how that process is implemented and automated and which tasks are slowed down. The basic idea is…

    Repetitive tasks are automated.
    Judgmental tasks are kept on humans interaction, and even there is specified pattern, on a long historical scale, they could too, be automated by using rules.


Trackbacks / Pingbacks

  1. Testing Bits – 8/24/14 – 8/30/14 | Testing Curator Blog - September 1, 2014

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.