Lahore Qalandars may have been eliminated from PSL but the Lahore Testers have definitely proceeded to the next round of Software Testing excellence by gathering for the 3rd Meetup this weekend.
The venue this time was the beautiful FAST-NUCES auditorium and event was sponsored by Systems Ltd. through the platform of Pakistan Software Testing Board. That makes a really good combination where Industry and Academia are working together towards building the community. A special thanks to Kamran Khan who made it happen.
The event started with refreshing remarks by our host Ali Gohar. And quickly we were onto the first technical talk on the topic of “IoT Testing” by Khubaib ur Rehman from CTO247. Khubaib after explaining what we mean by Internet of Things and how complex is the world, talked briefly on different types of testing that are important in this domain. He expanded couple of testing types like Compatibility in terms of hardware variation and Performance Testing being dealing with the big data that comes in through those devices. Audience got engaged in the discussion and Ali Iqbal who is part of ICT R&D fund commented on the IoT Excellence Center being setup by ICT R&D. The slides are here: internet-of-things-testing
(more photos are here)
Khurram Bhatti who is based in Sweden and took advantage of this visit to get engaged with the Pakistani Testing community, then graced the stage and talked about “The real potential of Software Tester in Agile”. Through entertaining GIFs and some witty statements, Khurram set the tone on why testing takes the center stage in Agile world. He then shared his vision of there things that a Tester must work on to be successful in agile: Communication, Business and Technical. Khurram gave a nice example on how engine oil can be assumed as a Communication mechanism such that if the oil is not there, engine will not run log. The presentation is shared here: khurrambhatti_real-potential-of-tester-in-agile-world
Systems Ltd.’s representative Nauman Farooqui who is currently serving as VP, Project Delivery and Customer Satisfaction was invited to share his thoughts on “Producing Quality Software”. Nauman took this opportunity to share that he started his career as Software Tester and encouraged the Testers in audience to step up the game. He quickly recapped the illustrious ~40 years of journey of Systems Ltd. which is proudly the first Software House of Pakistan. The thing that I liked most on his suggestions to producing Software with top notch Quality was that the Testers should absolutely learn the Business domain they are serving. He also gave some recent examples on how talking to the Clients on regular basis results in more chances of success.
The final technical talk was on “Lessons Learned in Test Automation” by Adeel Shoukat from Contour Software. In his typical relaxed yet always to the point way, Adeel debunked some of the Automation myths and made it clear that “Record and Replay will never work”. He also strongly held his view that Automation will not replace the manual testing thus thinking of Automation as a way to reduce testing force is not a way to go. He then shared some examples from his own experience that building a framework based on a hybrid approach to solve your problem is a good way to adapt. He suggested to start simple and show the business value that Automation can do and show the value to the management that Automation adds. His slides are here: leasson_learned_in_automation
Dr. Zohaib Iqbal, President PSTB was invited to share all needed information for the upcoming Pakistan Software Quality Conference being held at Marriott hotel, Islamabad on April 8th. Dr. Zohaib recapped the whole Quality journey of these meetups that happened in Islamabad, Lahore and Karachi. And this demands for a climax like the upcoming Conference. Dr. Zohaib offered a special discounted offer to testers from Lahore and explained the “Call for Speakers” and “Sponsorship model”. More details about the Conference can be found on the website.
The best part of the event then happened. Yes the yummy food with tea but No, the delicious talks over cup of tea were more interesting. Testers were seen talking to the Speakers and hosts of the event to ask the questions which couldn’t be asked during the session; Testers were seen seeking guidance on their technical and people management issues; Testers were seen discussing ideas on how we can make these events more successful. These are just a few highlights of the over one hour talks that happened and I am still feeling awesome about them.
That’s all from the Lahore meetup and we are looking forward to see all of these brilliant testers join us at the Conference!
The date was very close to the day when people show love for their loved ones. And the wind was somewhat “Basanty”. Yet the Testers from the twin cities showed love for their profession and enjoyed some high flying testing ideas instead at the Testers Meetup that happened this weekend.
The event was hosted at the beautiful auditorium of Bahria University and was sponsored by BroadPeak Technologies . Pakistan Software Testing Board (PSTB) provided the platform where 120 Testers representing over 40 organizations joined us.
(more pictures coming soon on facebook event)
After a quick introduction by the host, the first talk was delivered by Nabeel Ansar from VizTeck . Nabeel through his personal experience shared the vision of “End to End Test Automation” where automation starts from when the Requirements are written. He showed examples written in Gherkin and implemented in specflow to demonstrate the concept. The talk resulted in lot of questions being asked for this approach and also on Automation in general which were very well answered by Nabeel. The presentation is here: end-to-end-test-automation
The next talk was on an interesting topic of “Data Science and Software Testing” by Uridah Sami from RedBuffer (also known for her blog on testing QualityFrenzy ). Uridah set the scene by explaining what Data Science is, how companies like facebook or Netflix use this concept and how products like self driving cars rely on data. She then mapped the skills needed for Data Science and Software Testing and explained how these two worlds are connected. Read more here: data-sciences-and-software-testing
The third and final Technical talk was delivered by Amir Shahzad and Hasan Farooq of Stella Technology Inc. . In first half of the presentation titled “API testing using Spock Framework”, Amir briefly touched upon modern day web architecture and the importance of testing the backend RESTful APIs. He then explained why they prefer Spock over other frameworks which was mainly for it’s ability to be used in data driven testing. Hasan then demonstrated a live session of tests written in Spock which did the testing and validated the returning values. The audience took great interest in this idea. More details here: spock-presentation
Faisal Tajammul, Managing Director of BroadPK Technologies then took over the stage and expressed his thoughts on “Why Testing is important?” Faisal spoke out of his ~20 years of experience that if people chose Software Testing as a profession because they think that Programming is tough, they are wrong because in his opinion “Testing is tougher than Programming”. He engaged the audience by asking what is the single thing that can be done to deliver Quality products and then summed up this discussion by saying that if Testing is involved early in the process, it is the best way to produce Quality product.
The stage was then handed over to our own Dr. Zohaib Iqbal who is President PSTB to share more information on the exciting upcoming event: the first ever Pakistan Software Quality Conference . Dr. Zohaib in his typical relaxed yet very professional way, recapped the journey of testers meetup and expressed his gratitude for the interest shown by both the Testing practitioners and IT companies. He then talked about more specific details about the Conference which is happening on April 8th, 2017 at Marriott Hotel Islamabad and will feature Maik Nogens (known for holding Software Testing Worldcup) as Keynote Speaker along with other interesting talks. Dr. Zoahib urged the audience to get active and if they become Conference Champions by spreading the word, they can avail some additional discounts on top of the Early Bird being offered.
Certificates were then awarded to the very able BroadPeak testing team who organized this event under the leadership of Syed Qambar Rizvi. Shields were presented to the Presenters and all testers walked towards the Cafetria to enjoy very much needed cup of tea with snacks.
This was the best time of the event where we saw scenes like this: Testers meeting old friends, Testers making new friends, Testers getting to know more about other companies and their testing processes and techniques, Testers introducing their work to others who listened etc. etc. These conversations will go a long way in the careers of each Tester for sure.
We are very much looking forward to lead all this activity to the climax of Pakistan Software Quality Conference in April. There is another meetup being held in Lahore on February 25th in this build up, so will talk about that soon.
I’m sure that you do it in some form or other with may be a different name. And if you don’t do it, I think you should.
The idea is not wholely mine but this particular adaption is mine which I have been using for many years. The model looks good to be shared with other testers to use. it has some inspiration from Google’s ACC (Attributes Components Capability) model, and I have explained earlier too but to answer a specific question only.
Okay, this is what you should do:
- Divide your testing project in parts. The best count is 9 to help you draw the result in 3 x 3 shape. On what basis you can divide: if there are modules, or components, or facets, or attributes, or keywords or whatever works to help make decisions. The below example I have shows 9 core components in the Platform that we test.
- Decide on a color coding to show the “state of testing”. Testers also use happy, sad or angry faces to reflect on the fact how good is testing for a particular component, but colors help look it like a heat analysis. As if you expose this 3 x 3 cube to heat, how different parts will look after some exposure. The common color scheme is what you already know: Red for alarming situations, Yellow for cautions and Green for all good.
- You are good to go and do the analysis. You’ll need some definition on why you are marking a box as Red vs. marking it Green. Make sure you have both data and feelings to support it.
This is how it might look which uses an Orange color too:
So how this helps. In many ways:
- As appeared in my original article about three years ago, it helps you where to place the new tester. Obviously you are concerned on the Red areas which you want to make green.
- To make release decisions. How important are Red boxes for the current release? Remember, you’ll never get all Greens, so you have to make that decision on what is your acceptance criteria.
- As a measure of progress for example, we share this heat analysis to reflect on functional code coverage for the components. I know those numbers are not the only thing to look for, but you still need some metrics to guide you if you are on track.
Have you tried this or similar models before? What did you find out?
The first thought that comes to a mind that doesn’t understand Testing and Automation is this: “I’ll automate all tests that are being executed manually and it will save time”.
This is what I thought almost a decade ago and added a column in the test plans that I used to execute. That column was named something link “Automated Test Id” and I started assigning Ids to each test that I automated and cross referenced it here. Looks nice, isn’t it.
Soon cracks appeared in my proposed system. There were some tests that I couldn’t automate e.g. the ones that needed visual verification of Engineering components. There were some automated tests that when I wrote, I added additional checks than what the manual test proposed e.g. checking more negative testing scenarios as it was easy to do that in automated tests compared to manual tests. Then there were new tests that I started to think of while running either automated or manual tests. I wasn’t sure if I should first write a test script for them and then add it’s entry in the test plan sheets OR I should first add it to my manual test plan and then automate it. This was one of the problems where I started to seek help from the internet and I concluded that my strategy was flawed. But I learned the following:
Manual tests has no correlation with automated tests. If they appear same sometimes, that is just a coincidence.
I also thought that anyone who has spent any serious effort in test automation would know that. But last week I saw the following in a project being managed in Microsoft’s Team Foundation Server:
It was okay to have this dialog in 2006 but towards the end of 2016, it is a sin. I’m not sure if this is standard one coming from Microsoft or it was done by some geek who manages the project.
Joel Spolsky said around that time in this context of automating all tests:
And so one result of the new emphasis on automated testing was that the Vista release of Windows was extremely inconsistent and unpolished. Lots of obvious problems got through in the final product… none of which was a “bug” by the definition of the automated scripts, but every one of which contributed to the general feeling that Vista was a downgrade from XP. The geeky definition of quality won out over the suit’s definition; I’m sure the automated scripts for Windows Vista are running at 100% success right now at Microsoft, but it doesn’t help when just about every tech reviewer is advising people to stick with XP for as long as humanly possible.
When people have the above-mentioned thought process, they say/ask the following things. I’d suggest to stay away from these:
- Our testing is 75% automated. See this is because they believe that we have this many tests in total and out of which, that many are automated.
- Through automation we have saved 250 hours. Which means that if those tests were executed manually for this many times, such would be the time taken to accomplish this.
- By automating all our testing, we’ll downsize our testing team. Because they will replace the manual tests being executed by those testers.
- Which manual tests I should be automating?
The above statements keep coming because a) the tool vendors create this hype as they want to sell their phakkis and b) the management who runs the show has minimal understanding of what testing is and what automation is.
Have you said or seen above understanding on test automation? Do you believe in them or have any thoughts on this?
There is nothing that we can call “excessive testing”. The more you test, the better it is.
When we get to know the first few Laws of Testing and understand that “Testing can never be complete”, we get into the mode of limiting our testing to the constraints. This often includes selecting the best tests to execute in given time, leaving out some testing types and worse just having breadth in testing for the sake of leaving all the depth in testing.
Last week, I was discussing with my manager on the possible approaches we can take to add tests for multi-threading in our unit testing suite. He is based in the US, leads a big development team and has done many successful releases of complex systems, all approaches we discussed had some flaws and we couldn’t agree on the “best approach”. You can guess the decision we made. No, it was not to drop this type of testing altogether because we don’t know the best way to do that. The decision was to try the approach that is one of the candidates of becoming a good approach, do it for a while and learn from it.
That is what my manager described as a test strategy to have “Defense in depth” where you can imagine a castle which is built on the top of a hill with a big wall and then outside fence and then there are trenches and then there are landmines around it and then there are other ways to protect the castle. All of these layers of defense have some advantage over the other and though they cannot protect from all type of attacks, they still limit the chance of attack by capturing some of it.
Or you can imagine a soccer team that has the Italian strategy to never allow the opposition to score a goal.
(the image is taken from here: http://blog.smartbear.com/design/what-medieval-castles-can-teach-you-about-web-security/ )
I have often shared my views about Unit tests being a safety net which allows Programmers to make changes without fear that if they fall, they’ll not fall on the ground. So in that respect, why just have one safety net, why not have multiple nets such that if the fall is big and it raptures the first net, the other one saves you from falling on the ground.
In my experience, the following layers of protection or safety nets are kind of a must:
- Set of Unit tests that run with the individual code manipulations. They are by default automated and usually executed multiple times in a day.
- Set of tests that are executed immediately after the build is made. These are Smoke tests and other tests such as Performance which were not executed as part of the build. These should also be automated tests and run with each build.
- Set of tests that are Exploratory in nature and are executed by Testers who have knowledge of the domain and the overall system. Some experts believe that these tests are useless or repetition if we have the above two layers but as I mentioned above, they add another layer of defense. And I have seen many big bugs coming from this round of testing as the above two are merely Regression tests.
Do you believe in having depth in your software testing layers? How many layers you have or would like to have?
Karachi known as commercial capital of Pakistan has an iconic image such that if it doesn’t happen here, it doesn’t exist. As the activity based Testers Meetup met the vibrant and passionate Testers community in Karachi last Saturday after having successful rounds in Islamabad and Lahore, we can safely say that such activities are here to stay.
The event was well orchestrated by our partners VentureDive team gyrating around testing champion Zeeshan Asghar. The event was running under the banner of Pakistan Software Testing Board (PSTB) . The cafeteria of Dot Zero located at 14th floor at the famous Tariq Road was a perfect choice for the event where ~50 testers were invited to attend the event. The host Mohsin Ali started the proceedings once the teams were settled on their tables.
The event was kicked off by an opening note by Usman Zuberi who is Project Manager at VentureDive. Usman welcomed all on behalf of his organization and recapped a brief yet rising journey of the company. His message to the testers was very clear: “Don’t adapt this career just because you think you are misfit as a Programmer.” Usman affirmed the audience that if they take testing career as a passion, they have a long road of success ahead of them.
The day was about the participants broken into 9 teams doing live rounds of testing. The fiercely fought competition went through the stages of planning the tests, executing the tests, finding the bugs and finally presenting the findings to the audience. To keep with the tradition of so many TV shows running from Karachi, there were cash prizes for the top three teams. The application that was given to the teams was a typical eCommerce site with both customer and admin modules. The teams took part in each round with lot of energy and it was heartening to see so many brilliant testes showing their skills in action.
(More pictures are on our facebook page)
Since knowledge comes from doing things but it also comes from learning from people who have already done those things, the above rounds of testing were sliced through some knowledge sharing sessions.
Syed Azhar , Director DvOps at Careem explained how modern day apps testing can’t limit itself to just functional testing. By giving examples from the scope and depth of the architecture that Careem utilizes to make it a success, Azhar said that living in silos of Business Requirements, Product Management, Development, Testing, Deployment are not the way to go. DevOps brings all of them together and he also explained the 5 Cs of DevOps. His full presentation is here: devops-solution-qa
The most loved talk of the day was on lessons learned in Test Automation by Syed Tassaduq Hussain who now heads the Quality Assurance team of TPS. In his typical style, Tassaduq shared many rules through his personal experience where he split the lessons amongst Management and Engineers. One of his suggestion to the test management was to form the team such that it benefits maximum automation ROI because automation cannot be done by the manual testers on their spare time. For Engineers, he explained what Flaky tests are and how we can conquer them to build confidence in automation. The slides are here: automation-testing
The third talk was by our own Zeeshan Asghar on the challenges we face in the testing where so many platforms, so many devices and so many browsers are out there where Compatibility and Regression testing has to be run. Zeeshan suggested to use available tools to accomplish this so that testers focus on tasks that require human intelligence and give such routine work to the testing platforms out there. See more details here: software-trends
The event was also graced by two Meetup pioneers from Islamabad, Amir Shahzad who is Manager Testing at Stella and Nouman Umer representing Zigron. Amir shared his feelings on how such meetups help testers grow together and was happy to see that these forums are now reaching new locations.
Taking this opportunity, I briefly talked about what PSTB is, the certifications it offers and how it’s objectives are well aligned with the needs of Pakistan Testing community. I shared the formula of success:
Testers Meet + Learn together in fun setup + Build Connections = Testers Succeed
Also invited the testers to the proposed National level Testing conference which we plan to do in March 2017 Insha Allah (God Willing). The slides are here: pstb-and-meetups
The event was also graced by Asim Kazmi, leader of 10Pearls testing and who is one of the faces of community. Another popular figure Faiza Yousuf couldn’t join us but gave us full support and some selected University students of her joined us in what we calling making the future testers.
It would be an injustice to not name all the volunteering team that included Shahan Jahangir, Laraib Vaqar Patoli, Mohsin Ali, Mariam Hammad and Shafaq Zehra (pardon me if I missed any one) who made all the proceedings smooth. And did I forget to say that the “Biryani” lunch was lovely 🙂
Thanks to all the participants in joining us in the journey of Testers Excellence. Let’s work together to take it to the next level.
One of the sources of knowledge is to know the sources of knowledge. Each society has sub-societies in it and if you are not there, you’d think that this never happens in our city. This exactly is the case for me but I was lucky to become part of Pakistan Agile Development Society last year through the 2nd edition of Agile Conference Pakistan. That allowed me to learn more throughout the year and the best part came in the last weekend when along with a wonderful team of organizers, speakers, sponsors, volunteers and audience I enjoyed the Agile Conference Pakistan 2016 which was attended by ~250 Agile practitioners.
Below is a very very short summary on what happened there and more will be shared on Society’s facebook page and website. The first set of photos is out and soon the slides and recordings will also be shared.
Society’s chairman Suhail Iqbal said in opening note that it is a very happy trend where more and more people are joining the Agile conference. He also mentioned that though Agile movement arrived a little late in the Pakistan IT industry but we can work together to make it happen in a great way.
The first talk was “Agile Transformation vs. Agile Adaption” by Syed Ahmed who is CEO of DPL-IT and ex-chairman of P@SHA. Syed started the talk with difference between Process-centric companies and People-centric companies and rejected the notion that one process can change the world rather it is the human potential that changes the world. With some examples he explained that the yardstick of whether a company is process or people-centric is the answer to this question: “Do you allow people to make mistakes?” His view was that Agile Adoption is like doing the rituals whereas a Transformation will come when you really have belief in something. After establishing these facts, he presented the case study of his own company DPL-IT where they have gone through an Agile Transformation. He shared details on how they are promoting a “Rebel ethos” culture and how they have a “Sir-carr” which fines you for a certain money if you call people as “Sir” rather than with their names. He also suggested a flipped hierarchy as below (this is drawn by me and not the actual slide):
In the last few slides, he had some lessons learned which included a “Let Go” thinking in top management’s mind, making things transparent and having a belief that the results will start coming though there will be some rough patches.
The next talk was about some suggested recommendations on Managing Agile Transformation by Javad Ahmed who is COO of a US based company Oracular (Smart-IS in Pakistan). Javad gave his viewpoint on what Agile is, why we need it, and how he has implemented it in his own scope. Given his consultancy background, through various examples he shared the practical implications of doing an organization wide Agile transformation. He had lot of good advice to offer for bringing in the “business perspective” in making all decisions and below are some quotes that I noted:
- Nothing can be done without Top management’s commitment and they’ll only buy-in if they see business value.
- Agile is focused on “Right Results” and an “On-target” outcome.
- Agile is an ideology.
- Agile facilitates prioritization.
- Do not short-change staffing e.g. asking a Scrum Master to play as Product Owner too.
- Counselling and Coaching is very important in the Agile journey.
- Too many times the documentation is like a solved Pakistan Studies papers that happen in our education system where marks are given based on how lengthy and colorful is the answer.
Then Afif Zaid and Mujeeb Zahoor shared the story of Agile success at S&P Global’s Pakistan office. Mujeeb walked through a brief history of the overall organization and the stuff that happens in Islamabad. Afif then provided inside scoop of how they worked through this Agile Transformation phase and how they reached to the milestone of having 57 successful SCRUM teams across the globe. His tips for others to do the same were passing on the vision, help team on setting the priorities and making teams self-organized and empowered. One of the many experiments they are doing is playing a “Shark Tank” to allow everyone to throw ideas. Mujeeb then concluded the talk by sharing the business success that Agility has brought to them.
OPEN Islamabad’s President and CEO of Sybrid, Ather Imran was the first speaker after lunch to talk about “Agile Culture and Knowledge Organizations in Pakistan. Ather referred to Peter Drucker’s work on what a Knowledge Worker is and what is a Knowledge Organization. Then after briefly going through the Agile principles, he concluded that:
Knowledge Organization = Agile
He then brought in some interesting thoughts on the Power Psychology and how we tend to be followers by default. Through examples including the South Korean Airline experience from Malcolm Gladwell’s book Outlier, Ather made it clear that our culture is where people defer to the Power and want to be obedient to the authority. He then shared a few tips to break that with consistent efforts which included 10 Rules that he explained one by one. For example he wants to “Delegate to the level of Anarchy” because if you are delegating and are comfortable with that, there is still control in your hands. So getting uncomfortable with delegation is the key.
The next talk was by the leadership training maestro Mohsin Lodhi who talked about “Leadership in the Digital World”. Mohsin started the talk with establishing the fact that how technology is eating many jobs and many industries are now squeezed to hand held devices. Through his typical style of backing statements with numbers, he shared many new researches that enforces the reality that we are living in a fast paced and non-linear changing world. He connected the dots that Results are very important but one should never forget that the results come through processes and processes are due to a culture and creating a culture is the job of the leader. (The below is my notes and not actual slide)
He then through some moving and dancing slides, shared the lessons from the famous social experiment done with 5 monkeys and suggested not to react to the shower of culture around us by saying that “that’s how it has always been”. At the end he also explained the Change Equation and how leaders should work on the buy-in and show through their actions if they are serious about bringing any change.
An Agile Coach at VizTek and an Agile lover Nabeel Ansar then talked about “Avoiding common mistakes in Agile Transformations”. Nabeel started the talk with when and how Agile fails and emphasized on the conditions in which you apply Agile to be more responsible for the failure. He gave an interesting example of driving a Ferrari on the IJP road and then blaming the road for the poor performance of the car. Through his personal experience of Agile Transformation in few organizations, he recommended to first “show something by doing it” and then ask for others to help. He also shared 8 Agile Transformation Patterns which included the likes of “Start small” and “Seed and Grow”. The point that I most liked in the talk was Nabeel’s insistence on improving the Engineering practices because a fast delivery is only possible if you have the automation, testing and other build processes geared towards such delivery.
A short closing note was delivered by Farid Zafar, Associate Professor at Bahria University Islamabad thanking the audience for sparing their precious weekend day for the love of Agile.
Believe me my notepad had much more than the above and I’m sure that was the case with all the participants. Let’s keep this journey towards excellence together and I hope to share some after thoughts soon on some of the subjects listed here!