Inadequate Software Testing Jeopardized Elections in Pakistan, Why?
(On the eve of Pakistan General Elections 2018, everyone was anxiously waiting for the results when it was told by authorities that the newly deployed system to transmit and manage results have failed. Yes, the technology failed when it was most needed. Our regular contributor Sohail Sarwar takes a deeper look at what the problem was and more importantly how we can avoid such things happening again)
Conducting elections in any country is a major political activity since it decides the future “reign-holders” of a nation. Similarly, election 2018 was conducted by Election Commission of Pakistan (ECP) on a national scale (with 272 national and 676 provincial seats). A total of 12570 candidates contested to win hearts of 106 Million Pakistani voters at 85, 307 polling stations for representing them till 2023.
In order to manage the election activity at such a massive scale, ECP for the first time deployed a system named Result Transmission System (RTS) and Result Management System (RMS) for consolidating/compiling/tracking the election results promptly. RTS was made available as mobile app whereas RMS was installed in specific systems for the concerned election personnel.
However RTS appeared to be a nightmare for field personnel on “D-Day” i.e. on 25 July 2018. It was revealed that mobile phones would hang when RTS app was launched. The directed fix to this problem was removing the app, downloading it and re-installing every time. If this component of installing and launching worked fine, the submission of results (after attaching the images) made the application not responding.
Due to these issues, polling staff in field could not submit the results as expected by ECP and suffered a backlash from political entities due to non-responsiveness of RTS. Consequently, fairness and transparency of elections was questioned that could end up in a chaos. Also, competence of eminent professionals managing world class software products was also doubted. Why this trouble was caused? A number of reasons can be enlisted but our answer is “Lack of Thorough Software Testing”.
Some of the apparent reasons that we can think of from the perspective of software quality are:
- Deploying the systems in production without testing to ensure end-to-end functional completeness of both components i.e. RTS and RMS.
- Pilot testing of RTS was done on a very limited test bed i.e. only 2 constituencies. (https://www.ecp.gov.pk/frmGenericPage.aspx?PageID=3083). Obviously, health of application with load generated by 300-400 polling stations would never endure the load generated by 80,000+ polling stations.
- Overall testing of non-functional perspectives was ignored such as configuration testing (for uniformity of mobile types and OS versions); scalability testing and performance testing (load testing and stress testing).
- Load balancing for facilitating a seamless downloading from sources seemed missing since anomalies were identified while registering the polling personnel (8 members were directed to register at one time from one constituency that seemed impractical).
- No dedicated hardware testing infrastructure where applications were hosted with availability of minimum bandwidth requirements.
- Testing from Cyber security perspective so that any situations in Western countries could be countered. Not sure if this phenomenon really applies to the system developed by ECP.
- Relatively, a shorter span of time was kept for testing the products since these issues may prevail with uncooked applications.
These are few suggestions that may lessen the chances of getting into “Fire-fighting” situations as witnessed few days ago.
Do you know more about the episode? Or would like to share suggestions on avoiding such failures?