A hastily-developed app and combined with a lack of user testing caused a ruckus in Iowa Caucus voting this week. What’s in store for New Hampshire, Super Tuesday, and beyond?
It has been an exciting week in US politics. We had a State of the Union address and an impeachment vote. A whirlwind week by any standard! Before we could even get to those two events, we started the week with an outrageous technology failure in the Iowa Democratic Caucus. For purposes of this blog post, I’m not going into the differences between a caucus and a primary. Let’s just assume they accomplish the same result: selecting a candidate for political office.
For the Iowa Democratic Party, Monday night was a disaster and then it continued into Tuesday, Wednesday… you get the idea. As I write this blog post on Friday morning, I’m not even sure if they have an official determination of who won. The news stories seem to be conflicting.
So here is what we know about the app (IowaRecorder) failure based on available reports. The Iowa Democratic Party hired a marketing technology company to build an app which would be used, statewide, to report results of local caucus votes (I’m simplifying for purposes of brevity). The app was going to be used to submit voting results. Nobody was actually voting through the app.
This first really good article I read that outlined the technology implementation failure, came from Slate. Here was a good summary from a couple of days ago:
“It’s still unclear what exactly went wrong with the app, but all of these issues appear to have something in common: The Iowa Democratic Party clearly wasn’t prepared for any possible issues with the app and a more involved method of vote reporting introduced this year—and sure enough, it reportedly turns out that the app was never tested on a statewide scale. Shadow, which is run by alumni of the Barack Obama and Hillary Clinton campaigns as well as Google, was paid $60,000 to develop the app, but it had just two months after party officials decided to abandon plans to report results over the phone.”
There’s some updated information on the failure from Motherboard, which was released yesterday (along with the app code). Below, you’ll see that they released an app that was still in beta format:
“And Instead of going through proper app store review processes conducted by Apple and Google, Shadow used beta testing platforms like Apple’s TestFlight to distribute the software so it could meet the Monday deadline. So when it came time for the app to do its most critical role — letting Democratic precinct leaders report results from Iowa on Monday — it failed in every way imaginable.”
Expert Analysis:
As I do when these major stories break, I turn to Experts.com members to get insights. You may recall Dr. Stephen Castell. Dr. Castell, Chartered Information Systems Practitioner and Member of the Expert Witness Institute, is Chairman of CASTELL Consulting. He is an internationally acknowledged Independent Computer Expert who has been involved in a wide range of computer litigation over many years.
Dr. Castell and I wrote a blog post back in 2018, regarding West Virginia’s Blockchain voting program. They are actually expanding this plan, which may necessitate a separate blog post.
Below, please find my questions and Dr. Castell’s answers (Disclaimer: these questions and answers provided on February 5th, 2020):
Nick: From available reporting, it appears the Iowa Democratic Party failed to do a statewide testing of this vote reporting application. What type of tests would have been necessary to identify errors in the system before statewide roll-out?
Dr. Castell: As other ICT professionals comment in the reports, there should be thorough systems testing and QA procedures, including User Acceptance Testing and Pilot Trials, plus scaled-up ‘soak testing’, before contemplating any real-world launch, such as this statewide roll-out. You expect to get errors in systems testing – its main purpose is to identify faults and fix them. Sadly, software systems and Apps these days do seem often to be launched publicly without adequate systems testing, let alone with adequate prior User Testing and Pilot Trials. If such standard professional QA processes were omitted, or truncated, for something as high-profile and important as an App to collect and relay voter data in the Iowa Caucus, that does appear rather astonishing.
Nick: It doesn’t appear that all of the fault lies with the app developer. It seems the Iowa Democratic Party only gave the app developer 2 months to develop and deploy this application. What sort of time-frame would you anticipate to develop, test, and implement a software of this scale?
Dr. Castell: That is difficult to estimate without more knowledge of the actual detailed Customer Requirements Specification that the developer’s App was contracted to meet. On the face of it, an App simply to in-gather voting data, aggregate and transfer it, sounds in principle like fairly straightforward functionality to code, test (at scale) and implement, and 2 months may not have been an unrealistic timescale for development, testing and deployment.
Nick: Would you expect there to be a certain level of user sophistication for those using the app on this scale? Should there have been company representatives available at caucus sites?
Dr. Castell: Reports suggest that there was little prior familiarity, let alone ‘training’, or ‘user sophistication’, with the App on the part of those expected to employ it for real, in the high-pressure, real-time Iowa Caucus conditions. Whatever the state of compliance of the App with its contractual specification – perhaps reasonably well delivered to time, budget, specification, and of suitable quality, ‘fit for purpose’ – if there was no program for adequate user familiarity and training, plus some sort of support and trouble-shooting team from the developer company at caucus sites, that alone could account for the problems encountered in statewide roll-out operation.
Nick: From what I’ve read, it looks like the company was paid $60,000 to build this application. Is there any way to gauge whether this is too little or too much for this type of application development?
Dr. Castell: Again, that is difficult to gauge without more knowledge of the actual detailed Customer Requirements Specification, and thus the likely complexity of the functionality needed, and its associated software design and coding; also, there may have been a tight budget to which the developer company was obliged to work. It is not unusual for software developers to invest in a ‘plum’ assignment such as this high-profile Iowa Caucus project, for the promotional and marketing impact that gives them in securing hopefully more lucrative and profitable development jobs later. In this case, the $60,000 could have been much less than the true cost to the developer company of the analyst, designer, coder, tester, deployer and trainer man-days expended in building and launching the App with a statewide roll-out, against a tough deadline.
Nick: What sort of testing, trials, and quality assurance requirements would you have employed prior to such an implementation?
Dr. Castell: There should ideally have been thorough systems testing and QA procedures, including User Acceptance Testing and Pilot Trials, plus scaled-up ‘soak testing’, well understood by ICT professionals, before the real-world launch of this statewide roll-out. Relevantly, I teach a Course Avoiding IT Disasters – the Expert Way, the principles of which are also covered in my seminal paper “Forensic Systems Analysis: A Methodology for Assessment and Avoidance of IT Disasters and Disputes”, issued as a Cutter Consortium Executive Report, Enterprise Risk Management & Governance Advisory Service series (Vol. 3, No. 2, March 8, 2006).
We cannot say that New Hampshire is next. All available information tells us that New Hampshire is not using the same company/app used in the Iowa Caucuses. Furthermore, there was talk of Nevada using the app, but they have claimed they will not move forward with the application.
That’s the end of this particular blog post. Though, we’re already in talks about another post related to voting systems.