Designing Interfaces for
Voting Machines
Benjamin B. Bederson
Computer Science Department
Human-Computer Interaction Lab
University of Maryland
www.cs.umd.edu/~bederson
bederson@cs.umd.edu
February 4, 2005
Frustrated voters
 Voting technology and ballot design can
influence election outcomes
 Minorities and the poor are more likely to
cast their ballots on outdated systems
 Technology is in need of updating
When Interfaces Get in the Way
 Ballot design

Butterfly ballot
 Interaction


Hanging chad
Changing vote (i.e., how to unselect a candidate)
 Write-In problems


2004 - NY Times Editorial reported on San Diego mayoral
election where voters for write-in candidate Frye didn’t darken
a bubble.
2002 - Mt. Airy, MD mayor went from Holt to Johnson to Holt,
based on acceptable spellings.
Usability Part of Larger Issues
 Florida 2000 – Traditional technologies flawed



Mechanical levers – break down, difficult to
maintain, difficult to store and transport
Paper ballots – errors, difficult to process and
interpret
Punch cards – hanging chad, etc.
 Economics de-emphasizes usability
 Focus on security de-emphasizes usability
 Lack of research because of proprietary
systems and number of designs
Our Study
 Funded by

NSF (National Science Foundation), Grant #0306698

Carnegie Corporation, Grant #D05008
“Project to Assess Voting Technology and Ballot Design”
 Consists of:





Expert review
<= Focus today
Lab study
<= Focus today
New technology
<= Focus today
Field test
Natural experiments
 Co-Researchers with



Paul Herrnson – Univ. of Maryland (project leader)
Michael Traugott & Fred Conrad – Univ. of Michigan
Richard Niemi – Univ. of Rochester
Small-scale studies to
demonstrate potential
challenges and inform
future research
Does not address
accuracy, affordability,
accessibility, durability, or
ballot design
This represents partial
results mid-way through
a 3 year study. Future
work will address
accuracy, ballot design,
and more
Partners




Federal Election Commission (FEC)
Maryland State Board of Elections
National Institute of Standards and Technology (NIST)
Vendors





Diebold
Hart InterCivic
ES&S
NEDAP
Avante
 Advisory Board
Machines Looked At
 Avante Vote Trakker
 Diebold AccuVote TS
 ES&S Optical Scan
 Hart eSlate
 NEDAP LibertyVote
 UMD Zoomable system
As available for testing.
Some machines have
been deployed with
different options.
Some machines have
since been updated.
Vendors (except NEDAP)
implemented ballots for
best presentation.
Machines selected to
represent specific
features
Avante Vote Trakker
All photos taken by our
research group – not
provided by vendors.
Diebold AccuVote TS
ES&S Optical Scan
Hart eSlate
NEDAP LibertyVote
UMD Zoomable System
www.cs.umd.edu/~bederson/voting
Demo
Expert Review
 12 HCI experts one evening




1 voting interaction specialist
1 government usability practitioner
5 academic HCI researchers
6 private usability practitioners
 Each used



6 machines
2 ballot types where available (office block, party column)
~15 minutes each
 Asked to list concerns
 Followed worst case perspectives of





novice
poor language skills
older voters
stressed voters
system errors
Most experts did not
have background in
voting systems
Subjective responses
require interpretation
Expert Review
Rating System
 Each issue given a severity rating
(1-low, 5-high)
 Concerns listed with average severity,
# of instances
Avante VoteTrakker
Concerns (average severity, number of instances)
5.0
4.0
4.0
3.0
3.0
3.0
3.0
3.0
3.0
1
2
2
2
1
1
1
1
1
Write-in requires last name
Record shown too fast and without instructions
No previous button1
1 Navigation focuses
Auto-forward confusing1
on progress with later
Smiley face inappropriate
review by design
Title too small
Instruction text poorly written
Didn't like this one at all
"Cast ballot to continue" not clear - it actually finishes
Avante VoteTrakker
(more I)
Concerns (average severity, number of instances)
3.0
3.0
3.0
3.0
3.0
3.0
3.0
3.0
3.0
3.0
1
1
1
1
1
1
1
1
1
1
Timed out, but didn't see warning
Angle of machine is awkward
Lot of reflection on screen
Flashing instruction is distracting
Colors of text poor (green/white, black/blue)
No progress feedback
No way to cancel and leave 2
No way to start over
"Please make selection" message is distracting
no error-checking on write-in
2
Can time-out to cancel
Avante VoteTrakker
(more II)
Concerns (average severity, number of instances)
3.0
3.0
3.0
3.0
3.0
3.0
3.0
3.0
2.0
1
1
1
1
1
1
1
1
1
3
By design to minimize
Write-in association very small
under-votes
3
No way to go to end and cast ballot
Lack of color on amendment screen may appear to be an error
Disabled button is "white" which is very difficult to understand
Cast ballot button requires 2 presses
Can't say "no" to paper record - so why bother?
Have to pick contrast/text size before starting
No instructions after starting
Not clear what to do at beginning
Diebold AccuVote TS
Concerns (average severity, number of instances)
5.0
5.0
5.0
4.0
4.0
4.0
4.0
4.0
1
1
1
1
1
1
1
1
3.0
3.0
2
1
Ballot review confusing. Review colors don't match voting colors
No help on some screens
Write-in has no instructions
Contrast and text size controls not clear
Some font colors unclear (black on blue, red/blue)
Party not clearly indicated
Difficult to use while seated
Large font is good, but "issues" text runs over screen display area requiring
arrow navigation
Wait icon is too computerish and not clear
Card hard to enter
Diebold AccuVote TS
(more)
Concerns (average severity, number of instances)
3.0
3.0
3.0
3.0
3.0
3.0
3.0
3.0
1.0
1
1
1
1
1
1
1
1
1
Poor depiction of voting vs. reviewing state
"Card not inserted" error needs a diagram
Buttons have poor visual affordance
Instructions refer to "backspace" key, but is actually labeled "back"
Instructions unclear (i.e., "Vote for one")
Some text unclear (i.e., "2 of 4")
Multiple write-in unclear
Write-in not well associated with race being voted
Extra dots on help/instruction screens
ES&S Optical Scan
Concerns (average severity, number of instances)
5.0
5.0
4.0
4.0
4.0
4.0
4.0
3.5
3.5
3.0
1 Instructions not mandatory, errors likely
1 Write-in has high error mode (enter name, but not fill in circle)
2 Changing vote process is punitive - must start over which could cause
some to give up
1 Poor visual grouping (title could be associated with items below)
1 Could fold, bend or tear ballot
1 No instructions to review ballot before submitting
1 Instructions to turn over page not conspicuous enough
2 Font size is fixed, and will be too small for some older and other voters
2 No error checking on under-vote
2 No error checking on over-vote
ES&S Optical Scan
(more)
Concerns (average severity, number of instances)
3.0
3.0
3.0
3.0
2.7
2.0
1.0
1.0
1 Should use different highlight/feedback that vote was correct
1 Why two sets of matching instructions?
1 Instructions somewhat difficult for voters with limited English
proficiency
1 Instructions should say something about no extra marks on ballot
3 Needs a better table - low and shaky 1
1 Seated operation awkward
1 Different cost/quality
1 "Vote in next column" unclear
tables available
1 Appears to be an entry field at top of column
Hart eSlate
Concerns (average severity, number of instances)
5.0
4.0
4.0
4.0
3.5
3.5
3.0
1
1
4
1
2
2
1
Combining summary and cast ballot confuses actual casting
No way to jump to end
Dial slow to learn, hard to use 1
Red on blue text, and light fonts hard to read
After reviewing, it's hard to get back to a choice to change it
Blue movement on screen is disconcerting
Cast ballot button didn't accept push - required 3 presses
1
Compare to subjective/objective data later
Hart eSlate
(more)
Concerns (average severity, number of instances)
3.0
3.0
3.0
3.0
3.0
3.0
1
1
1
1
1
1
Poor progress indicator
May confuse with a touch screen
Can't clear entire vote and start over in one step
Write-in screen does not indicate office being voted for
Next/Prev and Dial ambiguous
Auto-forward on select, but not unselect (inconsistent interface)
NEDAP LibertyVote
Concerns (average severity, number of instances)
5.0
5.0
5.0
5.0
5.0
4.5
4.3
4.0
4.0
4.0
4.0
4.0
2
2
1
1
1
2
3
3
2
1
1
1
Write-in message after OK is confusing
No way to confirm/review write-in name
"No vote" light should be different color (difficult to see what wasn't finished)
No clear way to handle multiple write-ins
Poor feeling of privacy due to size
"Enter write-in" button doesn't seem to work
Under-vote message easy to miss
OK button for write-in too far away
Too much reflection
OK button with 4 arrows is weird
Propositions too far away
Hard to read/access from seated position
NEDAP LibertyVote
(more)
Concerns (average severity, number of instances)
4.0
4.0
4.0
4.0
4.0
3.7
3.5
3.0
3.0
3.0
3.0
2.0
1
1
1
1
1
3
2
1
1
1
1
1
Number pad unclear - what is it for?
Blue light coding (voted/unvoted) unclear
"enlarge" scrollbar un-obvious (to left of little message screen)
Buttons hard to press with poor tactile feedback
Scroll bar thing to right of message box unclear
Difficult to correct a vote
Write-in area too far away
"Partisan offices" unclear terminology
Can change language accidentally
Same color for race and candidate is unclear
Prefer sequence to "jump around" model of full face ballot
No second chance to cast vote - review is implicit
NEDAP Actual Ballot
UMD Zoomable
Concerns (average severity, number of instances)
4.0
3.0
3.0
3.0
3.0
2.5
2.0
3 Color of review & cast ballot buttons should be different than
progress indicator and selected items
1 Not clear how to get started
1 Feels like a game - possibly inappropriate
1 "Not voted" confusing when multiple choices available
1 Peripheral races too visually confusing
2 Progress/navigation buttons is partly a progress indicator, but not
clear enough
1 Overview buttons shouldn't split 4 sub-types
Lab Study
 42 members from Ann Arbor, MI voted on 6 machines


Paid $50 for 1-2 hours
Different Random orders for different people


Over selected for potential difficulty





Latin Square design
Most (69%) >= 50 years old
Most (62%) uses computers once very 2 weeks or less
Most (30) voted on office-block ballot
Indicated intention of (fictional) candidates by circling names on paper form
Study not controlled for prior experience, but Ann Arbor uses optical scan
 Data:



Satisfaction ratings reported after voting on each machine
Time measurement
Videotaped interactions
Lab Study (more)
 Looked at:




Time voters spend reading instructions
Response to paper or on-screen ballot
Response to the reporting of under- or over-voting
Ability to change a vote
 Complications and malfunctions of DRE or
Optical Scan Readers
Lab Study – Satisfaction Data
 Usability studies typically measure:


Speed, Accuracy, Satisfaction
We are currently reporting on two
(Speed, Satisfaction)
(1= Strongly Disagree, 7 = Strongly Agree)
Agreement
“The voting system was easy to use”
7
6
5
4
3
2
1
Diebold
Liberty
Avante
Hart
Zoomable
Average score by Machine
ES&S
(1= Strongly Disagree, 7 = Strongly Agree)
Agreement
“I felt comfortable using the system”
7
6
5
4
3
2
1
Diebold
Liberty
Avante
Hart
Zoomable
Average score by Machine
ES&S
(1= Strongly Disagree, 7 = Strongly Agree)
Agreement
“Correcting
my mistakes was easy”
7
6
5
4
3
2
1
Diebold
Liberty
Avante
Hart
Zoomable
Average score by Machine
ES&S
(1= Strongly Disagree, 7 = Strongly Agree)
Agreement
“Casting a write-in vote was easy to do”
7
6
5
4
3
2
1
Diebold
Liberty
Avante
Hart
Zoomable
Average score by Machine
ES&S
(1= Strongly Disagree, 7 = Strongly Agree)
Agreement
“Changing a vote was easy to do”
7
6
5
4
3
2
1
Diebold
Liberty
Avante
Hart
Zoomable
Average score by Machine
ES&S
Lab Study - Time to Cast Ballot
12
Subjective
Objective
Minutes
10
8
6
4
2
0
Diebold
Liberty
Avante
Hart
Zoomable
Average score by Machine
ES&S
Lab Study – Analysis Remains
 Why are some machines consistently most preferred
and others least preferred?
 Detailed coding of video interactions underway
 Planned analyses of video interactions:


Tally of problems by machine that do and do not lead to
unintended votes cast
Explanation of satisfaction data in terms of voters’ actions
 Remember that usability is only one characteristic of
overall performance

Accuracy, Accessibility, Affordability, Durability, Security,
Transportability, etc.
Future Parts of the Project
 Field Test



Assess usability among large, more representative sample
Assess impact of ballot designs on usability issues
Assess accuracy on different systems
 Natural Experiments


Assess impact of voting systems and ballot designs on
over-voting, under-voting, straight-party voting, and other
measures across jurisdictions and over time
Assess impact of changing from specific types of voting
systems (or ballots) to another system (or ballot)
Implications and Reflections
 Voter intention is the key goal
 Usability is as important as security
(and so is accuracy and accessibility as well as
affordability and durability)
 Being able to update interface is important
(i.e., certification may be interfering with usability)
 Ballot/machine combination important
(i.e., one size doesn’t fit all)
This talk available with vendor’s responses
www.capc.umd.edu