Electronic Voting System
Usability Issues
Benjamin B. Bederson
Computer Science Department
Human-Computer Interaction Lab
University of Maryland, College Park
Bongshin Lee, HCIL
Robert M. Sherman, HCIL
Paul Herrnson, Dept. of Government & Politics, UMD
Richard G. Niemi, Political Science, Univ. of Rochester
www.cs.umd.edu/~bederson/voting
Voting System Importance
University of Maryland, Human-Computer Interaction Laboratory
 The manner in which voters cast
ballots is important
 Voting technology and ballot design
can influence:



election outcomes
voter’s trust
which affects future elections…
Voting System Technologies
University of Maryland, Human-Computer Interaction Laboratory
 Wide range of available systems
from:



paper and tally sheets
punch cards with “chad”
mechanical lever machines
 To more recent technologies:


optical-scan machines
Direct Recording Electronic - DREs
(touch-screens)
US Political Scene
University of Maryland, Human-Computer Interaction Laboratory
 After the US 2000 general election, there
was a flurry of activity

major problems





4-6 million presidential votes lost in 2000
hanging chad, etc.
see [CalTech/MIT Voting Technology Report, 2001]
public awareness of unequal voting systems,
widespread complaints, and new knowledge of
system limitations
calls for new systems
 So, how to make a judgement?
Other Studies
University of Maryland, Human-Computer Interaction Laboratory
 A range of research in this area:


Historical ballot design studies
Caltech/MIT Study
 But few investigations about usability
Learn from the Field of HCI
University of Maryland, Human-Computer Interaction Laboratory
 Electronic voting systems offer:




Control font size
Switch languages
Accommodate disabilities
Accurate and fast vote tabulations
 But have unique challenges:

Must work for everybody




including elderly, disabled, uneducated, etc.
Walk up and use (no required training)
No external help (although it is allowed)
Not frequently used
Our Study
University of Maryland, Human-Computer Interaction Laboratory
 Requested by four counties in
Maryland to examine their new
machines (Diebold AccuVote-TS).



30 days from first call to report
No funding
Machines to be used by 1,000,000
people in 2002 general election
 We agreed to perform:



Expert review (5 reviewers)
Close-up observation (47 users)
Field testing (415 users)
Study Screens
University of Maryland, Human-Computer Interaction Laboratory
Study Screens
University of Maryland, Human-Computer Interaction Laboratory
University of Maryland, Human-Computer Interaction Laboratory
University of Maryland, Human-Computer Interaction Laboratory
University of Maryland, Human-Computer Interaction Laboratory
University of Maryland, Human-Computer Interaction Laboratory
University of Maryland, Human-Computer Interaction Laboratory
Expert Review
University of Maryland, Human-Computer Interaction Laboratory
 Five HCIL faculty/staff analyzed system.
 Concerns:









Inconsistent terminology/labeling (5)
Color usage (4)
Inserting / removing card (4)
Help / Instructions (4)
Layout (long names, many candidates?) (4)
Screen glare (3)
Changing selection (2)
No overvote feedback (2)
Privacy (1)
Expert Review:
Audio-only System
University of Maryland, Human-Computer Interaction Laboratory
 Concerns:







Inappropriate keypad mapping (5)
Audio quality (5)
No ballot review (3)
No button feedback (2)
No overvote warning (2)
Must review entire ballot (2)
Volume control (1)
Close Observation
University of Maryland, Human-Computer Interaction Laboratory
 Observed 47 UMD members





Primarily students
Same “toy” election
Used “think aloud” protocol
Average time was 2 minutes, 10 sec
One participant couldn’t figure out how
to “write-in” a candidate
Close Observation Results
University of Maryland, Human-Computer Interaction Laboratory
 Overall rating (1-9, 9 being positive)


Overall comfort: 7.7
Screen layout and color: 6.9
 Representative comments:







Easy to use, straightforward
Excellent idea
Inserting card was very confusing
Is it reliable?
Colors are not well chosen
Font could be bigger
Ballot layout was confusing
Close Observation Comments
University of Maryland, Human-Computer Interaction Laboratory
 System Failure

One card got stuck in reader
 Card Insertion

Many participants had difficulty,
expecting ATM-style interaction
 Language Selection

Participants got stuck
 Undervotes not highlighted
Field Study
University of Maryland, Human-Computer Interaction Laboratory
 Great expectations:



Observation and recording of interaction
Questionnaire
Administer to representative sample
 Reality:



Study executed by election officials
No observation or notes on use
Sample from wealthier and more
educated districts
Field Study Results I
University of Maryland, Human-Computer Interaction Laboratory
 Ratings (1-9, 9 being positive)

Overall impression




Felt comfortable using system




8-9, 80%
7, 10%
1-6, 10%
8-9, 86%
7, 7%
1-6, 7%
Ease of reading text



8-9, 86%
7, 8%
1-6, 6%
Field Study Results II
University of Maryland, Human-Computer Interaction Laboratory
 Terminology used



8-9, 83%
7, 10%
1-6, 7%
 Correcting mistakes was easy



8-9, 81%
7, 11%
1-6, 8%
 Trusted votes were recorded properly



8-9, 85%
7, 7%
1-6, 8%
Field Study Interpretation
University of Maryland, Human-Computer Interaction Laboratory
 ~10% that rated 6 or lower
equals 383,000 voters
 City/suburban dwellers rated higher
 Women rated higher
 Frequent computer users had lower
trust
Diebold AccuVote-TS
Deployed at 2002 General Election
University of Maryland, Human-Computer Interaction Laboratory
University of Maryland, Human-Computer Interaction Laboratory
University of Maryland, Human-Computer Interaction Laboratory
University of Maryland, Human-Computer Interaction Laboratory
University of Maryland, Human-Computer Interaction Laboratory
University of Maryland, Human-Computer Interaction Laboratory
University of Maryland, Human-Computer Interaction Laboratory
Conclusion
University of Maryland, Human-Computer Interaction Laboratory
 Studies leave us optimistic, but concerned

With elections called by 1%, leaving 10%
unconfident voters is a problem
 The requirements of DREs are unique, but
the design issues aren’t




Typical of public access information systems
Need closer work with HCI professionals
Need qualitative and quantitative user studies
Need further field studies
 Come see voting panel tomorrow morning
www.cs.umd.edu/~bederson/voting