DIS 2012:
Designing for Cognitive Limitations
Scott McCrickard
Virginia Tech
Clayton Lewis
University of Colorado, Boulder
Workshop overview
9.00-9.20
9.20-10.30
10.30-11.00
11.00-12.30
12.30-14.00
14.00-15.30
15.30-16.00
16.00-17.30
Workshop Introduction
Participant Introductions
Morning Break
Cognitive Walkthroughs
Lunch
Scenario Prototyping
Afternoon Break
Future Directions
What are cognitive limitations?


limitations that result in the user’s
performance on cognitive tasks falling
outside of a normal range
includes limitations from:



situation: driving, walking, multitasking,
interacting with others
disability: autism, ADD/ADHD, stroke, aphasia,
brain injury, Alzheimer's, dementia, …
may be temporary or long-lasting
About cognitive disabilities

difficulty with mental tasks






memory
attention
language
math
problem-solving (at various stages)
often accompanied by other disabilities



visual (low vision or blind)
hearing disabilities
physical disabilities
Designing for cognitive disabilities

address difficulty with mental tasks









avoid lengthy interactive processes
break interactions into separate pages
give reminders of progress (e.g., page 2 of 4)
explain how to fix errors
use visual cues to focus attention
support multimedia display and interaction
use language appropriate for the target audience
require minimal math skills, when possible
the same guidelines for cognitive limitations?
Capturing problems




Most “accepted” design knowledge doesn’t hold
when designing for cognitive limitations
We have been using claims to capture key
issues in designing for cognitive limitations
Claims always include a feature together with
potential upsides and downsides, and may
include other supporting material (pictures,
quotes, citations, etc.)
Claims are falsifiable hypotheses that are meant
to start debates, not serve as the final word
Example claim:
The use of smartphone technology to replace
dedicated accessibility devices
+ Accelerometers, gyroscopes and magnetometers offer
new input methods that can be more accessible than
traditional input devices
For people with sufficient neurological and motor ability, the accelerometers,
gyroscopes, etc. that are built into most modern smartphones offer the possibility for
spatial- and gesture-based interfaces (Li, Dearman & Truong, 2010)
— BUT, damage to a single multi-use device would have
multiple problems
As reported in (Kane et al., 2009) the independence gained from mobile devices has the
problem of putting the user at significant risk in the event of a failure. One blind
participant explained the decision to carry a dedicated GPS despite having a phone with
overlapping functionality: “If something happens to my phone, I'd still want to be able to
have my [GPS]. If my [GPS] breaks, I still want to be able to have my phone.”
Example
claim:
Peripheral interfaces for
cumbersome but
beneficial tasks (like
setting IM status)
+ can lead to neglected tasks being performed more
frequently (Hausen, 2012)
+ can be completed without direct focus by many people
after two weeks (Hausen, 2012)
- BUT may require training and/or a steep learning curve
(Hausen, 2012)
- BUT, it is difficult to assess effectiveness of peripheral
interactions
About the organizers

Scott McCrickard



Associate Professor of Computer Science and
member of the Center for HCI at Virginia Tech
research interests include design methods,
notification systems, divided attention situations
Clayton Lewis


Professor of Computer Science at the University
of Colorado, Boulder
Scientist in Residence at the Coleman Institute
for Cognitive Disabilities
Coleman Institute thrust:
shape mainstream technology
Participant introductions

Those with position papers:





Joshua Hailpern, HP Labs
Doris Hausen, University of Munich
Gavin Wood, Newcastle University
Young Seok Lee, Motorola Mobility
Others:



Justin Brockie, Therap Services
Margot Brereton, Queensland Univ. of Technology
Mathew Kipling, Newcastle University
Break


Reconvene at 11.00
Next up: cognitive walkthroughs of a camera
app on various Android devices

an expert review from the perspective of users
with cognitive limitations
The problem



Issues with using the camera app on various
Google Android devices (i.e., mobile phone,
small tablet, large tablet)
Consider goals of users with cognitive
limitations due to disabilities, use while
mobile, distraction, etc.
Consider different camera tasks, including
taking a picture, zooming, taking a blackand-white picture, emailing a picture, etc.
Cognitive walkthroughs



An analytic evaluation technique that focuses
on a target user population
Helps understand why errors take place
 identify opportunities for future design
To be applied to a common interface that is
often used by people with cognitive
limitations
Discussion

From each group:



Briefly summarize the cognitive limitation and task
sets that you considered, and any key findings
[How] did the cognitive walkthrough approach help
you identify key areas of concern?
From each participant:

What new knowledge did you gain from this
activity? What unsolved problems emerged? Can
you frame it in terms of a claim (UI feature,
upsides, downsides) that reflects the problem?
Product Testing Laboratory
Principal Investigator: Greg McGrew, MEBME
Usability Testing Program
User
Direct
Support
AT
The Usability Testing Program evaluates human factors issues pertinent to specific products through a
process of data collection on ‘real people’ performing ‘real tasks’. In essence, people with disabilities that
would normally use the type of product to be tested will be asked to perform tasks for which the product
was designed. These sessions, along with pre and post test interviews, are digitally recorded to gather
structured data and feedback from participants on the product. Usability metrics, including time on task,
errors, and satisfaction, are applied to the results.
The Testing Process
Formal usability testing involves capturing real users (people with disabilities or caregivers
that represent the target market for the AT or consumer product) using the product,
attempting to accomplish tasks for which the product was designed. The example at the
right shows a parent (participant) setting up a book reading product for their child’s use.
During the test this activity is recorded and then analyzed against an optimal performance
scenario. This enables us to identify problems, difficulties, and missteps made by the
participant, and analyze them to determine what design features or documentation may have
contributed to their cause.
Usability Metrics
From 5 to 20 participants are recruited for each product test session. This number depends on the
test objectives.
For each test, there is typically and optimal series of steps to complete it. Usability metrics are
applied to each test, comparing this optimal path to the one carried out by the participant. Metrics
include the number of errors, time-on-task, success/non-success, and efficiency. In addition, a
standardized measure of usability is applied through administration of the System Usability Survey
developed by IBM. Satisfaction is assessed through administration of a short post-test survey.
ACKNOWLEDGEMENTS: Funding is provided by the National Institute on Disability and Rehabilitation Research under the US Department of Education, Grant #H133E090003, the Coleman Institute for Cognitive Disabilities, the Department of Pediatrics and
the Colorado Intellectual and Developmental Disability Research Center, (IDDRC), University of Colorado.
Cognitive Walkthrough Method
Step 1


Pick a mobile device as an example
Work out how to take a picture and create a
scenario:




List the actions you must take, and…
…what is on the screen after each action, and…
…any other feedback, eg sounds
Since you have the phone in front of you, your
descriptions can be brief
Step 2

Critique each action





Will the user know what to try to do at this
point? Why?
Will the user know how to do it? Why?
Are there wrong actions that may look
correct?
If the correct action is taken, will the
feedback show this, or create doubt?
Very Important: This is not a user test!
Very Important!
This is not a user test!

Critique each action




Will the user [not you] know what to try to do at
this point? Why?
Will the user [not you] know how to do it? Why?
Are there wrong actions that may look correct [to
the user, not you]?
If the correct action is taken, will the feedback
show this, or create doubt [in the user, not you]?
Lunch



Follow Mat to lunch!
Reconvene here at 14.00
Next up: designing some prototypes
targeting people with cognitive limitations

we’ll use three techniques developed to
encourage the consideration of limitations
The problem



Issues with using the camera app on various
Google Android devices (i.e., mobile phone,
small tablet, large tablet)
Consider goals of users with cognitive
limitations due to disabilities, use while
mobile, distraction, etc.
Consider different camera tasks, including
taking a picture, zooming, taking a blackand-white picture, emailing a picture, etc.
Discussion

From each group:



Briefly summarize the cognitive limitation and task
sets that you considered, and any key findings
[How] did the cognitive walkthrough approach help
you identify key areas of concern?
From each participant:

What new knowledge did you gain from this
activity? What unsolved problems emerged? Can
you frame it in terms of a claim (UI feature,
upsides, downsides) that reflects the problem?
From problems to solutions

The first activity examined widely-used
interfaces through the perspective of a
person with a cognitive limitation
 a new perspective, new knowledge

The afternoon activity will apply our
collective knowledge to design problems
 deeper understanding of key questions,
appreciation for future directions
Prototyping tools

PIC-UP


Cognitive claims


a card set for notification systems that captures
key user interface claims (Wahid, 2011)
the set of claims extracted from the workshop
position papers (all of us, 2012)
Context cards

a card set that captures different user contexts
to be considered during design (haptimap.org)
Problem domains



Design a notification system for a senior
center, in which many of the target users
will have diminishing cognitive skills (PIC-UP)
Design a mobile application to help people
with speech loss from aphasia actively learn
to communicate (Cognitive Claims)
Design a mobile application to help a target
user population (of your choosing) navigate
around a university campus (Context Cards)
Break


Reconvene at 16.00
Next up: future directions



for the field (government initiatives, funding
opportunities, collaboration possibilities)
for each of us
for future workshops and other events
Discussion

From each group:



Briefly summarize the task sets you considered,
non-traditional populations, and any key findings
[How] did the card set help you identify key areas
of concern?
From each participant:

What new knowledge did you gain from this
activity? What unsolved problems emerged? Can
you frame it in terms of a claim (UI feature,
upsides, downsides) that reflects the problem?
Future directions



for each of us
for future workshops and other events
for the field (government initiatives, funding
opportunities, collaboration possibilities)
Profile-based presentation is a key aim of the Global
Public Inclusive Infrastructure initiative.
Gregg
Vanderheiden
Profile-based Presentation
Jutta Treviranus
OCAD University,
Toronto
The Fluid project is developing technology for
storing user preferences in user profiles
accessed from the cloud.
Profile-based Presentation
To respond to preferences when rendering
Web content, the approach uses a form of
Inversion of Control (IoC), enhanced to permit
specification of user preferences when services
are requested.
Colin
Clark
Antranig Basman
Future directions



for each of us
for future workshops and other events
for the field (government initiatives, funding
opportunities, collaboration possibilities)