Human Computer Interaction Institute
School of Computer Science
Carnegie Mellon University
Using Handhelds to Enhance
Classrooms and to Help the
Handicapped
Brad A. Myers
Carnegie Mellon University
bam@cs.cmu.edu
http://www.cs.cmu.edu/~pebbles
SUNY Technology Conference 2002
Overview in Previous Session

Pebbles project:


How handhelds can be used
at the same time as other devices
Many domains, applications and research
topics
SUNY Technology Conference 2002
Brad Myers
2
Overview of This Talk

Pebbles applications for classrooms

Slideshow Commander for the presenter



Or RemoteCommander or Shortcutter
Using handhelds for real-time “Concept Tests”
Pebbles applications for the Handicapped

Handhelds as an assistive technology for people
with Muscular Dystrophy and related disorders
SUNY Technology Conference 2002
Brad Myers
3
Presenter Use

Slideshow Commander for the Presenter
SlideShow Commander

For PowerPoint


Use PC to give the presentation


Full features of PowerPoint
Use hand-held as “remote control” for PC



PC Only: Windows 95, 98, NT, 2000, ME, XP
Can wander away from the keyboard
Two-way communication
View, Navigate, Highlight, Time
SUNY Technology Conference 2002
Brad Myers
5
Studies of Presentations


Summer study of 2 HCI Master’s students
Contextual Inquiry of 9 presentations



Found 220 “breakdowns”



Audience: 10 to hundreds
7 used PowerPoint, 4 used NetMeeting
Most were minor problems
Averaged 8.7 lost minutes per talk (14.5%)
Designed SlideShow Commander to
eliminate some of these problems
SUNY Technology Conference 2002
Brad Myers
6
View on Hand-Held: slide

See thumb-nail of current slide

Black and white or color
SUNY Technology Conference 2002
Brad Myers
7
Scribble on Current Slide

Can handwrite and draw on thumbnail picture


Like with overheads
Or check off items as you go
SUNY Technology Conference 2002
Brad Myers
8
New! Tap On-screen Buttons


Mode on handheld switches between
scribble and tap
Invoke sounds, animations, etc.
SUNY Technology Conference 2002
Brad Myers
9
New! Preview forward or backwards

iPaq version only

See what is coming next on
the handheld without
showing students

Or look back at a previous
slide

Jump to that slide without
showing ones in between
SUNY Technology Conference 2002
Brad Myers
10
View on Hand-Held: notes

See the notes of the current slide
SUNY Technology Conference 2002
Brad Myers
11
View on Hand-Held: titles

See the list of titles
SUNY Technology Conference 2002
Brad Myers
12
View on handheld: Time




Palm version only
Multi-function timer
View large
And in corner
SUNY Technology Conference 2002
Brad Myers
13
Control Demonstrations on PocketPC



Task list of PC applications
Tap brings one to front
PowerPoint continues
behind – easy to resume
SUNY Technology Conference 2002
Brad Myers
14
Control Demonstrations on Palm


Control PC applications and external devices
Uses Shortcutter



Create panels of controls for any app
Also can control lights and projector
Easy demo and resume show
SUNY Technology Conference 2002
Brad Myers
15
Commercialized

SlideShow Commander
commercialized by:
Synergy Solutions, Inc.
http://www.synsolutions.com/


Available from Office Depot,
CompUSA, etc.
Palm and PocketPC

(Some demonstrated features not available in
commercial version)
http://www.slideshowcommander.com/
SUNY Technology Conference 2002
Brad Myers
16
But what if not using PowerPoint?


Can use RemoteCommander to control any PC
application
Can use Shortcutter to send specific keystrokes


One professor uses Shortcutter to control Adobe Acrobat
Still only PC
SUNY Technology Conference 2002
Brad Myers
17
Student Use of Handhelds

Using handheld computers in classrooms



For testing
Improve large lectures
Provide immediate feedback to instructor
Hardware

HP donated 110 Jornada 680 and 100 720
computers


Windows CE
Lucent donated Wavelan wireless cards
SUNY Technology Conference 2002
Brad Myers
19
Context


Collaborator: Prof. David Yaron of Chemistry
Chemistry 09-106: Modern Chemistry II




Spring, 2000
About 90 students
Mostly freshmen
Loaned one Jornada and Wavelan card to
each student for the whole semester


So would get used to using it
Offset technical difficulties with benefits to student
SUNY Technology Conference 2002
Brad Myers
20
Wireless Andrew


CMU has almost complete coverage of
802.11b support
Wavelan (now ORiNOCO)



Can use computers wirelessly in all
classrooms and offices


Donations from Lucent, etc.
Funding from state of Pennsylvania, etc.
Most lawns, dorm rooms, etc.
Windows, Mac, Linux, Windows CE support
SUNY Technology Conference 2002
Brad Myers
21
Testing


Large lectures are a poor way to educate
Work on engaging students


So they pay attention and think about material
“Concept Tests”



Short tests given during lectures
Students discuss, and answer questions again
Evidence that significantly contributes to learning
— Mazur, E. (1997). Peer Instruction:
A User's Manual, Prentice Hall.
SUNY Technology Conference 2002
Brad Myers
22
Concept Tests




Students have to think about answers
Instructor can determine who is participating
Can determine what students understand
Computerize:


Use technology to provide capabilities not
otherwise available
Rather than just a new way to do old things
SUNY Technology Conference 2002
Brad Myers
23
Implementation




Use web-based tests for students
Web-based instructor’s display
Hand-helds and instructor’s computer use
standard IE browser
Server provides database of tests



Currently only multiple-choice
Server also stores students’ answers
Tests represented as XML
SUNY Technology Conference 2002
Brad Myers
24
Pebbles Tests


Create through html pages
Arbitrary html in questions
and answers



Pictures, formatting, links
Use FrontPage, etc. to
author
Embed tags to show
question and answers
SUNY Technology Conference 2002
Brad Myers
25
Concept Test

Students allowed to answer multiple times



Server stores all answers
Prof. Yaron displayed
questions and answers on
the board and didn’t
author in html
Used for lectures and also
during in-class chemistry
demonstrations and
experiments
SUNY Technology Conference 2002
Brad Myers
26
Instructor’s View

In class, instructor sees a chart of answers



Optionally refreshes every 3 seconds
May be
projected for
whole class to
see
Top displays
instructions for
students
SUNY Technology Conference 2002
Brad Myers
27
For Instructor



Faculty can get data from
prior tests
Enables grading,
trend analysis
By student name,
or anonymous

SUNY Technology Conference 2002
Brad Myers
28
Results



Unfortunately, no data about effects on
learning
Students thought concept tests were a
valuable part of the class
Students significantly preferred using
handhelds for concept tests.
Raise Hands Hold up cards Handhelds
25%
4%
63%
SUNY Technology Conference 2002
Brad Myers
29
Problems


Connection to
network worked?
How often
needed to reset?
SUNY Technology Conference 2002
Always
Sometimes
Never
15%
67%
4%
Never
4%
1-2 times 3-5 times > 5 tlmes
23%
19%
42%
Brad Myers
30
Problems, cont.

Biggest drawback:
Crashes too often
Other…
Not powerful enough
Too big
No major drawbacks
27%
23%
15%
12%
10%
Keyboard was:

Adequate
Too small
Prefer no
keyboard
15%
67%
4%
Screen size was:

Adequate
15%
SUNY Technology Conference 2002
Too small Want smaller
device
67%
4%
Brad Myers
31
Future
Hope for further study of impact of handhelds
 This software is available for you to download
off the main Pebbles web page:
http://www.pebbles.hcii.cmu.edu/classroom/



webserversoftware
Written in Perl5
SUNY Technology Conference 2002
Brad Myers
32
Handhelds for the Handicapped

Handhelds as Assistive Technology

As an alternative input device for a computer
History

Drew Rossman found
RemoteCommander
on the web



Found it helped his
daughter who has a form
of Muscular Dystrophy known as
Spinal Muscular Atrophy, Type II.
Wrote articles and created web
page to help promote its use
We adapted the software to try to
improve performance
SUNY Technology Conference 2002
Brad Myers
34
Muscular Dystrophy

About 250,000 people in the United States have
Muscular Dystrophy (MD)


Use computers for recreation, education, etc.
Often home-bound, so computers may be even-more
important



Increased susceptibility to infections
Web provides opportunities for employment
People with MD often have fine-motor control but
lose gross motor control


Difficulty with mouse and keyboard, but stylus OK
May tire easily, so having choice of different input devices
can be useful
SUNY Technology Conference 2002
Brad Myers
35
Alternative Input Devices

Commercial alternative input devices can be
expensive and difficult to use


Must move from keyboard device to trackball
device
So use inexpensive
Palm instead
SUNY Technology Conference 2002
Brad Myers
36
Adaptations to Remote Commander

Made on-screen buttons for left, right mouse
buttons and keyboard modifiers

Physical Palm buttons too difficult to press
SUNY Technology Conference 2002
Brad Myers
37
Adaptations, cont.

Turn off key repeat


Faster Acceleration


Smaller movements on Palm
More flexible tapping


So not repeating by accident
To get mouse button down
Long time before power-off

Palm power button too hard to press
SUNY Technology Conference 2002
Brad Myers
38
Adaptations, cont.



Various Keyboard Layouts
Bigger buttons
Alphabetic layout

Most common keys grouped
together to minimize travel
distance
SUNY Technology Conference 2002
Brad Myers
39
New! Word Prediction



Predicts word from first letters
Predicts next word from previous word
Layout to minimize distance from keys


Most likely choice in center and highlighted
Available with soft keyboard and Graffiti
SUNY Technology Conference 2002
Brad Myers
40
Case Studies


Observed four people with MD
using Remote Commander
Measured typing and mouse
speed with PC-based tests
SUNY Technology Conference 2002
Brad Myers
41
Observations

Design changes:





Positives:





Many of the design decisions already mentioned
Needed lighter & longer stylus
Needed better lighting
Cradle not sufficiently stable: needed long cord;
Tiny keyboard requiring small movements
Keyboard and mouse in one place
Ability to control PC from across room
Ability to work with regular keyboard and mouse, for shared
computers
Problems:


Battery life is a real barrier
Still much slower than regular keyboard
SUNY Technology Conference 2002
Brad Myers
42
Future work

Handhelds as assistive technologies in other
areas



Handhelds as interface to other devices
Control room lights, telephone, wheelchair, etc.?
Handhelds to help with other disabilities

For the visually impaired?



SlideShow Commander
Auditory interface to PC apps, and the environment?
Ideas welcome!
SUNY Technology Conference 2002
Brad Myers
43
Thanks to Our Sponsors!
Supported by grants from:
DARPA
Microsoft
Pittsburgh Digital Greenhouse
NSF
General Motors
And equipment grants from:
Hewlett Packard
Lucent Technologies
Palm Computing
Symbol Technologies
SUNY Technology Conference 2002
IBM
SMART Technologies, Inc.
Synergy Solutions, Inc.
Handango
Brad Myers
44
Human Computer Interaction Institute
School of Computer Science
Carnegie Mellon University
Using Handhelds to Enhance
Classrooms and to Help the
Handicapped
Brad A. Myers
Carnegie Mellon University
bam@cs.cmu.edu
http://www.cs.cmu.edu/~pebbles
SUNY Technology Conference 2002