Click Control: Improving Mouse Interaction for People with Motor Impairments
Christopher Kwan, Isaac Paquette, John J. Magee, Paul Y. Lee, Margrit Betke
Boston University - Department of Computer Science - Image and Video Computing Group
Contact: ckwan@cs.bu.edu
Website: http://cs-people.bu.edu/ckwan/clickcontrol/
Introduction
o
Click Control Usage
Camera-based mouse-replacement systems, e.g. [1,2,3,4] allow users with
motor impairments to control the mouse pointer with head movements, but
have common difficulties:
A user with motor impairments using the Camera
Mouse [1] mouse-replacement system.
A user tracking his nose with Camera Mouse (Available at:
cameramouse.bu.edu and www.cameramouse.org)
Difficulty: Accidental clicking
o
If user is not attentive to clicking state of system
o
Or because of lack of precision in using the system
Difficulty: Simulating capabilities of a real computer mouse (e.g. rightclicking, double-clicking)
o
Systems may impose additional physical requirements (e.g. winking [3])
o
Interactions may be tiring or error prone (e.g. when using tools [5])
Click Control:
o
To address difficulties: Click Control – a tool that augments abilities of
camera-based mouse-replacement systems
o
Works with any system allowing control of mouse pointer movement
o
Provides visual feedback and allows cancellation of unintentional clicks
o
Simulates capabilities of computer mouse with gesture-based interface that
does not impose requirements on movement abilities
Click Control Features
Gesture form:
o
Serves as indicator that the mouse-replacement system is about to click
o
Gives the user visual feedback
o
Gives opportunity to cancel a click if unintended
Initial User Studies
User study 1: C – high school student with cerebral palsy
o
Had difficulties knowing when Camera Mouse system was in clicking state,
had trouble with unintentional clicks
o
C was able to use Click Control to improve Camera Mouse experience –
able to open files on cluttered desktop, manipulate online video
o
“I am not struggling as much as before. . . now it feels like a real mouse”
User study 2: Middle school students with cerebral palsy
o
In previous studies, students used Camera Mouse with custom applications
but were often interrupted by accidental clicks
o
Click Control alleviated accidental clicking problem
o
F (speech language pathologist) used analogy of Click Control like a
teacher asking “Are you sure?”
o
Students were able to use Click Control with F’s help but its usability could
be improved
A user study participant using Click Control with Camera Mouse to operate an on-screen keyboard program.
Ongoing Work
o
Using Click Control’s gesture form to double-click on an icon (under green
crosshair) on a crowded desktop.
Gestures:
o
Touching combination of buttons in a certain order (e.g. Left, Top, Middle)
o
Does not depend on specific trajectory, no constraints on movements user
must be able to make
o
Easy to perform, requiring only ability to move mouse pointer
o
Explicit – unlikely to be performed unintentionally
o
o
o
o
More user studies to see longer-term effect on interactions
Quantitative measurements of usage, e.g. effect on task completion time
and error rate
Verify compatibility with other mouse-replacement systems
Customization options for gestures
Improving it from feedback and making it available for download
Acknowledgments
o
We thank the participants of our studies and also Robin Berghaus, Danna Gurari, Fletcher Hietpas, Tessa
Skinner, and Ashwin Thangali. We also thank the reviewers for their insightful feedback. Funding was
provided by the NSF (HCC grants IIS-0910908, IIS-0855065, and IIS-0713229)
References
Double-clicking on the sunflower icon on the taskbar with the gesture form.
Dashed and dotted lines show two different mouse trajectories.
[1] M. Betke, J. Gips, and P. Fleming. The Camera Mouse: Visual tracking of body features to provide computer access for people with severe
disabilities. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 10(1):1–10, Mar. 2002.
[2] D. Gorodnichy, E. Dubrofsky, and A. A. Mohammad. Working with a computer hands-free using the Nouse Perceptual Vision Interface. In
Proceedings of the International Workshop on Video Processing and Recognition, VideoRec’07. NRC, May 2007.
[3] E. Missimer and M. Betke. Blink and wink detection for mouse pointer control. In Proceedings of the 3rd International Conference on
PErvasive Technologies Related to Assistive Environments, PETRA ’10, pages 23:1–23:8, New York, NY, USA, 2010. ACM.
[4] J. Varona, C. Manresa-Yee, and F. J. Perales. Hands-free vision-based interface for computer accessibility. J. Netw. Comput. Appl., 31:357–
374, November 2008.
[5] Tools for additional mouse functions: ClickAid (http://www.polital.com/ca), Point-N-Click (http://www.polital.com/pnc), Dwell Clicker
(http://www.sensorysoftware.com/dwellclicker.html)