iCare-Reader - A Truly Portable Reading Device for the Blind
Terri Hedgpeth, Mike Rush PE, Vivek Iyer,
John Black, Mehmet Donderler, Sethuraman Panchanathan
Arizona State University
Tempe, AZ - 85287. USA.
email: terrih@asu.edu, mike.rush@asu.edu, vivek.iyer@asu.edu,
john.black@asu.edu, mehmet.donderler@asu.edu, panch@asu.edu
Abstract
According to the world health report of 1997, there are approximately ten million blind and visuallyimpaired persons in the United States [1], and forty five million blind individuals world-wide [2]. Though
there are many existing solutions to the problem of assisting individuals who are blind to read, however
none of them provide a reading experience that in any way parallels that of the sighted population. In
particular, there is a need for a portable text reader that is affordable and readily available to the blind
community. A truly portable text reader would allow the user to take the reader to the print, rather than
bringing the print to the reader. This paper describes our efforts to design such a device – the iCareReader.
1. Introduction
While there are many existing solutions that assist individuals who are blind with accessing print such as
Braille books, refreshable Braille devices, audio recordings, screen readers, and text scanners; none of
these provide a reading experience that parallels the ease with which sighted persons access print. For
example, these currently available technologies don’t allow the user to take a book from the shelf,
peruse the table of contents, and then flip through the pages of a book to find the desired page. Aside
from Braille and DAISY books, the user even finds it awkward to re-read difficult passages in a book, as is
often necessary when studying a textbook or a reference book. The goal of the iCare-Reader project is to
allow the readers who are blind or visually impaired to do all of these things.
2. Motivation
Though we rarely stop to think about it, sighted individuals are continually bombarded every day by the
printed word. Some of the sources of this abundance of print media in our environment include
transportation, advertising, news and commercial signs. Sighted individuals are continually exposed to
information from these and other sources, without even being consciously aware of this fact. However,
this is a phenomenon that people who are blind currently do not experience.
The workplace is another area in which the ability to read text is simply taken for granted. Staff
meetings and task groups often revolve around print and graphic based presentations from which an
employee who is blind is simply excluded from. Even the process of eating out is complicated by the fact
that few restaurants have menus in Braille.
All of these facts underscore the necessity for a portable text reader that is affordable and readily
available to the blind community. Such a device would go a long way towards filling this socially vital
need to freely access printed material. While several assistive devices have been developed in the past
to assist with the reading of printed text, they have all fallen short of the user expectations. In particular,
most have been too cumbersome and/or not readily available to be practical and truly portable. What is
needed is a portable reading device that would allow the user to take the reader to the print, instead of
having to bring the print to the reader. This is one of the important long-term goals of the iCare-Reader
project.
3. The Current Technologies for Book Reading
Before discussing our iCare-Reader, it is appropriate to review the existing methods that are being used
by people who are blind for reading. These include Braille books, audio recorded books and Optical
Character Recognition (OCR) based systems. Each of which has its advantages and disadvantages.
3.1. Braille Books
The primary advantage of Braille is that it allows users to read in their preferred manner such as
skimming the text, search for bold or highlighted text, headings, subheadings, paragraphs and check the
spelling of any name or word. However, their sheer bulk makes Braille books to cumbersome to store.
For example, the Webster’s Collegiate Dictionary in print is about 3 pounds, 3 inches thick and is only
one volume; but, the Braille copy of that same book is 75+ volumes each about 3-4 inches thick and
measures 12 x 12 inches square and it costs over 5 thousand dollars. So, although a large percentage of
sighted persons have a dictionary in their household, you can understand why it isn’t practical for a blind
individual to own their own copy. Another problem is the limited number of books available in Braille.
Due to the cost and large amount of room necessary for storage, there aren’t all that many Braille books
produced. Also, most of the written material encountered in day-to-day life, other than books, is
seldom converted into Braille. In the academic setting, this may include class handouts, homework
assignments, supplementary readings and exams.
3.2. Books in Audio Formats
Cassette tapes provide a means for storing and accessing audio recordings of information. However,
there are several disadvantages to the use of tape for reading books. First, tapes provide access to the
user in a strictly linear manner. If the user wants to start with the third chapter of a book, it is tedious
and time consuming to fast forward to the appropriate place on the tape. Second, if a particular passage
of the text is not fully understood on the first reading, the tape can be rewound. But the exact location
where the tape stops is not precisely controlled, so it’s difficult to backtrack a sentence or a phrase at a
time. Third, the reader can’t check spelling, as is possible when reading Braille. Fourth, the user has
virtually no access to the critical formatting elements such as bolding and italics, paragraph beginning
and ending or any other techniques that publishers use to emphasize specific terms or passages.
Then there’s mp3 recordings in the DAISY format, which allows the reader to navigate through a text by
word, line, sentence, paragraph, page and chapters, with the DAISY Consortium special Extensive
Markup Language. However, the user still cannot check spelling and there are still a very limited number
of text offerings when compared to the sheer volume of text available to sighted readers. Just consider
that a typical library is filled with thousands of books that have never been converted into any of these
highly specialized formats. Another major drawback is that Braille and audio recordings are conversion
processes that require the involvement of a sighted reader and special facilities to do the conversions.
3.3. Flat Bed Scanners and OCR software
One solution for accessing print is to use commercial off-the-shelf flat-bed scanners and OCR software. A
book is opened and placed face down on the scanner, allowing a page to be scanned. Then the OCR
software processes the image, producing a text file that can be then heard through the use of text-tospeech technology.
While this is a practical solution for scanning and reading fairly simple and small volumes of printed
material, it is inadequate for scanning math, science text and tables both simple and complex. Also the
user cannot easily read and/or reference two or more books simultaneously, which sighted students do
when they are studying or doing research. For example, it’s not hard to imagine the difficulties a student
encounters when told by his/her instructor to “read chapter 4” or “read pages 47 through 58” of the
textbook before the next class. Short of asking for assistance, the only option open to him/her is to
repeatedly leaf forward and backward through the book, scanning pages in search for the appropriate
page number or section.
3.4. Reading Common print
All of the solutions described above are intended to assist with the reading of hardcopy text in a
controlled environment. However, these solutions do nothing to provide access to much of the common
print that we all encounter in day-to-day situations, such as restaurant menus or vending machine
labels. In order to access this kind of text, it would be very helpful to have a wearable device that could
capture images of the environment, detect the presence of text, and then apply OCR technology to the
extracted text in real time.
4. The iCare-Reader
Ideally, a text reader should be wearable, so that the user can use it in any day-to-day situation. The text
reader hardware should also be unobtrusive, so that the user does not “look like a Martian” when
wearing it. It must not be power hungry, since it will be battery powered, and it must be adaptable to
environmental variations, such as lighting.
Such a reader is not hard to imagine. A tiny video camera could be mounted in a pair of stylish glasses,
and connected wirelessly to a Personal Data Assistant (PDA) sized device. This device processes the
video stream with OCR and text-to-speech software, and then synthesizes a voice that can be heard
through tiny sound emitters fixed to the ends of the leads on the glasses where they would fit neatly
and comfortably behind the ears, thus not impeding the users’ ability to hear. Unfortunately, today’s
miniature video camera technology is not yet able to deliver accurately auto-focused image streams that
are tolerant of lighting variations, and have enough spatial resolution to satisfy the needs of OCR
software. Rather than waiting for miniature camera technology to become available, we have chosen to
take a 3-phase approach to the development of our iCare-Reader.
The iCare-Reader is a device that can be used by people who are blind or visually-impaired to read
printed material, such as newspapers, magazines and books, in real time. It employs a collection of
commercial off-the-shelf hardware, including a video camera, OCR software, text-to-speech and voice
synthesis software, which is augmented and integrated with custom software to compensate for image
distortions and lighting variations.
4.1. Phase 1 - The Table-Based Book Reader
The first phase of the iCare-Reader project involves the development of a table-based book reader
prototype that is suitable for deployment in a library. Figure 1 shows our current prototype. The figure
shows a user seated at a desk-sized workstation, with a book open in front of him. A digital video
camera with a computer-controlled tilt and swivel mechanism is mounted above the desk. Buttons are
provided on the tabletop for controlling the book reader. A desktop computer controls the camera, and
responds to the user input.
Figure 1: The table-based book reader
When the user pushes a button instructing the book reader to read the open page in the book, the
camera, under the control of software running in a desktop computer, takes a snapshot of the table.
Image processing software then determines the exact position of the book on the table. Using the
resulting coordinates, the camera is zoomed in on the page to be read.
Several reading modes are provided. A “Table of Contents” mode can be used to read the titles of the
chapters, and the page numbers where each chapter begins. A similar “index mode” can be used to read
the index at the back of a book. When the user determines the desired page number, a “page number”
mode recites the page numbers as the user turns the pages of the book. When the desired page is
reached, the “Text” mode is used to read the text on that page. A special “finger tracking” mode allows
the user to guide the reader, backtracking when a difficult passage must be re-read.
The table-top book reader is intended for deployment in libraries, and additional prototypes are being
constructed for deployment in the various on-campus libraries at Arizona State University. Hopefully
other libraries will also be equipped with these book readers in the future. However, people who are
blind will not always be able to travel to a library to read books. For this reason, Phase 2 involves the
design of a portable book reader.
4.2. Phase 2 - The Portable Book Reader
It is anticipated that in the near future, greatly advanced and highly sophisticated smaller cameras and
computer technologies will allow the development of a portable book reader prototype, which can be
carried in a briefcase. The user would then be able to place this briefcase on a table at a location of
his/her choice and read a book. It is anticipated that the portable book reader will utilize a similar tiltand-swivel camera as in the table-top-book reader, though the portable system will be much smaller
and lightweight for true portability. The advancement into Phase 2 is primarily dependent upon the
development of the necessary smaller camera and computer hardware.
4.3. Phase 3 - The Wearable Text Reader
Phase 3 involves the development of a lightweight wearable text reader, which can be used for reading
common print, such as signage, vending machines and hardcopy text. This text reader would consist of a
miniature video camera in a pair of glasses and a belt-mounted PDA-sized computer, as describe
previously. As in the case of the portable book reader, this prototype must await advances in
technology.
However, there is one significant research problem that will need to be overcome in order to make this
approach practical. Since the camera will be aimed by head motions, instead of a computer-controlled
tilt-and-swivel mechanism, there will be a need for an interaction protocol between the computer and
the user, in order to aim the camera at the text to be read.
(a)
(b)
Figure 2: Developing a Human Computer Interface
In order to begin research in this area, we have constructed some head-mounted camera prototypes,
using commercial Firewire digital video cameras. Figure 2(a) shows one of these prototypes in use. Here
a sighted user wearing the head-mounted camera has been blindfolded to simulate blindness, but is
able to communicate verbally with a sighted person at a remote location through a headset. Figure 2(b)
shows that remotely located person seated in front of a video monitor, which displays the video stream
that is being captured by the head-mounted camera.
These two people then collaborate verbally, via their 2-way voice link, to aim the camera at the book
and to read the contents of the page. As they collaborate to achieve this task, the video and the audio
streams for the entire session are recorded and archived for later study. The goal of this experiment is to
learn what types of interactions are used between pairs of humans to aim and focus the camera at the
book and to navigate through the book to find the desired content. These interactions can then be
formalized into protocols, and the human in front of the video monitor can be replaced with a computer
system that implements these same protocols with voice synthesis and voice recognition devices.
5. Conclusion
The iCare-Reader will ultimately provide people who are blind with a level of access to print that
information that will be more like that of sighted people than other assistive devices have been able to
achieve. While the user interaction protocols are still being developed, using feedback from users who
are blind, it is already clear that this approach has significant advantages over the current flatbed
scanner approach to book reading. While the transition to the smaller portable “briefcase” hardware
awaits the advancement of technology, there isn’t any other obvious impediment to the development of
phase 2. However, the transition to a wearable device will present significant new challenges.
Specifically, it will require a much more collaborative interface between the user and the reader, to
allow the reader to “coach” the user in aiming the camera at the reading material by head movements.
Experiments that are currently underway are being used to develop the interaction protocols that are
necessary for accomplishing this complex task.
6. References
[1] http://www.the-fbc.org , Foundation for Blind Children.
[2] http://www.who.int/whr2001/2001/archives/1997/exsum97e.htm World Health Report 1997
Executive Summary
[3] http://www.w3c.org World Wide Web Consortium w3c.
[4] http://www.DAISYconsortium.org Digital Access to Information System.
[5] http://www.afb.org American Foundation for the Blind.