Trinetra: Assistive Technologies for the Blind
Patrick E. Lanigan, Aaron M. Paulos & Andrew W. Williams, Priya Narasimhan
Motivation
Architecture
Store
End-User
Remote Server – TCP Proxy
UPC
1
Input
Handler
Text to
Speech
Request
Handler
Local
Cache
Database
Driver
HTTP
Handler
2
Local
Cache
6620 Application
Aisle
RFID Reading
Barcode Scanning
1. Bluetooth
2. Application Protocol (TCP)
3. Database Driver
4. HTTP (TCP)
Barcode for
Mustard
MUSTARD
518159 $0.99
Rack
RFID Shelf Label
for Mustard
A blind person walking into grocery needs assistance from the
people behind the counter to find items
Shelf
Robs a blind person of his/her independence
Would be valuable for a blind person to be able to walk into
grocery and purchase items without any assistance
Objective
Cost-effective, independent experience for the blind
Product
Current Implementation
 Commercial off-the-shelf products used by blind people
 Involve blind person’s input from concept to deployment
Network
Module
3
Local DB
(Pricing)
(RFID)
4
WWW
Database
(UPC)
Sequential processing
 Products are processed one at a time
Blocking requests
 Finish current processing before handling next product
Caching
 Cache frequently used products on the phone
 Mitigates weak connectivity over the Internet
Concurrency
 Incoming calls, other applications may interfere
 Assume no other applications are running
Results
Development Environment
Hardware
 Nokia 6620 smart phone
Bluetooth serial module, Baracoda Pencil/IDBlue
Software
 TALKS
 C++ language
 Symbian OS on Nokia phone
Existing infrastructure
 UPC on products, cellphone’s Internet capability
Initial set of experiments
 Prototype completed, works with various products in Entropy
 Enabled our blind human subject to discover a product without assistance
Initial usability studies with blind human subject
 Limitations of COTS device (lack of aural feedback, timeouts)
 Never underestimate a blind person’s sensory capabilities
http://www.ece.cmu.edu/~trinetra