ASLLENGE
He Zhou
Hui Zheng
William Mai
Xiang Guo
Advisor: Professor Patrick Kelly
Department of Electrical and Computer Engineering
Outline








Background
System Requirements
Design Challenges
System Overview
Hardware and Software
Team Roles
Timeline
Q/A
Department of Electrical and Computer Engineering
2
Background
 Leverage Kinect technology to recognize and
offer feedback on American Sign Language
 ASL is the one of most commonly used language
in US(.5 to 2 million population use ASL as their
primary form of communication)
 Educational learning tool for ASL
Department of Electrical and Computer Engineering
3
Motivation
 Helping hearing impaired people to learn ASL.
 Facilitate communication between ASL speaker
and non-speaker
Department of Electrical and Computer Engineering
4
System Requirements
 Create a user-friendly interface that recognize
sign language and offer feedback
• Software must be easy to use
 Display translated word or phase being
demonstrated by user
• Provide quality feedback to user concerning
the accuracy of ASL
 Cost should be within the budget
Department of Electrical and Computer Engineering
5
Design Challenges




Human Factors
Background Interferences
Finger Gestures
Hardware limitation
• Kinect only has 320x240 resolution and 16 bit
depth
Department of Electrical and Computer Engineering
6
Kinect
 Has a depth and regular Camera
 Associated with software development kit
• Able to track a human being.
• Use computer generated algorithms
to form human skeleton data points
 Track upper body movements
Department of Electrical and Computer Engineering
7
System Overview
Department of Electrical and Computer Engineering
8
Video demonstration(Hand Rotation)
Department of Electrical and Computer Engineering
9
Video demonstration(Hand Crossing)
Department of Electrical and Computer Engineering
10
Video demonstration(Hand behind Head)
Department of Electrical and Computer Engineering
11
Video demonstration(Hand behind Back)
Department of Electrical and Computer Engineering
12
Hardware Implementation 1
 Increasing the flex sensor
output voltage with bend
 Voltage output goes into
an AD converter board
 AD converter digitizes the
output signal
 PC captures the signal via
USB or Bluetooth
Department of Electrical and Computer Engineering
13
Hardware Implementation 2
 Kinect Camera captures the raw image and the
depth image data
 The data send to PC to be processed into skeleton
tracker
 Combining the signal from flex sensor and search
in database for matching motions
Department of Electrical and Computer Engineering
14
Software Design Flow
Department of Electrical and Computer Engineering
15
Development Model
Department of Electrical and Computer Engineering
16
Cost of Major Components
Name of
Component
Flex
Sensor
A/D
Converter
Board
A Pair of
Gloves
Kinect
Sensor
Bluetooth
Module
Quantities
10
1
1
1
1
Unit Price
$12
$50
$10
$150
$20
Total Price
Department of Electrical and Computer Engineering
$350
17
Team Roles
 He - test and integrate software prototype
 Hui- image processing and programming
 William - build microcontroller to detect finger
and palm movements
 Xiang- website, overall testing and building ASL
database
Department of Electrical and Computer Engineering
18
MDR deliverables
The goal for our MDR is to be able to detect and
track arm and hand positions, possibly be able to
translate simple hand gestures
Department of Electrical and Computer Engineering
19
Conclusion
Our project is good for SDP:
• Contains both hardware and software
• Expandable: database, audio recognition, braile
Our project is a good product:
• Aid the disabled people
• Personal/commercial use
• Relatively inexpensive
Department of Electrical and Computer Engineering
20
Timeline
10/16/2011
12/5/2011
1/24/2012
3/14/2012
5/3/2012
PDR
Purchase Parts
Planning
GUI
Interface with the Sensors
Build Finger Sensors
Build Test Database
Prototype 1
MileStone MDR Deliverables
System Integration
Winter Break
Expand Database
Prototype 2
MileStone CDR Deliverables
Alpha Testing
Update
Beta Testing
Spring Break
Finalize Project
MileStone FRP Deliverables
Department of Electrical and Computer Engineering
21
Q&A
Department of Electrical and Computer Engineering
22