S M I LE © (Science and Math in an Immersive Learning Environment)
Project Overview
• A new immersive 3D learning environment to increase
math/science skills of deaf and hearing children
• The application teaches math/science concepts and ASL
(American Sign Language) math terminology through user
interaction with fantasy 3D virtual signers and
environments
• It can be displayed in spatial immersive devices (SID) such
as the FLEX [1] or CAVE [2], and includes a gesture control
system
•
•
[1] Fakespace Systems, FLEX http://www.fakespace.com/flexReflex.htm
[2] The CAVE: a VR theater http://www.evl.uic.edu/pape/CAVE/oldCAVE/CAVE.html
S M I LE © (Science and Math in an Immersive Learning Environment)
Project Sponsors
National Science Foundation -NSF-RDE. "Software for
Math Education of the Deaf". (grant#0622900)
PHS-NIH Nat. Inst. Deafness and Comm. Disorders.
“Modeling the non-manuals of American Sign Language”
(grant#511 1419 -1641)
The College of Technology at Purdue University -- I3 grant
(grant #00006585)
The Envision Center For Data Perceptualization
S M I LE © (Science and Math in an Immersive Learning Environment)
Research Team
PIs
Prof. Nicoletta Adamo-Villani, Department of Computer Graphics Technology
Prof. Ronnie Wilbur, Department of Audiology and Speech Sciences and Chair
of Linguistics
DEVELOPMENT
Robin Wright, ASL signer
David Jones, graduate student in Computer Graphics Technology
Edward Carpenter, graduate student in Computer Graphics Technology
Kelly Wright, senior in CGT
Jim Gilchrist, senior in CGT
Melissa Walsh, senior in CGT
Jac Piette, senior in CGT
COLLABORATORS
The students, teachers, and parents at The Indiana School for the Deaf (ISD)
Dr. Laura Arns, Associate Director of the Envision Center for Data
Perceptualization
S M I LE © (Science and Math in an Immersive Learning Environment)
Why this project ?
• Deaf individuals are very underrepresented in the fields of
science and engineering. Several factors contribute to this
disparity:
1. Difficulty in acquiring math concepts early in life
2. Significant delay in deaf children’s reading
comprehension
3. Inaccessibility to incidental learning (exposure to
media in which mathematical concepts are practiced
and reinforced)
4 . Difficulty of hearing parents (73% deaf children have
hearing parents) in conveying basic science and
mathematical concepts in sign language
S M I LE © (Science and Math in an Immersive Learning Environment)
Why VR as the technology of choice ?
• Existing data suggest that VR technology offers significant,
positive support for education in general
• Research shows that VR is particularly suitable to math and
science education
• In regard to disabilities education, literature findings suggest
that VR has advantages over other teaching technologies
because it can fulfill the majority of the learning requirements of
students with disabilities
S M I LE © (Science and Math in an Immersive Learning Environment)
Existing VR applications for the Hearing Impaired
• ‘Virtual Supermarket’ developed at the University of
Nottingham, England
• VREAL (Virtual Reality Education for Assisted Living)
project, funded by the U.S. Department of Education
S M I LE © (Science and Math in an Immersive Learning Environment)
User Interface Presentation Components
S M I LE © (Science and Math in an Immersive Learning Environment)
User Interface Input Mechanisms
1.
2.
A gesture control system, comprised of a pair of
pinch gloves and a wrist tracker, allows the user
to: (1) grasp and release virtual objects; (2) input a
limited number of ASL hand-shapes; and (3)
navigate the virtual environment.
or
A wand allows the user to: (1) grasp and release
virtual objects, and (2) navigate the virtual
environment.
S M I LE © (Science and Math in an Immersive Learning Environment)
Software and System Architecture
•
•
•
•
•
•
•
•
Maya 7.0
3D Studio Max
Motion Builder
OpenGL
OpenSceneGraph
VRJuggler toolkit
OsgCal
Cal3D
S M I LE © (Science and Math in an Immersive Learning Environment)
Innovative Features
• Seamless polygonal characters
• Realistic deformation
• Organic appearance
• Realistic and fluid signing motion
Motion is captured directly from non-hearing signers and
smoothly blended in real time
• Complex interactivity
S M I LE © (Science and Math in an Immersive Learning Environment)
Character design
S M I LE © (Science and Math in an Immersive Learning Environment)
Character Rigging
Rigs
S M I LE © (Science and Math in an Immersive Learning Environment)
Signing Animation
Animation
S M I LE © (Science and Math in an Immersive Learning Environment)
Environment Design: the Clock Store
S M I LE © (Science and Math in an Immersive Learning Environment)
Environment Design: the Bakery
S M I LE © (Science and Math in an Immersive Learning Environment)
Environment Design: the Candy Store
S M I LE © (Science and Math in an Immersive Learning Environment)
Interactive content
S M I LE © (Science and Math in an Immersive Learning Environment)
Evaluation Methodology
• User task analysis
• Expert panel evaluation
(1) usability of the program
(2) overall quality of the virtual world
(3) quality of the signing motion
• Formative evaluation
(1) usability of the program
(2) appeal/fun
• Summative evaluation (ISD, Fall 2007)
(1) Learning Outcomes
S M I LE © (Science and Math in an Immersive Learning Environment)
Evaluation Results
•
•
•
•
•
Minor Usability Problems
Need to improve the gesture control system
Need to include alternative travel methods
Need to include way-finding techniques
Suggested use of narrative as a key element
to increase the users’ level of engagement
S M I LE © (Science and Math in an Immersive Learning Environment)
QUESTIONS?
Visit the project website at: http://www2.tech.purdue.edu/cgt/i3/smile/
Read about SMILE in the Animation World Magazine: http://mag.awn.com/?article_no=3015