Current R&D at the NTID Center on
Access Technology
E. William Clymer
November 18, 2010
1
Center on Access Technology
www.rit.edu/ntid/cat
Rochester Institute of Technology
National Technical Institute for the Deaf
52 Lomb Memorial Drive
Rochester, NY 14623
2
Recent & Ongoing Access Technology
Projects
•
•
•
•
•
•
NSF Enrichment
CAT Innovation Lab and Director
PDA Projects
Cisco Grant – Three Strands
Cisco Evaluation – Desktop Systems
NTID Technology Symposium
3
NTID’s Location
Lyndon Baines Johnson Building (LBJ)
Located in Rochester, New York
The RIT Campus
•
Image: Map of Northeast USA showing Rochester, NY and RIT and NTID campus drawings.
4
Rochester Institute of Technology
CAST
COB
College of
Business
College of
Applied
Science &
Technology
CIAS
COE
GCCIS
College of
Engineering
Golisano College
of Computer &
Information Sciences
College of
Imaging Arts &
Sciences
CLA
COS
College of
Liberal Arts
NTID
National
Technical
Institute
for the Deaf
College of
Science
The Eight Colleges of RIT
≈17,000 Students
5
Rochester Institute of Technology
CAST
COB
COE
CIAS
CLA
COS
NTID
Supports and Educates
1,300
Deaf Students
6
Rochester Institute of Technology
CAST
Transfer Programs
COB
COE
118 Students
CIAS
CLA
COS
(11%)
NTID
Supports and Educates
Bachelors & Masters
Degrees
454 Students
1,300
Deaf Students
(41%)
7
Rochester Institute of Technology
CAST
COB
College of
Business
College of
Applied
Science &
Technology
CIAS
COE
GCCIS
COS
College of
Liberal Arts
College of
Engineering
Golisano College
of Computer &
Information Sciences
CLA
College of
Imaging Arts &
Sciences
NTID
NTID
National
Technical
Institute
for the Deaf
College of
Science
Direct Service Model
•Teachers Sign
•Teach Students Directly
8
Rochester Institute of Technology
CAST
COB
GCCIS
COE
CIAS
CLA
COS
NTID
Support Model
For Bachelor’s and Master’s Degree Students
•Access Services Provided
•Academic Support Provided
9
Access Services
•
•
•
•
•
Interpreting
Captioning (C-Print)
Notetaking
Tutoring
Counseling
10
Interpreting Services
 Interpreting services provided by:
–
–
–
–
122 FTE staff interpreters (October 2010)
Freelance interpreters
Deaf interpreters for Deaf-blind students
Students from interpreter training program
11
Classroom Interpreting
12
Notetaking




Notetaker coordinators
Trained student notetakers (paid)
Notes are scanned to the web
Support faculty evaluate the content matter of
the notes
13
Classroom Notetaking
14
Classes
Web Distribution of Notes
Scanning
Center
Web
Site
15
Captioning
• Captioning Options
– Speech to Text Systems
– C-Print
– CART
• 50 C-Print operators at
RIT
• CART is outsourced
http://www.pepnet.org/fa
q/pdf/FAQ04.pdf
16
Access Services
http://myaccess.rit.edu
17
Course Listings
Interpreter, Notes and C-print
18
Manager
19
Interpreter Schedule
20
Student Download
Notes and Captions
21
C-Print Captions File
22
Example of Notes and Captions
23
Mainstreamed Students &
Support Services
Hours of Service
AY
2008-09
#Students
Interpreting
Notetaking
Academic
Support
635+
110,000
63,290
17,636
C-Print
17,330
24
Center on Access Technology
www.rit.edu/ntid/cat
Rochester Institute of Technology
National Technical Institute for the Deaf
52 Lomb Memorial Drive
Rochester, NY 14623
25
Mission
To investigate, evaluate, and report on the most effective and efficient use of access
technologies
To train individuals in their use in order to accelerate the widespread implementation
of best practices within deaf education at the postsecondary level.
To focus its efforts on technologies that have a high likelihood of improving access to
postsecondary educational opportunities for deaf students within the next several
years.
Four strands include:
–
–
–
–
Classroom Access Technologies
Mobile Technologies
Audio and Sound Technologies of Interest to Hard-of-Hearing Persons
Training and Evaluation Services
26
Mission
The Center seeks projects that fall within the
following parameters:
– Those that can adapt/ adopt existing technologies
– Those that utilize existing professional networks
– Those that involve education and training
– Those that bring together faculty, staff, students
and others at NTID, RIT and external groups
27
NSF Enrichment Grant
• Testing the concept of a virtual Alliance for
deaf and hard of hearing STEM students at the
postsecondary level
28
Specifics
• NSF Award # HRD-0927586
• 18 Months
– 9/1/2010 through 2/28/2011
• $199,585
• http://www.rit.edu/ntid/cat/enrichment
29
Team
•
•
•
•
•
•
•
•
•
Bill Clymer
Jim DeCaro
Peter Lalley
Gary Behm
Gail Hyde
Elizabeth Lawley
Walter, Foster
McKee
Lang, Brooks, Ladner, Gan
30
Goal
• Prepare for an NSF Alliance grant proposal in
the amount of $3M due Feb. 2, 2011.
• Purpose of an Alliance….
Alliances use evidence-based practices, as well as
beta-testing novel interventions, to increase the
quality and quantity of students with disabilities
completing associate, baccalaureate and graduate
degrees in STEM disciplines and entering either
STEM graduate degree programs or our nation’s
science and engineering workforce.
31
Concept
• Use cyberinfrastructure to establish a “virtual”
RDE regional alliance to provide student
support to deaf and hard of hearing STEM
students.
– Communication Support
• Interpreting and captioning
– Academic Support
• Tutoring
• Notetaking
– Advising and mentoring
32
Communication Support
Communication Support
Academic Community
33
Specifically this grant will…
• Examine national baseline data on deaf students
in STEM programs
• Analyze programs that have deaf students
enrolled in STEM programs
• Conduct 4 focus groups
• Indentify 4-5 postsecondary institutions as
potential partners
• Identify elements of successful Alliances
• Develop and pilot rudimentary
cyberinfrastructure system for a virtual alliance
34
Expected Work Products
•
•
•
•
Summary report of baseline data
Annotated list of programs
Recommendation report from focus groups
List of 4 or 5 postsecondary programs to
participate
• Analysis and summary of features, format,
participants and budget for a virtual alliance
• Target plans for an alliance
• Functional model of a web-based application for
communication and academic support
35
Time Line
• Year 1 (12 Months)
– Project website
– Data search
– D/HOH STEM students
– Focus groups – PEPNet
– Review best practices
– Review platforms for delivery of services
36
Time Line
• Year 2 (6 Months)
– Identify potential partners
– Social network (academic community) prototype
– Dissemination
– Write and submit Alliance proposal
37
Evaluation Plan
• Dr. Barbara McKee
• Formative evaluation
– Timely and accurate completion of project goals
• Compliance with RIT and NSF reporting
guidelines
• Summative evaluation
38
Dissemination
• Web site
– http://www.rit.edu/ntid/cat/enrichment
• National presentations
– NTID Techsym (June 2010)
– NSF JAM (June 2010)
– Accessing Higher Ground (November 2010)
39
Budget
• Original Budget
Awarded
Spent
Personnel
98,877
46,069
Benefits
23,515
12,988
Travel
6,400
1,788
Materials, etc
13,261
2,439
• Percentage Spent
– 51.4%
• One year no-cost extension?
40
CAT Innovation Laboratory
The NTID Center on Access Technology Innovation
Laboratory ("CAT Lab"), a first-of-its-kind
initiative, provides a place for students to get
involved in the innovation process.
The CAT Lab is a place where faculty and associate
degree-level deaf and hard-of-hearing students
can collaborate on multidisciplinary projects
related to developing and adapting access and
instructional technologies, health care
technologies and services, and more.
41
Personnel
James J. DeCaro – Director of CAT
E. William Clymer – Associate Director of CAT
Peter A. Lalley – Researcher / Investigator
Gary W. Behm – Director of CAT Innovation Lab
Gail Hyde – Project Manager
42
Types of Projects
• Funded
– Internally (RIT or NTID)
– Externally
• Proposed to funding agencies
• Concept
43
Funded – Internally (NTID)
PDA / Cell Phone Notification System
Names
Roles
Dr. Jim DeCaro
Project Sponsor / Owner
Bill Clymer
Supervisor
Gary Behm
Engineering Project Manager
Wendy Dannels
Engineering Subject Matter Expert
Joe Stanislow
Engineering Subject Matter Expert
Cory Behm
Engineering Design Assistant Lead
Matthew Poe
Engineering Design Assistant
44
Funded – National Science Foundation
Testing the Concept of a Virtual Alliance for
Postsecondary Level STEM Students who are
Deaf and Hard-of-Hearing




NSF Grant Partners
Roles
Names
NTID Center on Access Technology
PEPNet
Lab for Social Computing at RIT
Other NSF-supported regional alliances
PI
Bill Clymer
Co-PI
Gary Behm
Co-PI
Peter Lalley
Project Manager
Gail Hyde
45
Proposed – National Science Foundation
Improving Distance Learning for Deaf Students by use of a
3D Signing Avatar
Whiteboard
Sign
Language
Interpreter
Hearing Student
Instructor
Deaf Student
3D Avatar
Conferencing
Service
Whiteboard
Undivided
Attention
3D Signing Avatar
NSF grant partners




NTID Center on Access Technology
RIT Computer Science Department
NTID Department of Engineering Studies
Other NSF-supported alliances
Roles
Names
PI
Joe Geigel
Co-PI
Gary Behm
Project Manager
Gail Hyde
46
Proposed – National Science Foundation
Developing an Innovative See-Through Face Mask
for More Effective Communication
Roles
Names
PI
Jim DeCaro
Co-PI
Gary Behm
Co-PI
Changfeng Ge
Co-PI
Massoud Miri
Project Manager
Gail Hyde
NSF grant partners







NTID Center on Access Technology
NTID Department of Engineering Studies
Department of Chemistry in the College of Science
Packaging Science in College of Applied Science and Technology
Rochester General Health System (RGHS)
RIT IP Management
IBM
IBM has agreed to assign
this patent to RIT
47
Proposed – NTID, RIT/RGHS Alliance, NSF
Digital Image Relocation Device to Assist Individuals
with Low Vision
NSF Grant Partners
Co-PI - Gary Behm
Project Lead / Developer
NTID Center on Access Technology
Project Mgr. - Gail Hyde
Project Management NTID Center on Access Technology
- Bill Travis
Consultant
End User
PI
- Dr. Andreas Savakis Developer
RIT Department of Computer Engineering
- Dr. Andreas Savakis’ Team
Developer
RIT Department of Computer Engineering
Co-PI - Dr. Gwen Sterns
Ophthalmologist
RGH Ophthalmology’s Laboratory
- Dr. Gwen Sterns’ Team
Ophthalmologists
RGH Ophthalmology’s Laboratory
- Deaf Co-op Student Software Engineering
NTID Department of Engineering Studies
- Deaf Co-op Student Design Assistant
NTID Department of Engineering Studies
48
Funded – Cisco Foundation
Preliminary Investigation of Cisco Technologies and Access
Solutions for Deaf and Hard-of-Hearing Individuals
• State of the Art and Recommendations Related to 911-411-211 Telephone Response Systems
•Evaluation of Possible Use of Avatars to Enhance Direct Communication Support for Deaf and Hardof-Hearing Users
•Evaluation of TelePresence Technologies for Face-to-Face and Remote Communication for Deaf and
Hard-of-Hearing Users
This project is the start of a multi-year effort.
Partners
 NTID Center on Access Technology
 Cisco Foundation
 NTID Department of Engineering Studies
 RIT Golisano College of Computing &
Information Studies
49
A Investigation of Cisco
Technologies & Access Solutions
50
Cisco and RIT/NTID
• Cisco has supported RIT Information Science
and computer programs
• Jim Ebenhoch and NTID past president
Hurwitz visited Cisco, with introductions to
Cisco’s Accessibility, Compliance and
Certification office by Len Mudrock ’84 who is
a senior software engineer at Cisco
51
Proposal
• Three strands
• One year effort
• Establish our understanding and areas where we
can contribute to Cisco product development
• Development of position papers and a call for
further research
• Deaf perspective
52
Strands of R&D
• Provide Cisco with a deaf perspective on their
products and the services they support
• Three strands
– 911-411-211 Communication
– Avatars
– TelePresence
53
Award
• From the Silicon Valley Community
Foundation, following the recommendation of
the Cisco Accessibility Team
• To the NTID Center on Access first year of
what is anticipated to be a multi-year effort
• Approximately $65,000 cash, and $35,000
equipment
54
But…
• TelePresence equipment
donation was upped to the
equivalent of $300,000!
– One 65” HD screen
– Three 65” HD screens
– System installation and
furniture
55
General Structure of Work
• Form teams of RIT/NTID, local and national
experts
• Conduct a literature review to understand issues
• Produce “White Paper” and disseminate for
comment
• Convene meeting(s) and focus groups
• Revise “White Paper”
• Produce final recommendations
56
Web-based Resources
• http://www.rit.edu/ntid/cat/cisco
57
911-411-211 Communication
• Project Team
– Bill Clymer, Project Leader
– Gail Hyde, Project Coordinator
– Kelly Masters, Focus Group and Evaluation
• RIT/NTID experts
• External experts
• Cisco experts
58
Issues
• Access to Emergency Information and Services
through 9-1-1 emergency public safety answering
points (PSAP’s) through receipt of text and video
• Identify barriers and offer solutions
• Next generation 9-1-1 communication
• Recognize the “community” of interested persons
• Fail safe technology that emergency responders,
telecommunication companies, equipment
companies and users can accept
59
Major Tasks
• Develop set of issues from literature
• Convene focus groups at RIT
– Policy
– Technology
– Users
•
•
•
•
Literature review & “White paper”
Post and collect comments
Review and comment by experts
Publish final report
60
Signing Avatar
• Project Team
–
–
–
–
Joe Geigel, Project Co-‐Leader
Gary Behm, Project Co-‐Leader
Gail Hyde/Bill Clymer, Project Coordination
Kelly Masters, Focus Group Expert and Writer
• RIT/NTID experts
• External experts
• Cisco experts
61
Signing Avatar
• Goal is to explore:
– Possibility of signing avatars, guided by voice-to-text
or other input processes, to be used effectively for
signing communications support (over videophone,
TelePresence or conferencing systems)
– Similar to what is provided by live, professional sign
language interpreters.
62
Signing Avatar
• An avatar is a computer
representation of a person
in the form of a threedimensional model used in
virtual space.
• Signing avatar – use of a 3D
avatar for displaying signed
messages.
http://www.vcom3d.com/vault_files/making_forest_asl/
63
Applications
• Automated sign language translation in public
spaces
• Education
• Remote sign language
support
– Virtual spaces
– Teleconferencing
64
http://www.visicast.co.uk/demo/tessa/tessa_demo.htm
Idealized Signing Avatar System
AVATAR
audio
Speech to
text
Text to SL
text
SL to
gestures
Sign
language
Motion
capture
gestures
gestures
Live interpreter
65
Expertise Required
•
•
•
•
Sign Language / Linguistics
Current signing Avatar systems
Speech to text
3D Graphics / Animation
– Motion capture (body and hand)
– Facial analysis, modeling, and animation
• Deaf communities
– Cyber/virtual
– Educational
66
Deliverables
• “White Paper”
–
–
–
–
State of the art
Future possibilities
Identify key questions and challenges
Recommendations
• Focus Group Meeting
– Discussion among experts in areas previously listed
– Refine “White Paper” based on discussions
67
TelePresence Evaluation
• Goal is to explore:
– Possibilities and applications of TelePresence to
support communication:
• in instructional, laboratory and social environments for
students who are deaf or hard-of-hearing.
• of closed caption, presentation, signing avatars, social
networking, and others.
68
TelePresence
http://matthewwall.typepad.com/.a/6a00d8341c68d853ef0111
68a477f0970c-800wi
• Telepresence refers to a set
of technologies which
allows a person to feel as if
they were present, to give
the appearance that they
were present, or to have an
effect, at a location other
than their true location.
• TelePresence makes remote
or distant people appear or
feel present and part of the
local activity by using
telematics technology. 69
• Education
Applications
– Classroom and Laboratory
• Direct
• Indirect
– Tutoring / Mentoring
• Remotely (different colleges)
– Job Interview
• Remotely
• Social
– Networking
• Communication Support
– Remote Sign language
• virtual spaces
– Teleconferencing
– Closed Captions
– Others?
70
TelePresence Network System
71
Expertise Required
•
•
•
•
Instructors for Deaf / Hard-of-Hearing people
Deaf / Hard-of-Hearing Students (direct and indirect)
Sign Language / Linguistics
3D Graphics / Animation / Avatar
– Motion capture
– Facial analysis, modeling, and animation
• Communication Support
– Closed captions / presentation
– Other
• Deaf Communities
– Cyber / virtual
– Educational
72
Deliverables
• White Paper
–
–
–
–
State of the art
Future possibilities
Key questions and challenges identified
Recommendations
• Focus Group Meeting
– Discussion among experts
– White paper refined, based on discussions
73
Team
• Gary Behm, NTID
• Wendy Dannels, NTID
• Kelly Masters, outside consultant
• Gail Hyde / Bill Clymer, NTID
74
Instructional Technology
Symposium
http://www.rit.edu/ntid/vp/techsym/
• June 21-23, 2010 in
Rochester
75