Tablet PC-Based Indexed Captioned Searchable Videos for STEM Coursework
Jaspal Subhlok (PI: University of Houston) Richard Alo (PI: UH, Downtown) David Coco (Lead: Texas School for The Deaf)
Background
Deployment
ICS Video Player
Index Panel
Lecia Barker (Lead Evaluator: UT Austin)
Tablet PCs allow free
mixing of prepared
viewgraphs and on-the-fly
annotations for teaching.
PC Video lectures are
widely used for STEM
education.
Keyword
Search Box
Integrated with VNET Course management system at University of Houston for extensive
deployment and usage. Current use by over 20 courses. Main usage centers:
• UH Biology, Computer Science, Geology, Mathematics, and Physics
• University of Houston Downtown Campus Computer Science
• Testing and feedback from the Texas School for the Deaf
Objectives
Development of “ICS
Videos” Framework for
quick access to video
content with:
Search Results
• Freely available to all interested educational institutions
Indexed video
segments with
keyword match
Evaluation
• Index Snapshots
• Keyword Search
• Text Captions
Profile of UH Students Surveyed:
• Very diverse
• Long commutes, avg 40 mins
• 60% work part time
• Only 66% native English speakers
• 90% used videos when available, yet almost
all believe it is important to go to class
Text Captions
Timeline with selectable Index
Indexing
 Summary: Videos are automatically divided
into logical segments, each represented by a
visual index snapshot. User can start video
playback from any index point.
 Automatic indexing: 1) Identify all “Transition
points” where the picture on the video changes
significantly and 2) Select “Index points” as a
subset of transition points that are roughly evenly
distributed.
 Status: Deployed since Summer 10.
Pairwise comparison of video frames to identify scene transitions
Contributors
Reasons For Using Videos
Search
Captioning
 Summary : User types a keyword; all video
segments that match the keyword are
presented.
 OCR: Off the shelf OCR tools to extract the
text from video frames. Enhanced with a suite
of preprocessing image transformations.
Accuracy significantly enhanced with image
transformations -- from ~91% to ~97%.
 Status: Evaluated Since Fall 10, Deployed
form Spring 11.
 Summary: Scrolling text for the
audio. Main purpose is to make
mainstream lectures available to deaf
students.
 Speech Recognition can be
employed, but exhibits high error
rates. Extensive instructor-specific
training needed for acceptability.
 Status: Captioning is supported by
the ICS video player but manual
transcription of audio text is currently
employed. Design influenced by the
Texas School for the Deaf. Wider
availability of captions from Spring 11.
Original Image
Inversion
Dilation
Edge Detection
Richard Alo, Andrea Arias, Salvador Baez-Franseschi, Lecia
Barker, Gautam Bhatt, David Coco, Zhigang Deng, Erin
Hodges, Olin Johnson, Pradeep Krishnan, Raja Kushalnagar,
Joanna Li, Yumei Liu, Shishir Shah, Jaspal Subhlok, Tayfun
Tuna, Varun Varghese, Anshul Verma, Chang Yun.
• Videos of Tablet PC based lectures proved to be a versatile academic resource beyond
Specific Evaluation Items
Mean
Video lectures help me to clarify material
that was not clear in class.
5.47
Video lectures are useful for reviewing.
5.64
Having access to video lectures for this
class is important to me.
5.60
The video lectures helped me to study
for quizzes or tests.
5.55
3.56/4
Scale: 1 = Disagree Strongly , 6 = Agree Strongly
Videos versus Text Book versus Notes
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Power of Indexing
Very important
Somewhat important
Slightly important
Not at all important
The textbook
and other
reading
sources.
Conclusions
Students Strongly Value Videos
How important was use of the video
lectures for this class for getting the
grade you hoped for?
Percent of Students
Evaluation of ICS Videos
for STEM Coursework with
extensive deployment and
assessment with student
surveys and instructor
interviews.
Main Video
Playback
The evaluation is based on student surveys from
Fall 2009, Spring 2009, and Spring 2010, totaling
1,839 usable surveys.
Notes you
take during
lecture and
reading.
Professor's The lecture
lecture notes.
videos.
Survey item: The Index made it easier to
find the part of a lecture I was looking for
expectations. Videos have special value for students striving for life balance with, e.g., jobs, long
commutes, or caregiving to children or the elderly.
• Key shortcoming of ordinary videos is the difficulty of locating content of interest. Indexing is
shown to help significantly in this. Users of Indexing perceived videos to have a greater value.
• We expect to establish that keyword search has impact similar to indexing. Captioning is being
deployed to make the videos accessible to deaf students.
What Students are Saying: Representative Comments
On Video Usage: “To review dense or quickly presented material that I could not take adequate notes on
in class. I listened to it on my ipod whenever I get stuck in traffic coming to or leaving campus’’
printed by
icsvideos.cs.uh.edu
NSF DUE-0817558 CCLI Phase-II
• Automatic methods were effective in driving indexing and keyword search. Speech recognition
for captioning is a challenge -- hybrid approaches found to be most practical.
On Indexing: “It enabled me to jump directly to my trouble spots” “I didn't have to wade through the rest
of lecture just to answer one question” “Well, sometimes I would have to pause the lecture to take care
of other responsibilities that I had to attend too, and when I was ready to come back to the lecture I'd pick
up exactly where I was at, it was great!”