Project Goals: To reduce substantially the
human input necessary for creating and
accessing large collections of multimedia,
particularly multimedia created by capturing
what is happening in an environment.
Outcomes: (provide each where applicable)
Ideas
Captured Experience
During a Lecture,
displayed as a WWW
page
EIA9806822: “Automated Understanding of Captured Experience”
Georgia Institute of Technology, College of Computing
Investigators: Irfan Essa, G. Abowd, C. Atkeson, & K. Ramachandran
Extended use of automated capture in everyday environments (classroom/office/home) over
longer periods of time and the investigation of providing automated speech and visual transcription,
with accompanying search, retrieval and visualization techniques.
Tools
Tools that support the inclusion of automatically sensed context as meta-data to tag
captured information. This architectural approach has been instantiated as a toolkit (the Context
Toolkit) and used to produce prototype capture applications. Other TOOLs include INfrastructure for
Capture and Acess (INCA) and Stampede.
People
The Classroom 2000/eClass system has been presented on a variety of local, national and
international science and education programs as a demonstration of how ubiquitous computing
technology can impact the future of education. Many Post-Docs (2), PhD & MS Students (>10), and
Undergraduates (>20) have helped develop and have used the technology developed in part by this
funding are now working in academic, research, and industrial positions.
EIA9806822: “Automated Understanding of Captured Experience”
Georgia Institute of Technology, College of Computing
Investigators: Irfan Essa, G. Abowd, C. Atkeson, & K. Ramachandran
Vision-based Tracking of Activities
Recognition of
Flipping Pages
using HMMs
Interactive Web pages generated
by the Classroom 2000 / Eclass
System after a Lecture.
Annotation of
Objects
Interacted with
EIA9806822: “Automated Understanding of Captured Experience”
Georgia Institute of Technology, College of Computing
Investigators: Irfan Essa, G. Abowd, C. Atkeson, & K. Ramachandran
Impact:
We developed methodologies that supported extended use of captured experiences classrooms
(primarily, with a few experiments in the home and office scenarios) over longer periods of
time. To support this work, we researched and made significant contributions by
1.
2.
3.
4.
5.
Taking the existing Classroom 2000 software systems and extended it to use automatic speech
transcription, with accompanying search, retrieval and visualization techniques.
Software engineering principle were extensively used to provide infrastructure to support such large scale
capture and access scenario and allow for easy interface between capture, sensing, and usabilty
requirements (with the goal of reducing the load on the user). Resulted in Context Toolkit and the
INfrastructure for Capture and Access (INCA) Toolkit
Developed a distributed Programming Structure (Stampede) to support such high-end multimedia
processing.
Developed computer vision-based methods to track people and recognize their activities within a specific
context.
System in use for educational transcription. Used by many students and educators.
Barriers & Opportunities:
Developed systems and infrastructures allow for extensions to many domains.
Aware environments will serve towards providing us with ability to sort through and
find relevant information as we go about our daily activities. Complex environments
like homes and offices provide very interesting opportunities. Further opportunities
lie in the support of challenges associated with cognitive and/or age-related
impairments.
Website: http://www.cc.gatech.edu/cpl/projects/AUCE