Dynamic Interfaces for Digital
Libraries: The Open Video
Project
Gary Marchionini
University of North Carolina at Chapel Hill
march@ils.unc.edu
NJ/ASIST Distinguished Lecture
April 4, 2002
Outline
•
•
•
•
•
•
•
User Interfaces as crucial elements of DLs
Overview of Open Video Project
Agile Views interface framework
Open Video examples of AVs
User study example and near term plans
Long term implications
Summary
Observations
• Physical libraries architect space to aid
information seeking
• Librarians interact with patrons in many ways:
– Show a welcoming face
– Clarify needs and queries
– Assist with retrieval and use
• Digital libraries depend on the user interface to
serve these purposes, thus user interfaces are
crucial to success of high-volume digital
libraries
Interface Principles for DLs
• Consider physical and conceptual interface issues.
• Consider representations and user control
mechanisms.
• Provide a multiplicity of indexes (representations):
Help people help themselves.
• Minimize user effort: A click is a radical act.
• Use a variety of iterative user studies to
understand user needs, common tasks, and
interface effects.
Open Video Project
• Goals
– Create an open source DL for use by researchers, students,
and the public.
– A testbed for interactive interfaces
– An environment for building a theory of human
information interaction
• Ongoing work: begun 1995 with colleagues at UMD
• Current funding: NSF# IIS-0099538, NCNI
• Collaborators: I2-DSI, ibiblio, CMU, UMD, NIST,
Internet Archive
• www.open-video.org
Current Status
•
•
•
•
•
•
•
~ 0.5 TB of content
~1600 video segments
~1100 different titles
~2000 unique visitors per month
I2-DSI video channel
OAI provider
Ongoing user studies
Distributed
Files
Open Video
Server
Search
MPEG etc
Database
AVI
(MySQL)
.
Browse
Client
(Browser)
Contribute
MPEG etc
.
MPEG etc
.
Digitization
Segmentation
Keyframe Extraction
Keyword (text)
Keyword (audio)
Surrogates
Metadata
Production System
Agile Views Interface
• Provide a variety of access representations
(e.g., indexes) and control mechanisms
• Usual search and browse capabilities
• Leverage both visual and linguistic cues
• Create and test surrogates for overview and
preview
Agile Views Framework
Evolution of Agile View Design
Techniques
• Various dynamic query interfaces (HCIL UMD)
• Relation Browser (BLS, SILS seminars)
– Federal statistics, overviews of relationships (several different
partitions). Useful for small number of attribute sets, each with
small number of attribute values. Backend database of metadata,
Java applet interface
• Enriched Links (Sony Labs)
– Complex web sites, previews, overviews, and reviews of pages.
Backend computation and Javascript interface
• Integrated overviews and previews (BLC UMD)
– Multimedia digital library, backend computation, Java applet
interface
Relation Browser
Enriched Links:Preview
Enriched Links: Overview
Enriched Links: Shared View
Overviews and Previews: One
Screen
The Open Video Project Case
Browse: by Categories & Attributes
Note
Results
Data
Search: by Category & Attribute
Search: by Free Text & Keyword
Search Results
Note
Mouseover
Popup for
Details;
Click yields
Next segment
Bib record
Segment Details
Video Transcript Text
Video Segment Preview
Research Agenda 2001-04
• What kinds of surrogates to provide for overviews and
previews?
• Currently, we are designing and testing cost-benefit
tradeoffs for:
– Storyboards
• text keywords
• audio keywords
– Slideshows
• text keywords
• audio keywords
– Fast-forwards
• Integration of many specific good ideas lead to emergence
or chaos?
AgileViews Overview – Genre: Documentary
Note
One keyframe
Per segment
Shown on
Mouseover;
Direct select
supported
AgileViews Overview – Genre: Education
AgileViews Overview – Color/B&W
Previews
Agile Views Preview – Faces
Agile Views Preview – Faces
Agile Views Preview – Superimposition
Agile Views Preview – Brightness
R&D Scheme
TASK
CHARACTERISTICS
Dependent Variables:
VIDEO
CHARACTERISTICS
PERFORMANCE
PREFERENCE
SURROGATE
CHARACTERISTICS
INDIVIDUAL
CHARACTERISTICS
User Study 1: Explore
Surrogate Space (Fall 01)
• five video surrogates compared
– Storyboard (6x6 grid of keyframes)
with text keywords
– Storyboard with audio keywords
– Slide show (keyframes displayed @
250ms per frame) with text keywords
– Slide show with audio keywords
– Fast forward (at 4 times original
speed)
• 10 subjects with video experience (about 2 hours
each)
• Two phases
– Three surrogates for each of four videos
• Preferences changed over time, support for ff development
• Gist meant: topicality, narrative structure, and visual style
– One surrogate (free choice) for each of 3 videos
•
•
•
•
•
•
write statement of gist
Select statements of gist
Object recognition (textual)
Object recognition (visual)
Action recognition
Visual gist
Best
Worst
Gist determination,
free text
Slide show w/
audio
keywords
Storyboard
w/ text
keywords
Storyboard
w/ audio
keywords
Fast forward
Gist determination,
multiple-choice
Fast forward
Slide show w/
audio
keywords
Storyboard
w/ text
keywords
Storyboard
w/ audio
keywords
Object recognition,
textual
Storyboard
w/ text
keywords
Fast forward
Slide show w/
audio
keywords
Storyboard
w/ audio
keywords
Object recognition,
graphical
Storyboard
w/ text
keywords
Slide show w/
audio
keywords
Fast forward
Storyboard
w/ audio
keywords
Action recognition
Fast forward
Storyboard
w/ text
keywords
Storyboard
w/ audio
keywords
Slide show w/
audio
keywords
Visual gist
Storyboard
w/ audio
keywords
Storyboard
w/ text
keywords
Slide show w/
audio
keywords
Fast forward
Next Steps (short term)
• Study 2 (spring 02)
– Compare different fast forward rates
– 30 subjects
• Interactive Shared Educational Environment
(ISEE) for video (remote study)
• CHI 02
– The commons with demos and study of narrativity
• Summer study (02)
– Eye-tracking study of effects of textual and audio cues
Next Steps (long term) From
DLs to the Sharium
The Sharium Work Space
M e s s a g i n g
Search/
Discovery
Search/
Discovery
Problem Solving/
Construction
Problem Solving/
Construction
Digital
Library
Contribution
Presentation
Contribution
Channels
Files
Tools
Presentation
Integration Hypothesis
• As information resources and technologies
are integrated as digital libraries (sharia or
collaboratories), institutional boundaries
will blur. Examples:
– Types of learning (formal, informal,
professional)
– Types of libraries
– Levels of government (local, state, federal)
Long-Term Implications for
Social Sciences Research
• What does it mean when you can have everything
you can possibly access anywhere, available
everywhere? Removing the bounds of access
implies ubiquity and augmented memory. What
does it then mean to be informed? Intelligent?
Consider a cascading set of issues such as trust,
ownership/IP, communication, human
relationships, and socio-technical symbiosis.
Summary: Open Video as Testbed
• Give people many ‘views’ to look ahead
• Make these views easy to manipulate (agile)
• Challenges
– Mapping video characteristics to surrogates (e.g.,
keyframes, keywords), mapping surrogates to control
mechanisms (e.g., mouse actions)
– Automating production processes
• Use the repository, contribute to the repository!
We would especially like to include cultural video
in the collection.
Pointers and Thanks
• www.open-video.org
• www.ils.unc.edu/idl
• Thanks to:
–
–
–
–
National Science Foundation
North Carolina Networking Initiative
Contributors
NJ ASIST