New Media Research
Profs. Ken Goldberg and Greg Niemeyer
CITRIS at UC Berkeley
Prof. Marilyn Walker
CITRIS at UCSC
Maurizio Forte
CITRIS at UC Merced
1
CITRIS 2010
California is the Mecca of new media
“The world spends over 110 billion minutes per month on
social networks and blog sites.” - NielsenWire, June 2010
Apple, Google, Facebook,
HP, Intel, Cisco, eBay,
Adobe, Agilent, Oracle,
Yahoo, Netflix, and EA.
Outline
Overview of CITRIS New Media Research
Case Studies
Game-Based Learning for Health Applications
Tele-Immersion and Archaeology
Crowdsourcing Insights and Innovation
Research Partners
Next Ten Years
3
CITRIS 2010
What is a medium?
4
CITRIS 2010
5
CITRIS 2010
6
CITRIS 2010
8
CITRIS 2010
9
CITRIS 2010
10
CITRIS 2010
Mission: To critically analyze and shape
developments in new media from crossdisciplinary and global perspectives that
emphasize humanities and the public
interest.
11
CITRIS 2010
machine learning and social interaction (ryokai)
CITRIS 2010
12
phenomenology and second life (dreyfus)
CITRIS 2010
13
trust and video conferencing (canny)
CITRIS 2010
14
donation dashboard (goldberg)
15
CITRIS 2010
16
CITRIS 2010
The New Media
Research Roundtable
A bi-weekly series for faculty and graduate
students to discuss current research in new
media with emphasis on identifying new
collaborative research opportunities among
participants and presenters.
17
CITRIS 2010
Public Events:
•ATC Lecture Series
•Design Futures Lecture Series
•Continuous Bodies Symposium
•Rip.Mix.Burn. Art Exhibit
•ParaSite Symposium
•Out of Time-Space
•Embodiment and New Media
•Conversation on Digital Film
•Artist Appropriation Rights
•Attention Literacy
CITRIS 2010
18
BCNM Research Lab and Reading Room
4th Floor
CITRIS 2010
19
Game-Based Learning for
Health Applications
Prof. Marilyn Walker
CITRIS at UCSC
Prof. Greg Niemeyer
CITRIS at UC
Randi Hagerman
CITRIS at UC Davis
20
CITRIS 2010
Spy Feet: Using mobile gaming to promote
physical activity in girls
 Research on preventive health has shown
that despite various interventions, physical
activity declines precipitously in
adolescents, especially in girls – leading to
obesity, diabetes and cardiovascular
problems.
 Novel interventions for motivating
teenagers to exercise would thus help
address a national health problem.
 The interest of youth in computer games
and in smart phone applications suggests
that mobile computer games aimed at
increasing physical activity could providing
compelling contexts for transforming healthrelated behaviors in young people
CITRIS 2010
21
New Media:
RPG’s + Dynamic Dialogue Generation
When a scheming mad scientist starts
taking over drivers in the player's
hometown, it's up to them to solve an
ever-deepening mystery. As a budding
Nature Warden, players learn to use their
secret abilities to speak with animal spirits,
uncovering a previously invisible world
where ants are foot soldiers and beetles
sing opera. Players will go on a series of
journeys through familiar streets now alive
with animals to befriend, missions to
accomplish, and mysteries to unlock. The
non-linear dynamically configured story
and dialogue generation gives players the
freedom to investigate only the characters
or story elements that interest them.
22
33
CITRIS 2010
Research Questions
 Technical:
 Hypotheses: Dynamic elements will increase
motivation to play, replayability, and
immersion
 SpyGen: A new generation engine to support
dynamic adaptive dialogue generation for
characters in role playing games
 Grail GM: a new role playing game manager
that supports dynamic reconfiguration of
quests to allow user choices to matter
 Societal:
 What types of motivational elements can
influence behavior change?
 What is the role of social interaction vs.
narrative world?
CITRIS 2010
23
Summary
 Societal
 Will be ready to test with users in November
 Experiment with role of technical elements in motivating
behavior change
 Technical




Android Platform very re-usable
Dynamic, easily reconfigurable architecture
Spy Gen 1.0
Grail GM 2.0
CITRIS 2010
24
37
From Anticipation to
Prediction: Games,
Data and Collective
Gain
Greg Niemeyer
CITRIS at UC Berkeley
Randi Hagerman
CITRIS at UC Davis
The CITRIS Promise: Multi-Campus
Interdisciplinarity
• Presentation of "Balance Game" at UC Davis TeleImmersion conference
• Seed funding for Multi-Campus pilot project
• Close collaborations with Drs. Randi Hagerman,
Susan Rivera and Faraz Farzin now include Fragile X
game (TrackFX) and RuleMaker.
• Collaboration led to international joint venture with
the Montreal Neuroscience Institute.
The CITRIS Promise: Service to Society
The CITRIS Promise: Service to Society
• Reduce age at
time of
intervention
• Distribute
application at
lowest possible
course
• Aggregate data
broadly
• Play anywhere
• Use existing
platforms and play
frameworks
• Android version
announced
Track FX: A study about game-based
learning and therapeutic intervention
• Game Type: Intervention and Outcome Measure
• Initial study with Typicals age 28 to 60 months at Child
Study Center
• Game played on Tablet PCs, Data collection on online
database.
• Four types of players seen in data (cf. Bartle,
Richard): Learners, Novelty Seekers, Explorers,
Competitors
• MOT skills accelerate at 36 months
• TrackFX is outcome measure for
Minocycline study
Playable Links:
• TrackFX: http://www.trackfx.org/motgame/motbugs/
• Thanks to CITRIS for connecting support, space,
doctors and game designers.
• Thanks to the Townsend Center for summer interns.
• Thanks to the Montreal Neuroscience Institute and UC
DAVIS for research support.
Cyber-Archaeology
Virtual Environments and New Media
Maurizio Forte
CITRIS at UC Merced
Ruzena Bajcsy
CITRIS at UC Berkeley
31
CITRIS 2010
Reconstructing the Material Past
 The reconstruction of the past, in
terms of cultural material, is one
of the biggest challenges for
contemporary societies.
 The link of archaeology and
digital technologies is
fundamental for revisiting,
interpreting and communicating
the past
 One of the bottlenecks in
archaeology is the difficulty to
contextualize and share data,
models, archives, metadata in a
collaborative way.
32
What kind of information can we transmit
to the future generations?
How do we preserve the knowledge of the
past?
How can we communicate this knowledge
in the Digital Era?
ARCHAEO-PEDIA 3D: A POSSIBLE FUTURE
COLLABORATIVE NETWORK ACROSS UC CAMPUSES
Outcomes and Perspectives
 The prototypal virtual collaborative work open very
challenging perspectives in other research and
educational areas in and out the UC system.
 Ideas > 3D learning, Virtual Classes, Museum Studies,
CRM, Visual Art, Image Processing, Environmental
Simulation, Environmental Monitoring, Virtual Labs, 3D
Modeling, 3D Publications, Teleimmersive Networks
 We are well positioned for the next ten years
 Migration and preservation of 3D digital archives and
datasets
 Simulation studies, Networking, Collaborative intercampus Scenarios, Intelligent Distribution of Digital
Resources, Advanced Cognitive Impacts
34
Results of the Prototype Platform (UC
Merced, UC Berkeley, UC Davis)
• User immersed in the virtual environment
• Two users in the shared virtual space (rendering with applied texture mapping)
Is the 3D
modeling
sufficient to
show and
explain the
tomb
iconographic
complexity?
TheVirtual
Cyber
Map
Reconstruction of a Chinese tomb at Xi’an
(Virtual Collaborative System)
The Powerwall at UC Merced
Collaborative VR. M.Kalmann, M.Forte
CITRIS 2010
37
UCM Students at the Powerwall
CITRIS 2010
38
Current and Future Partners
 NEH
 NSF
 Trimble Navigation
 Nextengine
 Erdas
 Avie Systems
 UNESCO
39
Social Media for
Crowdsourcing Innovation
and Insights
Profs. Ken Goldberg
CITRIS at UC Berkeley
40
CITRIS 2010
The challenge: too much of a good thing
20 sec. per comment X
35,387 comments
=
8 days
Goals
For Organizations
 Understand the diversity of their community
 Engage their community
 Solicit feedback and creative suggestions
 Rapidly identify patterns, insightful ideas
For Community Members
 Understand relationships with other
community members
 Engage with a diversity of viewpoints and
ideas
 Express ideas, and be heard
42
CITRIS 2010
hybrid vigor
Our Approach
1. Visualization
3. Wisdom of Crowds
2. Level Playing Field
4. Game Structure
Opinion Space
Step 1: Enter your opinions and response
Step 2: Visualize your position
Step 3: Read and rate others users
March-Oct 2010
4,963 Users
94 Countries
24,815 Opinions
48
CITRIS 2010
Interpreting Eigenvectors
Direction and magnitude of motion in PCA-space as slider value
changes for each individual proposition.
CITRIS 2010
49
Research Team:
David Wong:
Ephrat Bitton:
Siamak Faridani:
Elizabeth Goodman:
Tavi Nathanson:
Alex Sydell:
Sanjay Krishnan:
EECS MS Student
IEOR PhD Student
IEOR PhD Student
School of Information PhDStudent
EECS Graduate Student
EECS Undergraduate Student
EECS Undergraduate Student
Ken Goldberg:
Gail de Kosnik:
Kimiko Ryokai:
Meghan Laslocky:
Ari Wallach:
Steve Weber:
Peter Feaver:
IEOR, EECS, iSchool
Theater, Dance, Performance Studies
School of Information
Outside Consultant on Content
Outside Consultant on Content and Strategy
Outside Consultant on Content
Outside Consultant on Content
U.S. State Department:
Alec Ross:
Senior Advisor for Innovation
Katie Dowd:
New Media Director
50
CITRIS 2010
Partners thus far
"Opinion Space will harness the power of
connection technologies to provide a unique
forum for international dialogue. This is ... an
opportunity to extend our engagement beyond
the halls of government directly to the people of
the world."
- Secretary of State Hillary Clinton
New Media Research
Next 10 Years…
Profs. Ken Goldberg and Greg Niemeyer
CITRIS at UC Berkeley
Prof. Marilyn Walker
CITRIS at UCSC
Maurizio Forte
CITRIS at UC Merced
52
CITRIS 2010
Were the objectives and goals described in
the original CITRIS proposal met by our
research?
Absolutely! Our goals have been exceeded
but there is exciting work ahead for the next
10 years!:
53
CITRIS 2010
54
CITRIS 2010
55
CITRIS 2010
56
CITRIS 2010
57
CITRIS 2010
Thank you.