New Media Research
Profs. Ken Goldberg and Greg Niemeyer
CITRIS at UC Berkeley
Prof. Marilyn Walker
CITRIS at UCSC
Maurizio Forte
CITRIS at UC Merced
1
CITRIS 2010
California is the Mecca of new media
“The world spends over 110 billion minutes per month on
social networks and blog sites.” - NielsenWire, June 2010
Outline
Overview of CITRIS New Media Research
Case Studies
Game-Based Learning for Health Applications
Tele-Immersion and Archaeology
Crowdsourcing Insights and Innovation
Research Partners
Next Ten Years
3
CITRIS 2010
What is a medium?
4
CITRIS 2010
5
CITRIS 2010
6
CITRIS 2010
7
CITRIS 2010
8
CITRIS 2010
9
CITRIS 2010
10
CITRIS 2010
11
CITRIS 2010
12
CITRIS 2010
14
CITRIS 2010
15
CITRIS 2010
16
CITRIS 2010
Mission: To critically analyze and shape
developments in new media from crossdisciplinary and global perspectives that
emphasize humanities and the public
interest.
17
CITRIS 2010
machine learning and social interaction (ryokai)
CITRIS 2010
18
phenomenology and second life (dreyfus)
CITRIS 2010
19
trust and video conferencing (canny)
CITRIS 2010
20
donation dashboard (goldberg)
21
CITRIS 2010
22
CITRIS 2010
The New Media
Research Roundtable
A bi-weekly series for faculty and graduate
students to discuss current research in new
media with emphasis on identifying new
collaborative research opportunities among
participants and presenters.
23
CITRIS 2010
Public Events:
•ATC Lecture Series
•Design Futures Lecture Series
•Continuous Bodies Symposium
•Rip.Mix.Burn. Art Exhibit
•ParaSite Symposium
•Out of Time-Space
•Embodiment and New Media
•Conversation on Digital Film
•Artist Appropriation Rights
•Attention Literacy
CITRIS 2010
24
BCNM Research Lab and Reading Room
4th Floor
CITRIS 2010
25
Game-Based Learning for
Health Applications
Prof. Marilyn Walker
CITRIS at UCSC
Prof. Greg Niemeyer
CITRIS at UC Berkeley
26
CITRIS 2010
Spy Feet: Using mobile gaming to promote
physical activity in girls
 Research on preventive health has shown
that despite various interventions, physical
activity declines precipitously in
adolescents, especially in girls – leading to
obesity, diabetes and cardiovascular
problems.
 Novel interventions for motivating
teenagers to exercise would thus help
address a national health problem.
 The interest of youth in computer games
and in smart phone applications suggests
that mobile computer games aimed at
increasing physical activity could providing
compelling contexts for transforming healthrelated behaviors in young people
CITRIS 2010
27
UCSC SpyFeet CITRIS Team
 Marilyn Walker
 Research: Adaptive Algorithms,
Dynamic Dialogue Management,
Expressive Generation
 Noah Wardrip-Fruin
 Research: Playable Media, Game
Design
 Sri Kurniawan
 Research: Assistive Technologies
and Human Computer Interaction
CITRIS 2010
28
32
New Media:
RPG’s + Dynamic Dialogue Generation
When a scheming mad scientist starts
taking over drivers in the player's
hometown, it's up to them to solve an
ever-deepening mystery. As a budding
Nature Warden, players learn to use their
secret abilities to speak with animal spirits,
uncovering a previously invisible world
where ants are foot soldiers and beetles
sing opera. Players will go on a series of
journeys through familiar streets now alive
with animals to befriend, missions to
accomplish, and mysteries to unlock. The
non-linear dynamically configured story
and dialogue generation gives players the
freedom to investigate only the characters
or story elements that interest them.
29
33
CITRIS 2010
Research Questions
 Technical:
 Hypotheses: Dynamic elements will increase
motivation to play, replayability, and
immersion
 SpyGen: A new generation engine to support
dynamic adaptive dialogue generation for
characters in role playing games
 Grail GM: a new role playing game manager
that supports dynamic reconfiguration of
quests to allow user choices to matter
 Societal:
 What types of motivational elements can
influence behavior change?
 What is the role of social interaction vs.
narrative world?
CITRIS 2010
30
SpyGen
CITRIS 2010
31
35
Grail GM + Informant Story Manager
CITRIS 2010
32
36
Summary
 Societal
 Will be ready to test with users in November
 Experiment with role of technical elements in motivating
behavior change
 Technical




Android Platform very re-usable
Dynamic, easily reconfigurable architecture
Spy Gen 1.0
Grail GM 2.0
CITRIS 2010
33
37
Track FX: A study about game-based
learning and therapeutic intervention
• Initial study with Typicals age 28 to 60 months at Child
Study Center
• Game played on Tablet PCs.
• Data collection on online database.
• Four types of players seen in data (cf. Bartle,
Richard): Learners, Novelty Seekers, Explorers,
Competitors
• MOT skills accelerate at 36 months (cf. Lantern Mind)
• TrackFX is outcome measure for Minocycline study
• Game Type: Intervention and Outcome Measure
37
CITRIS 2010
Learner
RuleMaker: Abstraction and Memorization
• Initial study with typicals age 8 to 11 years at MIND
institute (Rivera, Langer)
• Game played on web page via Flash
• Data collection to an online database
• Study: What regions of the brain store abstract
concepts, and what regions store explicit
memorization?
• Game type: research game providing researchers with
deep interaction data.
40
CITRIS 2010
41
CITRIS 2010
Neuropolis: A game design for
neuroscience awareness
• Intended audience is anyone who has a reason to find
out how the brain works.
• Client: MNI.
• Data collection on online database.
• No player data collected yet, but future versions will
collect deep data.
• Game type: Awareness game (The Brain: a playable
owners' manual)
42
CITRIS 2010
Pokerwalk: A health game promoting
urban awareness
• Intended audience is anyone who wants to understad
pedestrian circulation in a city
• Client: ArtPrize Grand Rapids.
• Data collection on online database.
• Collected data produces circulation maps
• Game type: awareness game (city, health)
43
CITRIS 2010
Playable Links:
• Web-based:
o Rulemaker: www.rulemaker.org
o TrackFX: www.trackfx.org
• Thanks to CITRIS for connecting support, space,
doctors and game designers.
• Thanks to Townsend Center for summer interns.
44
CITRIS 2010
The CITRIS Promise: Multi-Campus
Interdisciplinarity
• Presentation of "Balance Game" at UC Davis TeleImmersion conference
• Seed funding for Multi-Campus pilot project
• Close collaborations with Drs. Randi Hagerman,
Susan Rivera and Faraz Farzin now include Fragile X
game (TrackFX) and RuleMaker
• Collaboration led to international joint venture with
the Montreal Neuroscience Institute, Neuropolis.
45
CITRIS 2010
Cyber-Archaeology
Virtual Environments and New Media
Maurizio Forte
CITRIS at UC Merced
Ruzena Bajcsy
CITRIS at UC Berkeley
46
CITRIS 2010
Reconstructing the Material Past
 The reconstruction of the past, in
terms of cultural material, is one
of the biggest challenges for
contemporary societies.
 The link of archaeology and
digital technologies is
fundamental for revisiting,
interpreting and communicating
the past
 One of the bottlenecks in
archaeology is the difficulty to
contextualize and share data,
models, archives, metadata in a
collaborative way.
47
What kind of information can we transmit
to the future generations?
How do we preserve the knowledge of the
past?
How can we communicate this knowledge
in the Digital Era?
ARCHAEO-PEDIA 3D: A POSSIBLE FUTURE
COLLABORATIVE NETWORK ACROSS UC CAMPUSES
Outcomes and Perspectives
 The prototypal virtual collaborative work open very
challenging perspectives in other research and
educational areas in and out the UC system.
 Ideas > 3D learning, Virtual Classes, Museum Studies,
CRM, Visual Art, Image Processing, Environmental
Simulation, Environmental Monitoring, Virtual Labs, 3D
Modeling, 3D Publications, Teleimmersive Networks
 We are well positioned for the next ten years
 Migration and preservation of 3D digital archives and
datasets
 Simulation studies, Networking, Collaborative intercampus Scenarios, Intelligent Distribution of Digital
Resources, Advanced Cognitive Impacts
49
Results of the Prototype Platform (UC
Merced, UC Berkeley, UC Davis)
• User immersed in the virtual environment
• Two users in the shared virtual space (rendering with applied texture mapping)
Is the 3D
modeling
sufficient to
show and
explain the
tomb
iconographic
complexity?
TheVirtual
Cyber
Map
Reconstruction of a Chinese tomb at Xi’an
(Virtual Collaborative System)
The Powerwall at UC Merced
Collaborative VR. M.Kalmann, M.Forte
CITRIS 2010
52
UCM Students at the Powerwall
CITRIS 2010
53
Current and Future Partners
 NEH
 NSF
 Trimble Navigation
 Nextengine
 Erdas
 Avie Systems
 UNESCO
54
Social Media for
Crowdsourcing Innovation
and Insights
Profs. Ken Goldberg
CITRIS at UC Berkeley
55
CITRIS 2010
The challenge: too much of a good thing
20 sec. per comment X
35,387 comments
=
8 days
Goals
For Organizations
 Understand the diversity of their community
 Engage their community
 Solicit feedback and creative suggestions
 Rapidly identify patterns, insightful ideas
For Community Members
 Understand relationships with other
community members
 Engage with a diversity of viewpoints and
ideas
 Express ideas, and be heard
57
CITRIS 2010
hybrid vigor
Our Approach
1. Visualization
3. Wisdom of Crowds
2. Level Playing Field
4. Game Structure
Opinion Space
Step 1: Enter your opinions and response
Step 2: Visualize your position
Step 3: Read and rate others users
March-Oct 2010
4,963 Users
94 Countries
24,815 Opinions
63
CITRIS 2010
Interpreting Eigenvectors
Direction and magnitude of motion in PCA-space as slider value
changes for each individual proposition.
CITRIS 2010
64
Research Team:
David Wong:
Ephrat Bitton:
Siamak Faridani:
Elizabeth Goodman:
Tavi Nathanson:
Alex Sydell:
Sanjay Krishnan:
EECS MS Student
IEOR PhD Student
IEOR PhD Student
School of Information PhDStudent
EECS Graduate Student
EECS Undergraduate Student
EECS Undergraduate Student
Ken Goldberg:
Gail de Kosnik:
Kimiko Ryokai:
Meghan Laslocky:
Ari Wallach:
Steve Weber:
Peter Feaver:
IEOR, EECS, iSchool
Theater, Dance, Performance Studies
School of Information
Outside Consultant on Content
Outside Consultant on Content and Strategy
Outside Consultant on Content
Outside Consultant on Content
U.S. State Department:
Alec Ross:
Senior Advisor for Innovation
Katie Dowd:
New Media Director
65
CITRIS 2010
Partners thus far
"Opinion Space will harness the power of
connection technologies to provide a unique
forum for international dialogue. This is ... an
opportunity to extend our engagement beyond
the halls of government directly to the people of
the world."
- Secretary of State Hillary Clinton
New Media Research
Next 10 Years…
Profs. Ken Goldberg and Greg Niemeyer
CITRIS at UC Berkeley
Prof. Marilyn Walker
CITRIS at UCSC
Maurizio Forte
CITRIS at UC Merced
67
CITRIS 2010
Were the objectives and goals described in
the original CITRIS proposal met by our
research?
Absolutely! Our goals have been exceeded
but there is exciting work ahead for the next
10 years!:
68
CITRIS 2010
69
CITRIS 2010
70
CITRIS 2010
71
CITRIS 2010
72
CITRIS 2010
Thank you.