The Entertainment Technology Center Presents:
An Extensible Platform for Interactive,
Entertaining Social Experiences with an
Animatronic Character
Will Bosley, Dave Culyba, Sabrina Haskell, Andrew
Hosmer, TJ Jackson, Shane Liesegang, Seema
Patel, Christine Skarulis, Peter Stepniewicz, Jim
Valenti, Salim Zayat, and Brenda Harger
Carnegie Mellon University
August 1st, 2005
OUTLINE
•
•
•
•
Motivation
Goals
Related Work
Hardware
 Robot
 Kiosk
• Software
 Control Software
 Content Authoring Tools
• Rapidly Developed Content
• Future Work
MOTIVATION
• What is the Interbots Initiative?
• Recent Explosion in Consumer Robotics
Industry
• Popularity of AIBO and Robosapien
demonstrate that Entertainment Robotics is a
viable Industry
 Unlike traditional theme park animatronics, novelty
is in interactivity and unique experience
GOALS
1.
Leverage the success of household
entertainment robotics to introduce
interactivity and uniqueness to higher end
custom animatronics

Develop a fully interactive, autonomous
animatronic character that is both believable and
entertaining
2.
Give non-technologists the ability to author
content for interactive animatronic
experiences

Develop an extensible platform that allows artists
to rapidly design and create complete,
entertaining social experiences with
animatronic characters
RELATED WORK
• Within academia, most similar to that of Breazeal
et al
 Animatronic robot Leonardo centers around humanrobot collaboration and learning
• Our robot, Quasi, is influenced by behavioral, emotional,
and environmental motivators, but does not replicate
artificial intelligence
 Interactive Robot Theatre featuring an anemone-like
creature
• Work differs in depth of interactivity
• Within industry, most similar to Disney
Animatronics
 Lucky the dinosaur
• In many ways, we attempt to bridge academia
and industry
THE IBI PLATFORM
• In 14 weeks a team of 6 interdisciplinary
graduate students created:
 A believable animatronic character named Quasi
 A suite of software tools that allow for the
development for interactive social experiences with
Quasi
• The platform allows for experiences to be
developed quickly and easily. No
programming or technical knowledge is
required.
HARDWARE
Quasi the Robot
Quasi’s Kiosk
FABRICATION
QUASI’S HARDWARE
KIOSK HARDWARE
THE SOFTWARE
• Control Software
 Character State Control
System (CSCS)
 Real-time Show Control
System (RSCS)
 cVision, Inter-Process
Communication (IPC),
iButton Reader, MIDI
Reader, Babble
• Content Authoring Tools
 Behavior Authoring Tool
(BAT)
 Maya Plug-In
 Macromedia Director &
Flash
CONTROL SOFTWARE
• Character State Control System (CSCS)
 Backend run-time decision mechanism for a
character
 Depends on set of system inputs and network of
related states and transition rules (essentially a FSA)
 This behavioral model is stored in a database format
 Inputs are split into internal “factors” and external
“facts”
 Facts represent knowledge acquired from the character’s
environment
 Direct (e.g. distance reading from IR sensor)
 Indirect (interpreted, e.g. “someone’s standing in front of me”)
 Factors represent character’s internal environment
 Emotional State and Motivations (happiness, aggression,
loneliness, cleanliness, attention)
CONTROL SOFTWARE
 A character’s behavioral database is (at the high
level) a system of “superstates”
 Original Quasi personality included Idle, Sleep,
Greeting, Farewell, and GiveCandy superstates
 Each superstate (except the default Idles) has a set
of entry requirements
 Superstate is entered when facts and factors meet
entry requirements
 Superstates also have “stay requirements”
 Superstates are not connected, when stay
requirements aren’t met CSCS finds all superstates
that match entry requirements and randomly picks
one, weighted by priority
CONTROL SOFTWARE
 Within each superstate is a
network of substates
 Traditional FSA, substates
are connected by transitons
 Upon entry to a substate:
 Adjustments are made to
internal factors
 Actions take place (e.g.
animations, lighting changes,
sound, candy dispensing,
video display, etc.)
 After adjustments, all exit
transitions considered. If
none, CSCS exits current
superstate.
CONTROL SOFTWARE
• Real-Time Show Control System (RSCS)
 Actual software interface to hardware
 Receives message from CSCS specifying which
animation file to load, which DMX channels to play
them on, and any sound files that need to be
broadcast
 Sends DMX data to servos, and any other DMX
controlled hardware
 Has ability to linearly blend animations together,
and seamlessly transition from one animation to
another
CONTROL SOFTWARE
• cVision
 Face tracking tool; implementation of Intel’s OpenCV face
tracking library
 Inter-Process Communication (IPC)
 Message passing protocol that utilizes Inter-process
Communication Library developed in 1994 at CMU by Reid
Simmons
 MIDI Reader
 Listens to input signals from sensors, converts them to
IPC messages and transmits them to CSCS
 iButton Reader
 Allows Quasi to uniquely identify people through log-in
 Babble
 Implementation of open source Sphinx voice
recognition package developed at Carnegie Mellon.
Enchanced to pass messages over IPC.
CONTENT AUTHORING TOOLS
• Goal:
 Create content authoring tools that are so intuitive
and easy to use that no prior programming or
technical skills are required
 Accomplished By:
 Leveraging power of popular mass-market software
that artists are already familiar with
 Creating custom, graphical authoring tools
CONTENT AUTHORING TOOLS
• Behavior Authoring Tool
 One of the most powerful components of IBI Platform
 Used prior to run-time
 Graphical application for composing networks of
substates and superstates
 Written in Macromedia Flash, interfaces to
database through SQL commands
 Provides visualization of each superstate, making it
easy to understand complete picture of a
character’s personality
CONTENT AUTHORING TOOLS
Superstate Panel
CONTENT AUTHORING TOOLS
• Maya Plug-In
 Industry standard application for 3D modeling and
animation
 Natural choice for opening up animation of animatronic
characters to 3D animatiors
 Plug-in is a MEL script that converts keyframes to DMX
values for appropriate DMX channels; values are stored in
a file that is read in by RSCS and sent to servos
 Not limited to joint rotations– can control anything in
robot’s environment that can talk to DMX and is keyable
to some attribute in Maya: lights, colors, etc.
 Also created Animation Editor Tool (AET)
 For those unfamiliar with Maya
 Consists of a series of sliders for controlling joints
and lights
CONTENT AUTHORING TOOLS
Animation Editor Tool
RAPIDLY DEVELOPED CONTENT
• Usasbility of Content Authoring Tools put to
the test in Fall 2004 Building Virtual Worlds
course at Carnegie Mellon
 Two week project cycles
 Interdisciplinary teams of 4-5 students (1
programmer, 1 texture painter, 1 3D modeler, 1
designer, and possibly 1 audio artist)
 None of the students had prior experience with IBI
Platform
 Tools met expectations admirably– all teams
were able to rapidly develop complete,
entertaining experiences
RAPIDLY DEVELOPED CONTENT
 Quasi’s Interactive Dreams
 Addressed problem of how to
keep Quasi engaging while
allowing his motors to “rest”
 Quasi’s Photo Kiosk
 Allows guests to have their
picture taken with Quasi
 E-mails picture to guest
 Adds picture to album
 Displays Quasi’s life story
RAPIDLY DEVELOPED CONTENT
• Quasi-oke
 Finale of Fall 2004 BVW show, a spot typically reserved for the best
piece of the semester
FUTURE WORK
New portable cable driven robot
FUTURE WORK
Guided Performance Interface
FUTURE WORK
• Virtual Show Control System
 Sends DMX Signals to 3D model in Panda3D instead
of RSCS
• Character Authoring Tool
 Expands BAT to allow for plug and play hardware
ACKNOWLEDGEMENTS
• Brenda Harger
• Todd Bowers
• Zac Pavlov
• Dan Kanitz
• The ETC
• Art Institute of Pittsburgh
QUESTIONS?
• For more information please visit:
 www.interbots.org
 www.etc.cmu.edu