Virtual Populace
Brain Springs: Fast Physics for
Large Crowds in WALL•E
Paul Kanyuk ■ Pixar Animation Studios
C
omputer animation is a developed, thriving
field that has benefitted greatly from the
techniques of traditional animation and its
insights into what elements of motion create appealing characters. Pixar Animation Studios introduced the adaptation of such elements as squash,
stretch, follow-through, and anticipation to the
field,1 which guide some of the most successful
computer animation to date. However, these elements are often neglected in crowd simulations,
where rendering challenges, group
behavior, and appearance variaA major challenge of making
tion are the dominant focus.
WALL•E was creating believable (For more information, see the
physics for human and robot
sidebar “Related Work in Crowd
crowds. To do this, Pixar
Simulation.”)
technical directors combined
We’ve developed a set of techa custom spring-physics
niques that help incorporate key
system in the Massive software elements of appealing character
animation into crowd simulaplatform with traditional
tions. In particular, we’ve devised
simulation methods. The
a procedural noise-based system
performance was fast enough
for creating “hover” motion, a
to scale for large crowds and
simple technique for anticipatmaintain interactivity for
ing turns, and most significant,
previews.
brain springs, an efficient physics
approximation for spring-based
motion. We’ve applied these techniques to various scenarios, ranging from thousands of flying
robots to crowds tumbling on a tilting spaceship
(Figure 1 shows one example). These scenarios
were varied enough to require extensive iteration
on large crowds, necessitating both efficiency and
interactivity.
Conventions
Before we explain our techniques, a frame of reference for translation and rotation is necessary.
From a crowd agent’s frame of reference, let the
24
July/August 2009
y-axis be up, the z-axis be forward, and the x-axis
be left and right. Rotation around the z-axis is
banking, around the x-axis is leaning, and around
the y-axis is turning.
Also, we created many of the following methods
using Massive, commercially available crowd software. (See the sidebar “Massive and WALL•E” for
implementation details.)
Physics Driving Motion
A defining aspect of crowd locomotion in WALL•E
is the characters’ ability to effortlessly float above
the ground. This applied to robots flying through
hallways as well as humans leisurely drifting on
levitating lounge chairs. We broke this look down
into three main elements: hover, anticipation, and
reaction.
Hover
This element is the naturalistic oscillation in translation and rotation that keeps a floating agent from
looking abnormally static (as if balanced between
slowly varying forces). (For an example of this element, see Video 1; all videos mentioned in this
article are at http://computingnow.computer.org/
wall-e.) We first attempted to create this effect by
using sine waves at different amplitudes and frequencies that vary over time, for each root degree
of freedom (x, y, and z translations and rotations).
Simply tuning the amplitudes and frequencies appeared to work at first, but the simulation often
degenerated after a few seconds and yielded “offkilter” motion (see Video 2).
To solve the problem, we first had our animators
keyframe a pleasing hover cycle, to serve as a reference and a model for a procedural solution. The
animators used sinusoidal curves for each degree
of freedom, too; however, this animated cycle’s
key feature was the phase relationship between
Published by the IEEE Computer Society
0272-1716/09/$25.00 © 2009 IEEE
the sine waves. One set of phases can create the
impression of a gallop, whereas another can create
a bounce, for example.
We adjusted our procedural model to maintain
the phase relationships that the animators assigned, and let only those parameters vary that
preserved relative phase (overall amplitude or
frequency, wave shaping, amplitude per degreeof-freedom, and so on). Varying these parameters
from agent to agent would cause the hovers to
remain distinct over time and impart a different
“affect.” (For example, a small, energetic cleaning robot would have a very-high-frequency hover,
whereas a large, lumbering cargobot would have
broader noise curves.) Parameterizing our noise in
this way let animators fine-tune our characters’
hover personalities (see Video 3).
Anticipation
This element is a flying agent’s tendency to bank
into a turn before reaching it, which is a natural
reaction for rapid cornering. Craig Reynolds described a similar effect in which the agent’s acceleration vector determines its banking magnitude
and direction.2 That technique, however, is reactive, in which the agent rotates only as the turn
is in progress.
Anticipation requires executing this banking motion before the actual turn, which gives the agent
a sense of volition and intent. We modeled this effect in our agent “brains” (See the “Massive and
WALL•E” sidebar) by looking ahead of each character’s path and measuring the difference in path orientation, which drove bank rotation (see Figure 2).
Owing to our agents’ extreme speeds and frequency of turns, this signal was very jagged. We
oversampled and filtered the signal as a postprocess, using a basic quaternion filter to achieve adequate smoothing. The result is a flying agent that
appears to “carve” into the turns, showing a sense
of awareness (see Video 4).
Reaction
This element is the natural motion resulting from
physical forces being applied to the character, as
opposed to the character’s intent at that time.
These effects can be well modeled using spring
dynamics. Rather than use a physical simulator,
however, we implemented this system within our
characters’ Massive “brains,” using fuzzy-logic
nodes (see “Implementing Brain Springs” in the
“Massive and WALL•E” sidebar). Our fake physics
was successful because it was stable, allowed for
interactive playback with large numbers of characters, and was easily parameterized to provide
controls based on the distance to the camera and
other metrics.
The brain spring, our system’s key unit, converts
an arbitrary input—typically a physical signal such
as acceleration or velocity—into an approximation
of damped simple harmonic motion that’s then
connected to an animation output. For example,
Figure 1.
Robot crowds
reacting and
hovering in the
movie WALL•E.
Throughout
much of the
film, crowds
of robots
and humans
on hover
chairs play
an important
role in the
environment
and action.
(© 2007 Disney/
Pixar. All rights
reserved.)
Look-ahead
Theta
Figure 2. The angle theta is the difference between the agent’s
orientation and intended path at the current location, and that of the
path a certain distance ahead. This angle drives banking before turning.
IEEE Computer Graphics and Applications
25
Virtual Populace
Related Work in Crowd Simulation
I
n feature films, animated and otherwise, crowd simulation is an area of ongoing research and development. The
predominant direction has been refinement of agent behavior and decision-making. Early research modeled flocking
behavior through simple local rules, which provided compelling navigation for groups of fish or birds.1 Later developments added knowledge and learning.2 Other research has
explored diverse additions to behavioral models considering factors such as sociology3 and continuum dynamics.4
However, behavior—navigation and motion choice—is only
one aspect of compelling crowd simulation.
Dynamics and the impression of real physics are key
in making animated characters believable but are often
overlooked for large crowds. This is especially important
for animated films, where exaggeration of dynamics is the
stylistic norm. Pixar Animation Studios describes how 2D
animation principles such as exaggeration and followthrough can enhance a 3D character’s believability.5 One
can model many of these nuances through the dynamics of a spring (damped simple harmonic motion), and
considerable research has focused on techniques for this
modeling. Physics simulation is common but often proves
too slow for real-time and crowd applications.
“Fake” physics techniques that avoid simulation yet
model this behavior are preferable for crowds. Two examples are a nondynamic technique for rope and spring
motion6 and “Wiggly Splines,” a novel spline interpolation
scheme that models spring oscillation.7 Another fake-physics
technique is NeuroAnimators, which use trained neural
networks to emulate physical systems.8 Many of these
techniques, however, are cumbersome for crowd simulation, NeuroAnimators owing to the neural-net training,
and others because of lack of speed or specificity issues.
One class of techniques that does work well for crowds
is motion signal processing, which uses filters to process
existing motion data to generate an effect.9 Such techniques are often used in computer puppetry to take live
performance input data and create more complex output.
Examples of this application include Protozoa’s Alive!
(described at www.awn.com/mag/issue5.03/5.03pages/
dannacherdot.php3) and Mike the Talking Head (http://
mambo.ucsc.edu/psl/mike.html).
One motion-signal-processing technique creates oscillatory motion using an infinite-impulse response filter
that can easily operate in real time on vast amounts of
motion data.10 Our brain-springs technique (see the main
article) furthers the state of the art in this area by modeling
damped simple harmonic motion in crowd agents through
a highly flexible motion-signal-processing method. With
brain springs, animators can create efficient, controllable,
and believable spring physics in crowd animation.
References
1. C.W. Reynolds, “Flocks, Herds and Schools: A Distributed
Behavioral Model,” Proc. Siggraph, ACM Press, 1987, pp.
25–34.
2. J. Funge, X. Tu, and D. Terzopoulos, “Cognitive Modeling:
Knowledge, Reasoning and Planning for Intelligent Characters,”
Proc. Siggraph, ACM Press, 1999, pp. 29–38.
3. S.R. Musse and D. Thalmann, “A Model of Human Crowd
Behavior: Group Inter-relationship and Collision Detection
Analysis,” Proc. 1997 Eurographics Workshop Computer Animation
and Simulation, Eurographics Assoc., 1997, pp. 39–52.
4. A. Treuille, S. Cooper, and Z. Popovi, “Continuum Crowds,”
ACM Trans. Graphics (Proc. Siggraph), vol. 25, no. 3, 2006,
pp. 1160–1168.
5. J. Lasseter, “Principles of Traditional Animation Applied to
3D Computer Animation,” Proc. Siggraph, 1987, ACM Press,
pp. 35–44.
6. R. Barzel, “Faking Dynamics of Ropes and Springs,” IEEE
Computer Graphics and Applications, vol. 17, no. 3, 1997, pp.
31–39.
7. M. Kass and J. Anderson, “Animating Oscillatory Motion
with Overlap: Wiggly Splines,” ACM Trans. Graphics (Proc.
Siggraph), vol. 27, no. 3, 2008, pp. 1–8.
8. R. Grzeszczuk, D. Terzopoulos, and G. Hinton, “Neuro­
Animator: Fast Neural Network Emulation and Control of
Physics-Based Models,” Proc. Siggraph, ACM Press, 1998,
pp. 9–20.
9. A. Bruderlin and L. Williams, “Motion Signal Processing,”
Proc. Siggraph, ACM Press, 1995, pp. 97–104.
10. P.C. Litwinowicz, “Inkwell: A 2-D Animation System,” Proc.
Siggraph, ACM Press, 1991, pp. 113–122.
when an agent quickly stops, we might want it to
lean forward and back in a spring-like manner,
eventually coming to rest (see Figure 3).
To achieve this effect, we used a brain spring
to convert translational acceleration into damped
harmonic motion driving rotation around the
agent’s x-axis (leaning). Expanding on this, our
typical character’s brain contained springs for
translational velocity and acceleration for leaning, and rotational velocity and acceleration for
26
July/August 2009
banking. For example, we can filter the rotationalacceleration signal with a brain spring and add
it to the agent’s z-axis rotation to have the agent
bank left and right in a spring-like manner owing
to turn accelerations and decelerations. Figure 4
diagrams these relationships; for a demonstration
of the effect on a simple robot turning, stopping,
and starting, see Video 5.
At its core, the brain spring model uses signal processing to shape its input. One related example used
Massive and WALL•E
T
o execute the motion synthesis of our crowd agents,
we used Massive, an agent-based crowd simulation
system by Massive Software (www.massivesoftware.com).
With Massive, we created small motion programs called
“brains” to implement the techniques described in the
main article. A sketch presented at Siggraph 2007 describes
how we integrated Massive into our animation pipeline.1
use an unclamped “or” node so that the values aren’t
clipped at 0 to 1. Usually, it’s a good idea not to connect
the summation’s output directly to the agent curves but
to the joint just after the root joint that affects the whole
agent. Once the brain spring is working, make a macro out
of the output and sum nodes so that you can easily reuse it.
Motion Blending
Bots on Paths
The robot crowds in WALL•E traveled mainly on predetermined paths in the spaceship Axiom. In Massive, this is most
easily implemented with the lanes feature. Lanes conveniently provide a “look ahead” function that’s suitable for
calculating the angle necessary for computing “anticipation
banking” (see the section “Anticipation” in the main article).
Implementing Brain Springs
To implement a brain spring in Massive, use an ouput node
for each bounce (half oscillation), and enter the filter width
and delay into the filter and delay fields. To sum the bounces,
filtering to add overshoot and undershoot to animation signals.3 Our technique builds on these observations and uses filtering to approximate damped
simple harmonic motion. We model each half of
an oscillation (or bounce) as a filtered, shifted, and
scaled version of the original signal, where increasing filter widths, increasing delays, and decreasing
scales at alternating signs are used as we accumulate the bounces to create the desired pattern.
Figure 5a shows this process’s mathematical
formulation; Figure 5b provides a more intuitive
illustration. It shows how we progressively filter,
scale, and shift an initial negative acceleration input to form each half bounce of a two-oscillation
spring. We can tune a brain spring’s parameters
to achieve a range of motions from small, highfrequency oscillation to broad, slow oscillation
(see Figure 6).
This system provides convincing reaction of the
root transforms for each character. However, many
of our robots had limbs that would naturally react
to forces differently than the body as a whole. To
model such limbs’ overlapping motion, we applied
the same brain springs to individual joint rotations with varying amplitudes and frequencies.
We added more filtering and delay to the brain
springs as the joints got further down the skeletal
chain (see Figure 7). The resulting agent would
then react to a sudden force first at the root level,
with springy motion propagating up the skeleton
toward each limb’s tip, providing convincing reaction and overlap of motion (see Video 6).
This term refers to interpolating different animation data
to create a new animation that’s a mix of the original data.2
In Massive, this is simply called a blend and is useful for
modifying animation cycles in crowd simulation.
References
1. D. Ryu and P. Kanyuk, “Rivers of Rodents: An AnimationCentric Crowds Pipeline for Ratatouille,” Tech. Memo 07-02,
Pixar, 2007.
2. A. Bruderlin and L. Williams, “Motion Signal Processing,”
Proc. Siggraph, ACM Press, 1995, pp. 97–104.
Agent rotates around x-axis
Moving forward
Large negative translational acceleration
Physics Driving Acting
The climactic sequence in WALL•E involved a
scene with complex physical interaction between
thousands of characters and their environment
(we couldn’t fake this one with signal processing).
In particular, the spaceship Axiom’s human passengers fall from their levitating chairs, bounce
off one another and their remaining chairs, which
pivot like bar stools, to collect on the ship’s side.
It initially seemed best to model the agents’
overall motion with a rigid-body simulation. Accordingly, we integrated a rigid-body simulation
engine into our pipeline (see the “Open Dynamics Engine” sidebar). This provided our characters’
root transformations, on top of which we layered
motion cycles and procedural animation. To do so,
Figure 3. An
agent moving
forward comes
to a stop and
leans forward.
Note how the
“large negative
translational
acceleration”
signal causes
rotation around
the x-axis. To
set up this
relationship,
we use a brain
spring.
IEEE Computer Graphics and Applications
27
Virtual Populace
Translational
velocity in z
Brain spring
Rx offset
Open Dynamics Engine
T
(Lean)
Translational
acceleration in z
Brain spring
Rx offset
Brain spring
Rz offset
(Lean bounce)
Rotational
velocity around y
he rigid-body simulator we integrated into
our pipeline is the freely available Open
Dynamics Engine (ODE). For the tilting simulation, we represented the ship geometry with
a smoothed height map, and the agents as
combinations of a cube and sphere. The simple
collision bodies yielded more-stable simulations
than arbitrary meshes yield. For a demo of a
simplified rigid-body simulation, see Video 7 at
http://computingnow.computer.org/wall-e.
(Banking)
Rotational
acceleration around y
Brain spring
Rz offset
(Banking bounce)
Figure 4. Relationships between physical inputs and animation outputs
using brain springs. Rx stands for rotation around the x-axis; Rz stands
for rotation around the z-axis. Altogether, the figure shows four springs:
two to drive leaning and two to drive banking.
bspring ( x ) =
∑ filter (x , w , d ) ∗ a
i
i
i
i ∈[0 ,n ]
a is a set of n amplitudes with monotonically decreasing magnitudes and
alternating signs—for example, n = 4, a = [-2, 1, -0.5, 0.25].
w is a set of n filter widths with monotonically increasing values—for
example, n = 4, w = [0.5, 1, 2, 4].
d is a set of n delays with monotonically increasing values—for example,
n = 4, d = [0, 0.1, 0.2, 0.4].
(a)
First bounce
Third bounce
Fourth bounce
Second bounce
Rapid negative acceleration
(complete stop)
(b)
Figure 5. Brain springs: (a) the mathematical formulation and (b) the
corresponding graph of an example, which provides a more intuitive
illustration. The brain spring filters the original signal (red) four times to
create two oscillations.
28
July/August 2009
we used the same brain springs that we designed for
reaction. In addition to creating simple reaction rotations, the springs’ outputs drove motion blending
between custom animation cycles (for more details,
see “Motion Blending” in the “Massive and WALL•
­E” sidebar). So, the physical forces measured by the
springs appeared to influence the character’s acting.
For instance, for a single acting clip—“panicked
flailing”—our animators produced a version under
the influence of forces from the six axis directions
(see Video 8). Then, on the basis of the translational
acceleration’s magnitude and direction, the characters blended their flailing accordingly (see Figure 8).
That is, if an agent received a negative z translational velocity, its animation clip was blended with
the version being pushed backward, on the basis of
the signal filtered through a brain spring.
For example, Video 9 shows an agent being
pushed backward in a simple simulation without
blending animation clips. Video 10 shows that
same simulation with translational velocity driving a motion blend with the clips shown in Figure
8, and is noticeably more realistic.
This let us use the rigid-body simulation’s results
to influence our agents’ acting, bringing the crowd
to life. Once we simulated this motion, we applied
a soft-body simulation to the close-up characters
requiring it.
For information on a workflow demo for the
climactic scene in WALL•E, see the “Shot Progression” sidebar.
Results
We developed these fake-physics techniques to create motion for crowd simulation that was compelling, easy to control, and cheap. You can judge for
yourself by watching WALL•E, but we’ve also provided a short demonstration of our crowd robots
in flight (see Video 17). The robots clearly aren’t
just following a path but are hovering, anticipating their turns, and reacting to physical forces in
a spring-like manner.
Bottom heavy—
stiff spring
Second bounce
Moving forward
Bounce 2
First bounce
Bounce 1
Large negative translational acceleration
A1: 180
A2: –40
Rapid negative acceleration
F1: 0.25
F2: 0.50
Moving forward
Second bounce
Bounce 1
Large negative translational acceleration
A1: –160
A2: 80
Brain springs are also remarkably easy to control, tune, and adapt for novel tasks. At one point
in the film’s production, the director asked for a
group of crowd robots to come to a complete stop
and bounce, yet begin their bounce before the stop
for exaggeration. This would be impossible for a
traditional physics simulation because the desired
result isn’t physically plausible. However, with brain
springs, this is a small modification. Rather than
use the real accelerations to drive the brain springs,
we painted a custom texture map on the ground in
Massive, containing bright pixels where the agents
should begin to bounce. The agents queried this
texture map and remapped its information into
the brain springs to trigger early bounces (see Video
18). So, brain springs can also filter nonphysics signals, allowing for highly customizable behavior.
As for speed and interactivity, we tested performance by running several 1,000-frame,
1,000-agent robot flight simulations with and
without brain springs (four springs as described
in Figure 4). With brain springs, the time increase was 287 seconds on average, giving a perframe, per-agent cost of 0.000287 seconds. This
cost was negligible compared to the other simulation aspects (for example, posing, ray casting,
and OpenGL drawing time). So, brain springs
proved highly interactive.
D2: 0.250
First bounce
Top heavy—
loose spring
Bounce 2
D1: 0.125
Rapid negative acceleration
F1: 0.50
F2: 1.0
D1: 0.250
D2: 0.500
W
e believe this research can easily be generalized to crowd simulation tasks other than
robots for WALL•E. Characters with more complicated skeletal hierarchies, such as animals, could
benefit from springy motion overlap in limbs using the skeletal-chain extension of brain springs.
Also, hovering or driving characters, common in
interactive games, could benefit from our techniques’ cheap cost and ease of implementation.
Potentially, brain springs could find use outside
computer graphics as a general filter for arbitrary
signals, although we expect the most common applications will be crowds and games.
Acknowledgments
I thank Pixar Animation Studios and Massive Software for support.
References
1. J. Lasseter, “Principles of Traditional Animation
Applied to 3D Computer Animation,” Proc. Siggraph,
1987, ACM Press, pp. 35–44.
2. C.W. Reynolds, “Flocks, Herds and Schools: A
Distributed Behavioral Model,” Proc. Siggraph, ACM
Press, 1987, pp. 25–34.
3. J. Wang et al., “The Cartoon Animation Filter,” ACM
IEEE Computer Graphics and Applications
Figure 6. Two
brain spring
configurations
and the
resulting effect
on a translating
flying agent
quickly
stopping. A1,
F1, and D1 are
the amplitude
scale, filter
width, and
delay of the
first half
oscillation; A2,
F2, and D2 are
those for the
second half
oscillation. F
and D values
are in seconds.
(a) A “bottomheavy stiff
spring” effect.
The agent first
leans forward
when stopping
(indicating
the center of
mass is in the
agent’s lower
half), then
quickly leans
back with low
magnitude and
high frequency.
(b) A “topheavy loose
spring” effect.
The agent first
leans back
(bottom swings
forward) when
stopping
(indicating the
center of mass
in the agent’s
upper half),
then slowly
leans forward
with high
magnitude and
low frequency.
29
Virtual Populace
Brain spring
2×
Clavicle
Rx offset
Brain spring
4×
Shoulder
Rx offset
Translational
acceleration in z
Brain spring
8×
Elbow
Rx offset
Translational
acceleration in z
Brain spring
16×
Hand
Rx offset
Translational
acceleration in z
Translational
acceleration in z
Elbow
(8× filter width
and delay)
Hand:
(16× ...)
Shoulder
(4× filter width and delay)
Clavicle
(2× filter width and delay)
Large negative translational
acceleration
Moving forward
Figure 7. Modeling a robot limb’s overlapping motion: (a) a flowchart showing how the input motion data is related to animation
outputs through brain springs and (b) the corresponding schematic. The skeletal hierarchy (blue) experiences a large negative
translational acceleration, and the brain springs (orange) applied to each joint rotate the bones. The larger filter widths and
delays cause each successive joint to react more gradually to the force, creating overlapping motion in the limb.
Trans. Graphics (Proc. Siggraph), vol. 25, no. 3, 2006,
pp. 1169–1173.
Blend into upward-impact animation clip
bspring
Acceleration
downward
Acceleration
right
bspring
bspring
Blend into
left-impact
animation clip
Acceleration
left
bspring
Acceleration
upward
Blend into
right-impact
animation clip
Blend into downward-impact animation clip
Figure 8. How forces from each direction drive motion blends. In
practice, this let us give the impression that each agent’s acting was
being influenced by physical forces.
Paul Kanyuk is a technical director at Pixar Animation Studios, with credits on the films Cars, Ratatouille, WALL•E, and Up. His specialty is crowd
simulation, and he’s responsible for the procedural
animation of numerous crowd spectacles, including
the hordes of rats in Ratatouille, the deluge of falling passengers in WALL•E, and the vicious pack of
dogs in Up. He also teaches courses in RenderMan and
crowd simulation at the Academy of Art University in
San Francisco. Kanyuk has a BS Eng. in digital media
design from the University of Pennsylvania. He’s a
member of ACM Siggraph and Tau Beta Pi. Contact
him at pkanyuk@gmail.com.
Shot Progression
T
o show how our physics-based acting workflow looks in action, six videos at http://
computingnow.computer.org/wall-e present a
rough shot progression for the scene in WALL•E
where the Axiom tilts and the passengers slide
across the deck (see the section “Physics Driving
Acting” in the main article). Video 11 shows part
of the rigid-body simulation used to generate
30
July/August 2009
the final animation. Video 12 shows the acting of
individual agent types independent of the rigidbody simulation. Video 13 shows the noisy simulation with a subset of the final agents with acting
layered on top. Video 14 shows the agent positions
after filtering and smoothing. Video 15 shows the
unlit shot with final animation and cleanup. Video
16 shows the final result with lighting.