Greg Maguire (2010)
High-level Facial
Animation Interface
Video game
[Piloted in 'Star Wars:The Clone Wars' (2008) [:ilm
and episodic TV series] ; 'Rock Band
2' (Harmonix
2008) [game] and implemented in 'A New Dawn'
(2010)
ABSTRACT
One of the greatest challenges in computer animation is the
realistic portrayal of the human face.
As realism within computer generated lighting models
increases through the use of algorithms and mathematical
principles derived from real world lighting, an artist with an
understanding and knowledge of lighting principles can easily
generate a photographic still life. The complexity required to
generate motion through animation on organic surfaces
particularly the human face to express believable emotion to
entertain a sophisticated audience must be able to attain the
same realism.
A high-level interface that can mask the complex systems
required to achieve facial realism enables the animator to
achieve the most appropriate performance for a character.
In 2005, I was invited to join, LucasLilm Animation a division
of George Lucas’ LucasLilm, as Research and Development
Supervisor on what would become Emmy Award-winning Star
Wars: The Clone Wars (2008), I identiLied the facial
performances of the Lilm to have critical importance to its
success. This provided an opportunity to research facial rigging
within the context of a high-end episodic television series.
The three main issues to be addressed in creating a facial
animation system are the form of the head, its articulation and
the controls presented to the animator.
The form of the head must a) represent the character and b) be
modeled in an appropriate way to enable articulation, i.e. the
topology of the edge and vertex arrangement should naturally
Llow along the areas of deformation to avoid unsightly folds
giving away the polygonal nature of the object.
Although the face is primarily driven by a series of muscles, it
is the outward appearance and deformations of the skin from
it’s neutral position towards an extreme that are interpreted as
expressions. In Animating Facial Expressions (Platt and Badler,
1981), describes how to construct expressions using musclebased facial model. Their work used the Facial Action Coding
System (FACS, Ekman, Friesen, 1978) to determine the most
appropriate articulations of the human face. In Stop Staring
(Osipa, Sybex, 2003), Jason Osipa’s describes a method of
moving objects within a scene that presents a familiar interface
to an animator enabling them to quickly grasp their function
and editing of their timing curves.
Artist Name: Output Number: Output Title: Main Publication: Year:
Greg Maguire 2 of 4
High-Level Facial Animation Interface
Video Game 2010
Research Methodologies
My approach would be to standardise a topology between all
the characters in the series, build upon Platt and Badler’s
primitive FACS based model and apply it to a stylised character
consisting of much greater resolution and to build upon Osipa’s
interface optimising its layout, add more functionality and
enable an animator to push the character beyond what would
be perceived to be exposed to them.
The Facial Action Coding System (FACS, Ekman, Friesen,
1978)is the most widely used method to describe facial
behavior worldwide. It is a comprehensive, anatomically based
system for describing all observable facial movement and is
recognized as the gold standard in facial measurement in both
academia and industry. FACS describes all observable facial
movement on the basis of 44 visually discernible action units
(AUs), each with a deLinable underlying muscular basis, by
describing how the AU moves and enumerating the changes in
appearance in facial topography (via furrows, lines, muscle
bulges, positions, etc) that occur when each AU or muscle
group contracts.
Form
Facial Action Coding System. Copyright © 1978 Ekman and Friesen. All Rights
Reserved.
Neutral poses of Anakin Skywalker & Ashoka from Star Wars: The Clone Wars
Copyright © 2008 LucasLilm Animation and Warner Bros.
The edgeLlow should follow the direction of the contracting
muscle. eg. in Action Unit 1 (AU1) the frontalis muscle is
anchored in the tissues of the scalp and attached to the skin at
the eyebrows. It runs vertically on the forehead and pulls in an
upward direction. In order for the volume to remain
consistent, the skin folds upon itself creating furrows running
horizontally across the forehead.
Action Unit 4 (AU4), describes the corrugator muscles
contraction originating at the inner orbit of the eye near the
root of the nose and inserting into the skin of the forehead
above the centre of each brow. It pulls the eyebrows and skin
from the center of each eyebrow to its inner corner medially
and down, forming vertical wrinkles in the glabella area and
horizontal wrinkles at the bridge of the nose.
Combining the natural direction of AU1 and AU4 creates a grid
like pattern across the brow that dips towards the root of the
nose. Designing and incorporating a facial topology that can
best mimic the skin deformations of all 44 Action Units creates
an efLicient computer model without exposing any artifacts
that would betray the polygonal foundations of the character.
Articulations
Action Unit 1 (AU1) performed by Neutral Pose of Heavy Singer in Rock Paul
Ekman. Band 2.
Action Unit 1 (AU1) target geometry of Heavy Singer in Rock Band 2.
There are two main methods to deform geometry a) assigning
vertices to a number of transformation centres, known as
skinning, enveloping or binding. Moving the transform centre
moves its attached vertices linearly based upon their weighted
assignments or b) shape animation, manually moving each
vertex to it’s extreme position for the AU and saving it as a
target enabling the software to morph from the neutral face to
the target AUs.
Shape animation is both labour intensive and relies heavily on
the skills of a modeler whereas enveloping relies more on the
skills of the rigger to position and assign the vertex weights. I
chose to use shape animation as they give a more precise
targets as each vertex can be moved individually.
A series of individual models to act as extreme targets would
therefore have to be sculpted to cover all 44 Action Units and
their symmetry, 108 targets in all.
Animation Controls
Targets attached to sliders. Copyright © 2003 Maguire. All Rights Reserved.
Jason Osipa describe a method of attaching an object to a target
slider. Describing a simple box, with sides north, south, east
and west. If the transform is at the centre of the box it’s
coordinates are (0,0) in two-dimensional space, with the Z-axis
being ignored. When the transform is in the north position, the
coordinates are (0,1), south (0,-1), east (1,0) and west (-1,0). A
relationship can be described in mathematical terms between
the output of the slider and a mapped target shape.
Modeling 108 targets results in 108 sliders. In the above image
only 22 sliders can be seen at any one time. For the animator to
move additional Action Units they must scroll the window to
expose its slider resulting in a frustrating and inefLicient
workLlow.
Targets which are naturally mutually exclusive can be summed
at the same time eg. AU26, Jaw Open can be invoked at the
same time as AU31 Jaw Clench. It is a challenge for an animator
to keep track of all 108 shape targets and editing their
contribution to the expression of the character.
Greg Maguire
Osipa Interface. Image Copyright © 2003 Sybex. All Rights Reserved.
AU12, Lip Corner Puller
AU18, Lip Pucker
S
(0,-1)
AU31, Jaw Clench
(0,1)
N WE
(0,
0)
(-1,0)
(1,0)
Box widget controlling mouth for Nvidia Corporation. Copyright © 2010
Maguire and Nvidia. All Rights Reserved.
When the transform object, in my case a nurb circle, is at north
position (0,1) then AU31, Jaw Clench target is invoked, when
the nurb circle is at south position (0,-1) then AU26, Jaw Open
is invoked. This places target shapes on one slider, cutting the
amount of controls required in two. It also places the targets at
opposing ends of the slider making it impossible to have both
on at the same time preserving their mutual exclusivity.
The layout of Osipa’s interface wasn’t optimised for an
animator in full production. The brow controls were
particularly challenging to operate as the box controls
relationship with the shapes was not obvious. Creating a bank
of single sliders, enabled an animator to rectangle select them
all at one time and only resulted in one additional motion
curve to edit.
I initially tried tried using expressions as described by Osipa
but found the linear link between the control object and the
target limiting. Instead, I opted for a relationship curve. This
enabled a Liner degree of control and allowed for a fall-off
when the control object was pushed passed it’s normalised
threshold allowing the animator to push past the extreme
target without hitting a hard clamp.
Selected
Research Outcomes
Star Wars: The Clone Wars is an Emmy Award-winning
American 3D CGI animated television series from George
Lucas. It is produced by LucasLilm Animation in California,
Singapore and Taiwan. The season launched in August 2008
with an animated feature. It Linished in 2013 after 108
episodes.
Star Wars: The Clone Wars became most-watched series
premiere in Cartoon Network history. The series averaged 3
million total viewers in its debut according to Nielsen Media
Research. Craig Glenday, editor of the Guinness World Records
presented Star Wars: The Clone Wars a certiLicate proclaiming
the cartoon series "the highest rated sci-Li animation currently
on television".
Artist Name: Output Number: Output Title: Main Publication: Year:
Greg Maguire 2 of 4
High-Level Facial Animation Interface
Video Game 2010
Rock Band 2: the video game was released in 2008 by
Harmonix Music Systems and distributed by Electronic Arts.
The Xbox 360 version was the 3rd biggest selling game in USA
selling 363,000 copies and achieved a 92% score on Metacritic.
Rock Band 2‘s total sales were 1.7 million units in the USA
during 2008. Game informer went on to list it within the “Top
50 Games of 2008” and one of the top 200 games of all-time.
Greg Maguire
New Dawn: Real-time facial animation system for Nvidia
Corporation to demonstrate the capability of their new Titan
GPU. Nvidia is a US based global technology company in
California with assets of $5.5 billion. Their GPUs are in most of
the worlds smartphones.