Muscle Based Facial
Animation
Jason Jerald
April 7, 2004
Overview
•
•
•
•
History
Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
Improvements
– Breton, Bouville and Pele 2001
– Bui and Nijholt 2003
Facial Muscle History
• 19th century - physiologist Duchenne
– applied electrical currents to freshly guillotined heads
to observe facial contortions
– Later applied same technique to old inmates of alms
houses to create artificial expressions
– Recorded with photography
• 1977 - Psychologists Ekman and Friesden
– Created the Facial Action Coding System (FACS)
– Notational-based environment that determines
emotional states from visible facial distortion
– Individual muscles are described as Action Units (AU)
– This work is commonly used in computer facial
animation
Facial Muscle History
• 1980 - Platt published masters thesis on a
physically based muscle–controlled facial
expression model
• 1987 - Waters published the seminal
paper on muscle based facial animation
using muscle vectors
• 21st century – improvements on Waters
model
Overview
•
•
•
•
History
Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
Improvements
– Breton, Bouville and Pele 2001
– Bui and Nijholt 2003
Various Muscle Models
• Free form deformations
– Deforms objects by manipulating control points arranged in a 3d
cubic lattice
– Surface regions corresponding to anatomical descriptions of the
muscle actions are defined
– Displacing control point is analogous to actuating a physically
modeled muscle
– More Intuitive than vector representations but cannot model
furrows, bulges, and wrinkles.
• Spline psuedo muscles
–
–
–
–
Deforming facial mesh in muscle-like fashion
Ignores underlying anatomy
Supports smooth and flexible deformations
Hierarchical splines allow more detail in specified regions
Various Muscle Models
• Mass-spring methods
– Forces applied to elastic meshes through muscle arcs
– Muscles represented as collections of functional blocks
– Action units created by applying muscle forces to deform the
spring network
• Layered spring meshes
– models skin, fatty tissue, and muscle tied to bones
– Spring elements connect each mesh node and each layer
– Realistic but computationally expensive
• Vector representations
– What this talk focuses upon
Overview
•
•
•
•
History
Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
Improvements
– Breton, Bouville and Pele 2001
– Bui and Nijholt 2003
Muscle Vector Model
• What is needed?
– Few dynamic parameters that emulate the primary
characteristics of facial expression
– Linear/parallel muscles that pull and sphincter
muscles that squeeze
– Factors determining nodal mobility are
•
•
•
•
•
Tensile strength of the muscle and skin
Proximity to the muscle node of attachment
Proximity to the bone
Elastic bounds of the relaxed tissue
Interaction of other muscles
Muscle Vectors
• Models the actions of muscles upon skin
• Each muscle has a zone of influence
• A muscle includes vector field direction, an
origin, and an insertion point
Advantages / Disadvantages
• Advantages
– Independent of facial mesh (facial mesh can be
exchanged)
– Compact representation
– Expression parameters can control groups of muscles
– Fast
• Disadvantages
– Positioning of muscles can be time consuming
– Does not take curvature into account
– Artifacts when a mesh vertex is under the influence of
multiple muscle actions
Muscle Vector Parameters
• Muscle attached at two points
– Point of Attachment A – the root
of the muscle attached to the
bone
– Point of insertion I into the flesh
flesh
• Muscle can therefore be
bone
considered as the vector AI
• With no contraction the points of attachment and
insertion do not move and the muscle vector maintains
its length
• Acts like a magnet attracting all the vertices within its
zone of influence. The skin contracts more near the
muscle.
Muscle Vector Parameters
• Parameters
– V is the mesh vertex
– Opening angles
• β is the opening angle
• α is the maximum angular limit
• Muscle contraction is faded as β raises to α
– Radial distances as a proportion of |AI|
• S is where the muscle influence starts to fade
• E is where the muscle influence ends
• Vertices are faded if they are in the band defined
by S and E
Equations
• C is the contraction factor (between 0 and 1)
•  A is the fading coefficient related to the angular
distance between AV and AI
•  R is the fading coefficient related to the radial
distance between V and S if V is in the fading band SE
Muscle Results
Sphincter / Mouth Muscle
• Waters models the mouth with a sphincter
muscle
– Described from a single point around which the
surface contracts as if drawn together like a string
bag
– Longitudinal and vertical axii allow elliptical shape
Sphincter muscle
Elliptical Sphincter muscle
Muscle sets
Facial Action Coding System (FACS)
• Developed by Psychologists Ekman and Friesden in 1977
• FACS is Description of facial muscles and jaw/tongue derived from
analysis of facial anatomy
• Notational-based system that determines emotional states from
visible facial distortion
• Action Units (AU) correspond to muscle vectors
Waters Results
Overview
•
•
•
•
History
Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
Improvements
– Breton, Bouville and Pele 2001
– Bui and Nijholt 2003
Breton et al
• Non-muscle parametric animation mixed
with muscle vectors (jaw, eyes, eyelids,
neck)
• Opening of mouth with a muscular system
requires distinction between lower and
upper lips
The eyes
• Eyes and Eyelids
– Modeled as spheres and hemispheres
– Simple rotations
– Random blinking
– Gaze looking forward when speaking
– Random gaze direction when not speaking
•
Jaw and Neck
• Jaw
– Single axis of rotation
– Lower lips not within jaw
influence
• Neck
– Three axii of rotation
– Center of rotation is the
center of the neck
bounding box
– Vertices of the head are
fully rotated
– Rotations of the neck
linearly faded with distance
Jaw boundary
Neck boundary
Lips
• Distinction between upper and lower lips must be made
in order to open mouth
• Distinction between upper and lower lips determined at
load time
Breton et al 2001 results
Overview
•
•
•
•
History
Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
Improvements
– Breton, Bouville and Pele 2001
– Bui and Nijholt 2003
Bui et al
• Multiple muscle action artifacts removed
by simulating parallelism
• Division into regions
• Wrinkles
Muscle action artifacts
• The problem
– Problem when a mesh vertex is under the influence of multiple
muscle actions
– Muscle actions are independent
– Actual nodal displacement determined by a succession of
muscle actions
– Unnatural results occur when a vertex is shifted outside the zone
of influence of adjoining muscle vectors
• The solution
– Combining muscle contractions done by simulating parallelism
– For a vertex inside multiple muscles’ zone of influence, small
units of contraction levels are applied until no more contraction
to apply
– Step sizes of 20% of full contraction found to have good results
Region Division
• Allows easier rendering
of special parts of the
face such as lips and
eyebrows
• Reduces artifacts
generated by
displacement of vertices
in regions that are not
affected by a muscles
contraction
Muscle Action Artifacts
Two muscles with no parallelism
Two muscles with parallelism
Three muscles with parallelism
Wrinkles
• Assume muscles lie parallel to the facial skin
and heights of wrinkles are the same
• Height and number of wrinkles are predefined
for each muscle
• To make wrinkles more visible use triangular flat
shading at vertex where wrinkle starts
Bui et al results (2003)
happy
neutral
Sad with close up of wrinkles
surprise
Demo
Conclusion
• Waters is the basic model
• Additional tricks can be added to improve
appearance
• Facial expressions can be defined by
muscle groups while individual muscles
can also be controlled.
• Simple yet effective and fast
References
• Breton, G., Bouville, C., and Pel, D. 2001. FaceEngine a 3d Facial
Animation Engine for Real Time Applications. in Proceedings of 6th
International Conference on 3D Web Technology, pp. 15-22.
• Bui, T. D., Heylen, D., and Nijholt, A. 2003. Improvements on a
Simple Muscle-Based 3d Face for Realistic Facial Expressions. in
Proceedings of 16th International Conference on Computer
Animation and Social Agents, pp. 33-40.
• Noh, J.-Y. and Neumann, U. 1998. A Survey of Facial Modeling and
Animation Techniques. USC Technical Report No. 99-705.
• Parke, F. I. and Waters, K. (1996) "Chapter 7 Skin and MuscleBased Facial Animation." In Computer Facial Animation, pp. 223257.
• Waters, K. 1987. A Muscle Model for Animating Three-Dimensional
Facial Expression. in Proceedings of 14th Annual Conference on
Computer Graphics and Interactive Techniques, pp. 17-24.