Muscle Based Facial Animation

Muscle Based Facial
Jason Jerald
April 7, 2004
Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
– Breton, Bouville and Pele 2001
– Bui and Nijholt 2003
Facial Muscle History
• 19th century - physiologist Duchenne
– applied electrical currents to freshly guillotined heads
to observe facial contortions
– Later applied same technique to old inmates of alms
houses to create artificial expressions
– Recorded with photography
• 1977 - Psychologists Ekman and Friesden
– Created the Facial Action Coding System (FACS)
– Notational-based environment that determines
emotional states from visible facial distortion
– Individual muscles are described as Action Units (AU)
– This work is commonly used in computer facial
Facial Muscle History
• 1980 - Platt published masters thesis on a
physically based muscle–controlled facial
expression model
• 1987 - Waters published the seminal
paper on muscle based facial animation
using muscle vectors
• 21st century – improvements on Waters
Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
– Breton, Bouville and Pele 2001
– Bui and Nijholt 2003
Various Muscle Models
• Free form deformations
– Deforms objects by manipulating control points arranged in a 3d
cubic lattice
– Surface regions corresponding to anatomical descriptions of the
muscle actions are defined
– Displacing control point is analogous to actuating a physically
modeled muscle
– More Intuitive than vector representations but cannot model
furrows, bulges, and wrinkles.
• Spline psuedo muscles
Deforming facial mesh in muscle-like fashion
Ignores underlying anatomy
Supports smooth and flexible deformations
Hierarchical splines allow more detail in specified regions
Various Muscle Models
• Mass-spring methods
– Forces applied to elastic meshes through muscle arcs
– Muscles represented as collections of functional blocks
– Action units created by applying muscle forces to deform the
spring network
• Layered spring meshes
– models skin, fatty tissue, and muscle tied to bones
– Spring elements connect each mesh node and each layer
– Realistic but computationally expensive
• Vector representations
– What this talk focuses upon
Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
– Breton, Bouville and Pele 2001
– Bui and Nijholt 2003
Muscle Vector Model
• What is needed?
– Few dynamic parameters that emulate the primary
characteristics of facial expression
– Linear/parallel muscles that pull and sphincter
muscles that squeeze
– Factors determining nodal mobility are
Tensile strength of the muscle and skin
Proximity to the muscle node of attachment
Proximity to the bone
Elastic bounds of the relaxed tissue
Interaction of other muscles
Muscle Vectors
• Models the actions of muscles upon skin
• Each muscle has a zone of influence
• A muscle includes vector field direction, an
origin, and an insertion point
Advantages / Disadvantages
• Advantages
– Independent of facial mesh (facial mesh can be
– Compact representation
– Expression parameters can control groups of muscles
– Fast
• Disadvantages
– Positioning of muscles can be time consuming
– Does not take curvature into account
– Artifacts when a mesh vertex is under the influence of
multiple muscle actions
Muscle Vector Parameters
• Muscle attached at two points
– Point of Attachment A – the root
of the muscle attached to the
– Point of insertion I into the flesh
• Muscle can therefore be
considered as the vector AI
• With no contraction the points of attachment and
insertion do not move and the muscle vector maintains
its length
• Acts like a magnet attracting all the vertices within its
zone of influence. The skin contracts more near the
Muscle Vector Parameters
• Parameters
– V is the mesh vertex
– Opening angles
• β is the opening angle
• α is the maximum angular limit
• Muscle contraction is faded as β raises to α
– Radial distances as a proportion of |AI|
• S is where the muscle influence starts to fade
• E is where the muscle influence ends
• Vertices are faded if they are in the band defined
by S and E
• C is the contraction factor (between 0 and 1)
•  A is the fading coefficient related to the angular
distance between AV and AI
•  R is the fading coefficient related to the radial
distance between V and S if V is in the fading band SE
Muscle Results
Sphincter / Mouth Muscle
• Waters models the mouth with a sphincter
– Described from a single point around which the
surface contracts as if drawn together like a string
– Longitudinal and vertical axii allow elliptical shape
Sphincter muscle
Elliptical Sphincter muscle
Muscle sets
Facial Action Coding System (FACS)
• Developed by Psychologists Ekman and Friesden in 1977
• FACS is Description of facial muscles and jaw/tongue derived from
analysis of facial anatomy
• Notational-based system that determines emotional states from
visible facial distortion
• Action Units (AU) correspond to muscle vectors
Waters Results
Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
– Breton, Bouville and Pele 2001
– Bui and Nijholt 2003
Breton et al
• Non-muscle parametric animation mixed
with muscle vectors (jaw, eyes, eyelids,
• Opening of mouth with a muscular system
requires distinction between lower and
upper lips
The eyes
• Eyes and Eyelids
– Modeled as spheres and hemispheres
– Simple rotations
– Random blinking
– Gaze looking forward when speaking
– Random gaze direction when not speaking
Jaw and Neck
• Jaw
– Single axis of rotation
– Lower lips not within jaw
• Neck
– Three axii of rotation
– Center of rotation is the
center of the neck
bounding box
– Vertices of the head are
fully rotated
– Rotations of the neck
linearly faded with distance
Jaw boundary
Neck boundary
• Distinction between upper and lower lips must be made
in order to open mouth
• Distinction between upper and lower lips determined at
load time
Breton et al 2001 results
Types of Facial Muscle Models
Muscle Vectors (Waters 1987)
– Breton, Bouville and Pele 2001
– Bui and Nijholt 2003
Bui et al
• Multiple muscle action artifacts removed
by simulating parallelism
• Division into regions
• Wrinkles
Muscle action artifacts
• The problem
– Problem when a mesh vertex is under the influence of multiple
muscle actions
– Muscle actions are independent
– Actual nodal displacement determined by a succession of
muscle actions
– Unnatural results occur when a vertex is shifted outside the zone
of influence of adjoining muscle vectors
• The solution
– Combining muscle contractions done by simulating parallelism
– For a vertex inside multiple muscles’ zone of influence, small
units of contraction levels are applied until no more contraction
to apply
– Step sizes of 20% of full contraction found to have good results
Region Division
• Allows easier rendering
of special parts of the
face such as lips and
• Reduces artifacts
generated by
displacement of vertices
in regions that are not
affected by a muscles
Muscle Action Artifacts
Two muscles with no parallelism
Two muscles with parallelism
Three muscles with parallelism
• Assume muscles lie parallel to the facial skin
and heights of wrinkles are the same
• Height and number of wrinkles are predefined
for each muscle
• To make wrinkles more visible use triangular flat
shading at vertex where wrinkle starts
Bui et al results (2003)
Sad with close up of wrinkles
• Waters is the basic model
• Additional tricks can be added to improve
• Facial expressions can be defined by
muscle groups while individual muscles
can also be controlled.
• Simple yet effective and fast
• Breton, G., Bouville, C., and Pel, D. 2001. FaceEngine a 3d Facial
Animation Engine for Real Time Applications. in Proceedings of 6th
International Conference on 3D Web Technology, pp. 15-22.
• Bui, T. D., Heylen, D., and Nijholt, A. 2003. Improvements on a
Simple Muscle-Based 3d Face for Realistic Facial Expressions. in
Proceedings of 16th International Conference on Computer
Animation and Social Agents, pp. 33-40.
• Noh, J.-Y. and Neumann, U. 1998. A Survey of Facial Modeling and
Animation Techniques. USC Technical Report No. 99-705.
• Parke, F. I. and Waters, K. (1996) "Chapter 7 Skin and MuscleBased Facial Animation." In Computer Facial Animation, pp. 223257.
• Waters, K. 1987. A Muscle Model for Animating Three-Dimensional
Facial Expression. in Proceedings of 14th Annual Conference on
Computer Graphics and Interactive Techniques, pp. 17-24.