Helper Joints:
Advanced Deformations
on RunTime Characters
Jason Parks
Character Technical Director
Sony Computer Entertainment America
Contacts, Reference, Credits
• Email:
– jason@jason-parks.com
– jason_parks@playstation.sony.com
• Webpage (Helper Joints-GDC 2005):
– http://www.jason-parks.com/HelperJoints
• PAN Arts AVIs by Warwick Mellow
• Human Model by Sven Jenson via
Miller/Thuriot’s MasterClass 2003
(Character Creation Toolkit)
• BodyBlend render by Zach Gray
• Wrinkle R&D by Tyler Crook (SCEA)
2
Intended Audience
• “Advanced – Requires experience
and familiarity with subject.”
– Maya Technical Artists who ‘Bind Characters’ for RunTime or
Offline usage
•
•
•
•
•
Paint weights (Artisan and Component Editor)
Apply deformers
Use Set Driven Keys
Write Expressions
Use Utility Nodes
3
Topics
• What are Helper Joints?
– Definition
• Why use Helper Joints?
– New uses for Extra Joints
– Smooth binding sucks & Where to use Helper Joints
• How to use Helper Joints?
– Our Goals, Disclaimer, Fusiforms
– Research: The Basic Building Blocks
• RunTime Rigs
– SetDrivenKeys
– Expressions
– Constraints
• Non-RunTime Rigs
– Baking Out
– Spline IK
– Jiggle Deformer
• Automating Production
–
–
–
–
Scripting
Workflow
Muscle Systems
Procedurally generated Helper Joints
4
Stop Talking, No more text, show
some Moving Pictures!
5
Helper Joints Defined
• “Helper Joints” are secondary joints offset
from the standard hierarchy which will
translate, rotate, or scale to achieve a “hand
sculpted” affect on an area of vertices.
• They can be within the primary hierarchy
(parented, grouped, rigged), or outside of the
primary hierarchy (point or parentConstrained)
• They can be controlled by SetDrivenKeys,
Expressions, Simple Rigs (constraint system),
or Super Advanced Rigs (really any simulation
you can think of) that are baked-out.
6
Why use any Extra Joints?
Unlike characters rigged, bound, and rendered in
Maya or any other high-end 3D package, your
characters at run-time need to follow a very
specific rule set depending on your Game Engine.
Because all run-time engines already support
joints, it is a tool we can exploit and push to its
limits.
•
•
•
Traditional usage of Extra Joints to
animate Props and Accessories
Extra “Helper Joints” to fix Smooth
binding
Extra “Helper Joints” to create the
next level in realism
7
Traditional Extra Joints
• Traditional usage of Extra Joints for
Props and Accessories:
–
–
–
–
Props, Rope
Hair, Ponytails
Capes, Necklaces, Clothing
Fat
8
New Ideas for Extra Joints
•Extra “Helper Joints” to
create the next level in
realism:
–
–
–
–
Skin sliding
Wrinkles
Fat/Muscle/Flesh Jiggle
Muscle bulging
9
Fix Smooth binding
• Extra “Helper Joints” to make
skin transform properly:
– Volume maintenance
– Crease fixing
– Complex Areas
• Multiple joints affecting a single vert
10
Smooth Binding Limited
• Vertex deforms by a weighted
combination of Linear
Translations around a joint
rotating
11
Smooth Binding Shoulder
12
Smooth Binding Problem Areas
• Elbow and Knee:
– Interpenetration on the inside of the crease
– Loss of volume on the outside of the crease (i.e. – the elbow tip or knee cap do not become
‘pointy’ when the elbow or knee bends)
• Wrist/Forearm:
– Loss of volume when the forearm/wrist is twisted too much down the length of the arm. (i.e. ‘Candy-wrapper’ affect)
• Upper Thigh/Gluteus:
– Interpenetration at the bottom of the butt cheek when leg goes too far back
• Shoulders:
– Interpenetration in armpit
– Interpenetration at the neck/shoulder junction area (trapezius)
– Lack of representation of scapula movements
13
Our Goals – The “Ideal”
• What are we really trying to get?
• Reference with Motion
– Books
• “Anatomy of Movement” by Blandine C. Germain
– Internet
• PAN Arts by Warwick Mellow
– Software
• Absolute Character Tools for 3DSMAX
14
Our Goals
• PAN
Research
– Pectorals
– Shoulders
15
Our Goals
• PAN
Research
– Clavicle
– Forearm
16
Our Goals
• Muscles – Independent Research
17
Our Goals
• Muscles - Absolute Character Tools
18
Disclaimer: RunTime Rig Support
• Individual Developer code will dictate
what will be evaluated in Real-Time and
what type of relation you can specify
between your primary skeleton’s
animation and your Helper Joints’
transformation.
• Hopefully you can get 1 or more of
these RunTime Rigging methods
supported:
– SetDrivenKeys
– Expressions
– Constraints
19
Disclaimer: RunTime Rig Support
• NO:
– Source Code provided for engine side.
• YES:
– Techniques for art-side production of rigs (in Maya)
• Applicable to all games on all
consoles
20
How To Get There
• Muscles are the drivers in reality so
lets create Helper Joints that
mimic the way muscles work.
• Simple FusiForm muscles are easy
to mimic in Maya so lets create
those and bind them to our skin
21
Research – FusiForm Muscle
• “FusiForm” muscle: Quick Definition
– ‘Tapering at each end; spindle-shaped.’
22
Research – The Building Blocks
• Let’s pick the Biceps as our test case to
create a muscle and mimic its action
with a Helper Joint being driven by
SetDrivenKeys:
– Rotating
– Scaling
– Translating
• The research explores the underlying
mechanisms of the rigs
• The ‘failed’ attempts still illustrate how
rigs can be put together
23
Research – The Building Blocks
UNDERSTANDING HOW THE
RIGS WORK
IS MORE VALUABLE
THEN MY SUGGESTED
SOLUTIONS
24
Research – Biceps (NURBs)
• Muscle as
Influence Object NURBs
25
Research – Biceps (Polys)
• Muscle as
Influence Object Polygons
26
Research – Biceps (Short)
• SetDrivenKey w/
Single Driver Axis
– Rotating
• Short
27
Research – Biceps (Long)
• SetDrivenKey w/
Single Driver Axis
– Rotating
• Long
28
Research – Biceps (Scale)
• SetDrivenKey w/
Single Driver Axis
– Scaling
29
Research – Biceps (Translate)
• SetDrivenKey w/
Single Driver Axis
– Translate
30
Research – Upper Arm (Translate)
• Single Driver Axis
– Translate
– Works pretty good for Single driving axis!
Warning: Not anatomically correct!
31
Research – Biceps (RollBone)
• SetDrivenKey w/
Single Driver Axis
– Translate
– Add Roll
– BREAKS!
32
Research – Biceps (RollBone)
• SetDrivenKey w/
Two Driver Axis
– Translate
– Add Roll
– w/ Extra Keys compensating
for roll
– Only fixes in when arm is
straight out
– When arm is bent it still
BREAKS!
33
Research – Biceps (RollBone)
• Add parent joint w/ another set of driven
keys for roll
– Rotation of Helper’s parent joint is percentage of primary hierarchies
rotation
34
Research – Shoulder (Multiple
Drivers)
• SetDrivenKey w/
Single Driver Axis
from Two bones
– Translate
– Combination is difficult
35
Research – Shoulders (Multiple Drivers)
• SetDrivenKey w/
Two Driver Axis
from Two bones
– Translate
– 2 Driving bones =
BREAKS!(kinda)
– + 2 Driving axis =
BREAKS!BAD!
36
RunTime Rigs – SetDrivenKeys
• Summation
– SetDrivenKeys are fast, easy to use and work great for
“simple” Helper Joints
• 1 driveR joint and 1 driveR axis per driveN Helper Joint
• Nest hierarchies of driveN Helper Joints if you have more than 1
driveR joint or axis
37
RunTime Rigs – SDKs for Volume
Preservation
• Forearm Helper Joint w/ SDK’s for Scale and
rotation
38
RunTime Rigs – SDKs for Wrinkles
• Scaling of Helper Joints (R&D by Tyler
Crook @ SCEA)
39
RunTime Rigs – SDKs for Wrinkles
• Joints are scaling on 2-axis (bulging), no
scaling on length-wise axis
• No translation or rotation
40
RunTime Rigs – SDKs for Wrinkles
• Weighting is very tedious
41
RunTime Rigs – SDKs for Wrinkles
• For shirt it takes many more joints because
of 3 D.O.F. and larger surface.
– 6 joints to do it right
42
RunTime Rigs – SDKs for Wrinkles
• Different joints scale for each direction in
each D.O.F.
• Weighting is time consuming
43
RunTime Rigs – SDKs for Wrinkles
• Incorporation:
– Could theoretically add a scaling joint as a child of
the helper joints
– Then just weight individual vertices to these scaling
joints to cause ‘wrinkling’
– Or just make separate scaling wrinkle joints wherever
needed
44
RunTime Rigs
• Done with SDKs
• Start Expressions
45
RunTime Rigs – Expressions (Simple:Elbow)
• Simple Helper Joints which only need a
Translation Expression
– Based on 1 axis from driving bone
dj_helper.ty = (base.ty + (maxOffset * (elbow.rz /maxAngle)))
46
RunTime Rigs - Expressions
(Complex:Biceps)
• Complex Helper Joints which need Translation
& Rotation Expression
– Translation: Use another expression to translate dj_bicep joint (parented to
Bicep_Helper) for bulge based on elbow rotation.
• Expression 2 – Describes bulge from rotation of elbow’s Z-axis
dj_bicep.ty = (base.ty + (maxOffset * (elbow.rz /maxAngle)))
base.ty = 5, maxOffset = 1.8, maxAngle = 110
– Rotation: Joint needs to be parented to both shoulder and bicep-roll joints. Use
Expression to take percentage.
• Expression 1 – Describes rotation of parent joint around bicep-roll joint. (60% from
shoulder, 40% from bicep-roll)
Bicep_Helper.rx = ((.4 * bicep.rx) + (.6 * shoulder.rx))
47
RunTime Rigs - Expressions
(Complex:Biceps)
Rotation: Bicep_Helper.rx = ((.4 * bicep.rx) + (.6 * shoulder.rx))
Translation: dj_bicep.ty = (base.ty + (maxOffset * (elbow.rz /maxAngle)))
base.ty = 5, maxOffset = 1.8, maxAngle = 110
48
RunTime Rigs – Expressions & Constraints
(Complex:Biceps)
• Complex Helper Joints which need Scale &
Rotation Expressions but looking at Distance
• Requires 6 extra parent helpers (can be joints or ‘nulls’)
• Component 1 – Scale (Bulge)
– Handled by expression based on distance of ‘shock-absorber’
bicepHelper.sz = (scaleFactor*(1/(boneLength))) : boneLength = sqrt ((foreArmOffset.tx –
bicepOffset.tx)² + (foreArmOffset.ty – bicepOffset.ty)² + (foreArmOffset.tz – bicepOffset.tz)²)
• Component 2 – Rotate
– Handled by expression base on % of angle of bicep_roll
Bicep_Helper.rx = ((.4 * bicep.rx) + (.6 * shoulder.rx))
• Requires 2 pointConstraints
49
RunTime Rigs – Expressions & Constraints
(Complex:Biceps)
• Scale Expression
based on distance
50
RunTime Rigs - Expressions
• Summation
– Very difficult to describe a motion of a Complex Helper Joint
based on 2 or more axis from driving joint. Even MORE
difficult when there are multiple bones acting as drivers.
51
RunTime Rigs
• Done with Expressions
• Start with Constraints
52
RunTime Rigs - Constraints
• pointConstraint
– Good for position only
• orientConstraint
– Good for orientation only
• parentConstraint
– Good for both position and orientation
• aimConstraint
– Robust and elegant
• vertexConstraint
– Allows the use of muscles
53
RunTime Rigs - Constraints
AIMCONSTRAINT
IS
DA BOMB!
54
RunTime Rigs – Constraints
(Complex:Biceps)
• aimConstraint
pointing at
forearm
– Arc problem
– Handles rotation of bicep
roll on extension but not
flexion
55
RunTime Rigs – Constraints
(Complex:Biceps)
– aimConstraint flexion
problem
• Handles rotation of bicep roll
on extension but not flexion
56
RunTime Rigs – Constraints
(Complex:Biceps)
– aimConstraint flexion problem handled on XZ plane w/
expression for Bulge
Bicep_Helper’sParent.scale = (baseScale + (maxOffset * (elbow.rz
/maxAngle)))
i.e. - baseScale = 1, maxOffset = .3, maxAngle = 110
57
RunTime Rigs – Constraints
(Complex:Biceps)
– aimConstraint still broken on Y plane during flexion
58
RunTime Rigs – Constraints
(Complex:Pectoralis)
• Requires 2 extra parent helpers (can be
joints or ‘nulls’)
• Requires 1 aimConstraint
59
RunTime Rigs – Constraints
(Complex:ArmPit)
• Requires 2 extra parent helpers (can be
joints or ‘nulls’)
• Requires 1 aimConstraint
60
RunTime Rigs – Constraints
(Complex:Pectoralis & Armpit)
• Requires 3 extra parent helpers (can be
joints or ‘nulls’)
• Requires 2 aimConstraints
61
RunTime Rigs – Constraints
(Simple:Neck)
• Look at normal smooth bind
62
RunTime Rigs – Constraints
(Simple:Neck)
• NURBs muscle as Influence Object
63
RunTime Rigs – Constraints
(Simple:Neck)
• Requires 1 helper joint and 1 target null
for aiming
• Requires 1 aimConstraint
64
RunTime Rigs – Constraints
(Simple:Neck)
• Comparison
65
RunTime Rigs – Constraints
(Simple:Neck - Back)
• Requires 1 helper joint and 1 target null for
aiming
• Requires 1 aimConstraint
66
RunTime Rigs – Constraints
(Simple:Neck - All)
• Requires 4 helper joints w/ 4 extra target nulls
• Requires 4 aimConstraints
67
RunTime Rigs – Constraints
(SuperComplex – Scapula)
• Look at normal smooth bind
68
RunTime Rigs – Constraints
(SuperComplex – Scapula)
• What is scapula really doing?
69
RunTime Rigs – Constraints
(SuperComplex – Scapula)
• Comparison
With Rig
Normal Binding
70
RunTime Rigs – Constraints
(SuperComplex – Scapula)
• Shrugging
71
RunTime Rigs – Constraints & (Exp./SDK)
(SuperComplex – Scapula)
– Requires 2 helper joints and 2 target nulls for aiming
– Requires 2 aimConstraints & 1 pointConstraint
– Requires expression w/ “if” statement:
If (lShoulder.rotateY >0)
{
l_scapulaConst.rotateX = ( $scaleFactor* (lShoulder.rotateY/$maxAngle));
}
Else
{
l_scapulaConst.rotateX = 0;
}
Ex: $scaleFactor = 30 degrees and $maxAngle =90 degrees.
OR SetDrivenKey
72
RunTime Rigs – Constraints &
(Exp./SDK) (SuperComplex – Scapula)
• Explanation of (SDK or “If” statement in
expression)
73
RunTime Rigs – Constraints &
(Exp./SDK) (SuperComplex – Scapula)
• Pros: Full Range Looks Great
• Cons: Complex and Weighting is Hard!
74
RunTime Rigs – Expression/SDK
(Complex – ArmPitBack)
• Normal armpit crease
75
RunTime Rigs – Expression/SDK
(Complex – ArmPitBack)
• New Helper Joint added
76
RunTime Rigs – Expression/SDK
(Complex – ArmPitBack)
• Translation of Helper Joint tip
driven by expression or
SetDrivenKey from shoulder
rotation
• Requires 1 extra parent Helper
Joint w/ percentage rotation of
shoulder driven by Rotate
expression or SetDrivenKey
77
RunTime Rigs – Expression/SDK
(Complex – ArmPitBack)
• armPitParent Joint rotating 60% of
shoulder joint
78
RunTime Rigs – Exp./SDK
(Complex – ArmPitBack)
• armPitTip joint translating down its local X
& Y as a function of shoulder rotation
79
RunTime Rigs – Exp./SDK
(Complex – ArmPitBack)
• Requires 1 simple expression for parent Joint’s
rotation:
HelperParent.rotateY = (offset angle + ((% arm rotate) * lShoulder.rotateY)))
Ex - dj_armPitBackParent.rotateY = (-35 * (.6 * lShoulder.rotateY));
• Requires 1 expression for translation of tip w/ “if”
statement:
If (lShoulder.rotateY <0){
HelperTip.translateX = (base.tX + ($scaleFactorX*(lShoulder.rY/$maxAngle)));
HelperTip.translateY = (base.tY + ($scaleFactorY*(lShoulder.rY/$maxAngle)));
}Else{
HelperTip.translateX = baseX;
HelperTip.translateY = baseY;
}
Ex: $scaleFactorX = 2 units, $scaleFactorY = -4 units and $maxAngle =65 degrees.
80
RunTime Rigs – Exp./SDK
(Complex – ArmPitBack)
• Weighting is powerful and tricky. Some
vertices now have 5 different weights.
81
RunTime Rigs – Scaling Entire Rig
• What if character needs to be scaled?
(i.e.- player defined proportions)
• A simple ‘Group’ node above the rig
mechanism in Maya’s hierarchy will
scale easily
• The Trick is figuring out how much to
scale. This is dependent on what ‘space’
the helper joint deforms, and how that
space has been scaled.
82
RunTime Rigs
• Done with Constraints
• Start with Muscles
83
RunTime Rigs - Muscles
• Muscles at RunTime allows you
to constrain Helper Joints to
the muscle surface and get the
effect of the muscle
• Requires some sort of
geometryConstraint
84
RunTime Rigs – Constraints & Bind
& Blendshape (Muscles:Biceps)
• Deformer Object is low-count poly
muscle rigid (or smooth) bound to
two joints to handle twisting of
bicep
• Requires a blendShape on
deformer object to handle bulge.
– *Note: blendShape is not required on main skinning surface.
• Complex Helper Joint is
‘VertexConstrained’ to vertex on
Deformer Object
85
RunTime Rigs – Constraints & Bind
& Blendshape (Muscles:Biceps)
• VertexConstrained
86
RunTime Rigs – Constraints &
Bind & SDK (Muscles:Biceps)
• Poly Muscle (Simplest Form)
– 1 triangle Rigid (or Smooth) Bound
• Handles rotation of bicep
– SDK (or simple expression [1 driving bone/axis, 1 driven
translating axis]) driving single vertex for bulge
– Complex Helper joint ‘VertexConstrained’ to single vert
– Finally skinning layer
87
RunTime Rigs – Constraints &
Bind & SDK (Muscles:Biceps)
• 1 Poly (Simplest Form)
– VertexConstraint
88
RunTime Rigs – Constraints &
Bind (Muscles:Pectoralis)
• Deformer Object is low-count poly
muscle rigid (or smooth) bound to three
joints to handle complex Pectorals
motion
• Requires MuscleDriver parent joints to
be offset from main skeletal joints
• MuscleDriver child joints are
‘pointConstrained’ to parent joints to
avoid rotation of parent joints
• Complex Helper Joint is
‘VertexConstrained’ to vert on Deformer
Object
89
RunTime Rigs – Constraints &
Bind (Muscles:Pectoralis)
• PointConstraints & VertexConstraints
90
RunTime Rigs to Non-RunTime
91
Non-RunTime Rigs
• Helper Joints Definition:
– They can be controlled by SetDrivenKeys, Expressions,
Simple Rigs (constraint system), or Super Advanced
Rigs (really any simulation you can think of)
that are baked-out.
92
Non-RunTime Rigs - Baked Out
• BakeOut
examples
– Bicep
– Complex Helper Joints
constrained to nurbsMuscle
influence object, baked out, 1
key per frame, scene cleaned.
93
Non-RunTime Rigs - Baked Out
• BakeOut
examples
– Pectorals
– Complex Helper Joints
constrained to nurbsMuscle
influence object, baked out, 1
key per frame, scene cleaned.
94
Non-RunTime Rigs - Baked Out
• BakeOut
examples
– Pectorals
– Lots!
95
Non-RunTime Rigs - SplineIK
• Great Skin-sliding effect
96
Non-RunTime Rigs - Jiggle
• Basic Jiggle is fairly easy to create
in Maya, either with a:
– Simple Expression
– frameCache node
• Could be turned into RunTime if
your code supported it
97
Non-RunTime Rigs - Jiggle
• Create a 1 (or
more) frame lag in
global space for a
joint
– BaseJoint = ‘Goal’
• Blue joint parented to spine
– EndJoint = ‘Weighted’
• Yellow joint lagging behind Blue
joint by 1 frame
98
Non-RunTime Rigs - Jiggle
• Expression:
int $frame = frame; // get the current frame
float $tx_0 = BellyJiggleBase.translateX;
float $tx_1;
// reset the attributes on start frame
if ($frame == 1){ // start frame
$tx_1 = $tx_0;
BellyJiggleEnd.translateX = $tx_0;
}
else{
BellyJiggleEnd.translateX = $tx_1;
$tx_1 = $tx_0;
}
Repeat for Y & Z axis
99
Non-RunTime Rigs - Jiggle
• frameCache Node
100
Non-RunTime Rigs - Jiggle
frameCache
Expression
101
Non-RunTime Rigs - Jiggle
• Incorporation:
– Could theoretically add a jiggle joint as a child of the helper
joints
– Then just weight individual vertices to these jiggle joints to
cause ‘jiggling’
– Or just make separate jiggle joints and their ‘base’ joint
wherever needed
– Where are they needed?
102
Non-RunTime Rigs to
Automation
103
Automating Production
• Scripts
• Workflow
• Offline Muscle Systems
• Procedurally generated
Helper Joints
104
Automating Production
• Scripts: Tools you’ll need
– Script to auto-build entire Helper Joint System on your character
• Selectable body parts
– Scripts to automate editing of Rigs, SetDrivenKeys, & Expressions for
the artists
– Scripts to do lots of mirroring:
• Helper Joint Rigs
• Weighting
• Set Driven Keys
– Export and Import Set Driven Keys
• Hint: look at the animCurves and add custom attrs for connections to them
upon export
• Demo
105
Automating Production
• Workflow
–
–
–
–
Duplicate your primary skeleton and skin
Apply your muscle system to this duplicate character
Constrain duplicate skeleton to primary skeleton
Hide duplicated skeleton, muscles, & rigs. Template
duplicated skin.
106
Automating Production
• Workflow
107
The Ultimate Reference
• Muscle Systems
108
The Ultimate Reference
• Muscle Systems
109
The Ultimate Reference
• Muscle Systems
– Auto imports muscle system
• Scaling to your skeleton
– AutoFits muscles to duplicated skin
– AutoSkins muscles to duped skin
110
The Ultimate Reference
• Once you have the “Ultimate
Reference”, you now have the
data to let a programmer put you
out of a job. All (s)he needs to
figure out is:
–
–
–
–
How many joints?
Where to place them?
How do they need to transform?
What does their weighting need to be?
111
Mohr/Gleicher Method
• Siggraph 2003 : “Building Efficient,
Accurate Character Skins from Example”
– White Paper on auto-computing placement, animation, and weighting of
Helper Joints.
112
Conclusion
• We’ve covered just a few examples of how to use SetDrivenKeys,
Constraints, and Expressions.
• My solutions are just first shot attempts. With refinement, you
can come up with much better solutions.
• With these simple tools you can use them in many combinations
to achieve nearly any effect you need.
• Of course everything is RunTime code dependent. As Character
T.D.s we need to fight for:
– SetDrivenKeys
– Constraints
• Point, orient, !!AIM!!, parent, geometry
– Expressions
– Spline IK
– frameCache (jiggle)
• Helper Joints are just a temporary solution until we can get
more advanced deformers at RunTime to simulate Muscles
– Per-vertexDeformer (influence objects)
– WrapDeformer (lattices)
– fusiForm deformer
113
. . . Talk more about this next year?
Thanks
• … for your time.
• Please fill out surveys so I
can make a better one next
year.
• Peace out
114
Contacts, Reference, Credits
• Email:
– jason@jason-parks.com
– jason_parks@playstation.sony.com
• Webpage (Helper Joints-GDC 2005):
– http://www.jason-parks.com/HelperJoints
115