Spacetime Constraints
Revisited
J. Thomas Ngo
Joe marks
Graduate Biophysics Program
Harvard University
Cambridge Research Lab
Digital Equipment Corporation
Spacetime Constraints
• Indirectly controlling physically
realistic motion of articulated
figures
• Animator Defines:
– Physical structure
– Actuators
– Criteria for evaluating motion
• Computer computes optimal
trajectory according to specified
criteria
Globally Optimal Solutions
• Current SC techniques are local in
nature
• Finding Globally Optimal Solutions
to SC Problems is tough
– Multimodality
• Exponential number of possible
trajectories
– Many may be locally optimal or near
optimal
– Search-space discontinuities
• Small change in actuators causes large
change in trajectory
A Way to Find Globally Optimal Solutions:
• Global search algorithm
– Generates multiple nearoptimal trajectories
• Encode trajectories as sets of
stimulus-response behavior
rules
• Genetic algorithm for choosing
behavior parameters
The Algorithm
• Dynamics module creates
physically correct
environment.
• Behavior module generates
creature behaviors
• Search module finds values
for stimulus-response
parameters.
Dynamics Module
• Uses forward dynamics
– Find motions from forces
• Articulated figure treated as
autonomously deforming
object with no DOF
– Deformations produced
kinematically by stimulusresponse control algorithm.
– Save CPU cycles.
Dynamics Module
• Friction is static when one
joint touches floor
• Slippage proportional to
contact force when two joints
touch
Friction
?
Behavior Module
• Uses parameterized
algorithm based on stimulus
and response
• Stimulus-Response Control
Algorithm
– Reflexes triggered by
conditions sensed in
environment
• Conditions – stimulus functions
• Reflexes - responses
Behavior Module - Responses
• Response – direction for
changing creature shape
smoothly.
– Time Constant
– Set of Target values for
creature’s internal angles
Behavior Module - Responses
• Throw values into critically
damped equation of motion for
each
 2 angle
 2  (   0 )  0
i
i
i
i

• i time constant
• i target value of angle
•
actual value of angle
0
• Creature approaches target
shape smoothly.
Behavior Module - Stimulus
Functions
• Sense variable
– Real-valued function of environment
• Proprioceptive senses
– Um, joint angles
• Tactile senses
– Force exerted by endpoint on floor
• Kinestheitic sense
– Vertical velocity of center of mass
• Position sense
– Vertical position of center of mass
Behavior Module - Stimulus
Functions
• Stimulus functions
– Scalar function defined over sense
V
space



0 2
W 1  max  j (v j  v j ) 


j 1



Sense Variables

{v1 , v2 ,..., vV }
Determined by search
v 0j
Determined by search
j
 j
W   log  min

j 1
 j
V
Weight




Behavior Module - Stimulus
Functions
• Constants normalized so sense
variables fall between 0 and 1
• Sensitive Region
– Locus of points in sense space
where stimulus-function is
positive
2 2
2
• Forms a hyper-rectangle
,
,...,


– Dimensions
 1 2
V 
v , v ,..., v 
0
1
– Centered at
0
2
0
V
Behavior Module - Stimulus Response Functions
• Set of SR parameters consists of array of
SR pairs
• To generate behavior:
Initialize creature state from SC problem description
Activate response 0
for t = 1 to T
Determine deformation for time t from active response
Simulate resulting dynamics for time t
Measure sense variables from the environment
Identify highest-valued stimulus function
Activate corresponding response if stimulus positive
end for
• Active response changed only if highestvalued stimulus function is positive
– Causes response to be active for several
time step
• Coherent motion
Search Module – Genetic Algorithm
• Parallel genetic algorithm
– Written in C* on Thinking
Machines CM-2 with 4096
processors
• Each processor evaluates
one genome per generation
Search Module
• Q: How to find values for
stimulus and response
parameter?
• A: Trial and Error.
(Genetic Algorithm)
Search Module – Genetic Algorithm
• Pseudocode:
do parallel
Randomize genome
end do
for generation = 1 to number_of_generations
do parallel
Evaluate genome
Select mate from another processor
Cross genome with mate
Mutate genome
end do
end for
Search Module – Genetic Algorithm
• Parameters of one SR pair in genome
Search Module – Genetic Algorithm
• Randomization
– All parameters initially chosen at
random
• Uses probability distribution
– Hill climbing to enrich initial gene pool
• Evaluate initial gene pool
• Mutate and re-evaluate solution 4 times
• On each processor choose best out of
five
– Makes population skewed in favor of
multi-step behaviors
Search Module – Genetic Algorithm
• Mate Selection
–
–
–
–
Processors laid on in imaginary 64x64 grid
Each processor does 10 step random walk
If best solution is itself, no mating
Else mates with best encountered genome
• Good genes diffuse through population
• Spreading is not instantaneous
so inferior solutions get some
processing time
• Population converged when one
population dominates
Search Module – Genetic Algorithm
• Crossover
– Typical linear layout for crossover
not meaningful
– Crossover must be tailored to
problem to get good performance
Search Module – Genetic Algorithm
• Mutation
– Tailored specifically for the SR
representation.
• One SR Pair creeps
• One SR Pair randomized form scratch
– But at least one corner of sensitive region of
new stimulus function must coincide with
original sense-space trajectory.
» Else new stimulus functions dominate
trajectory or don’t do anything
Results – Five-Rod Fred
• Five-Rod Fred
– 5 consecutively linked rods
• Middle rods have same mass
• Terminal rods five times heavier
• Evaluation Function – Move
COM as far right as possible
• Expected inch-worm
behavior
• But…
Results – Five-Rod Fred
After 64 generations…
Curling leap
Results – Five-Rod Fred
After 100 generations…
Curling leap w/ a roll
Results – Five-Rod Fred
• Final behavior generated by
5 of 10 SR pairs
– Two pairs produce initial
curling motion
– Two pairs produce leaping
motion
– One pair produces final curled
shape
Results – Mr. Star-Man
• Mr. Star-Man
– Five rods in star shape
– All rods have equal length,
mass
• Evaluation Function – Move
COM as far right as possible
Results – Mr. Star-Man
After 20 generations…
Sideways cantering
Results – Mr. Star-Man
After 37 generations…
Ghetto tripping cartwheel
Results – Mr. Star-Man
After 94 generations…
Sideways shuffling
Results – Beryl Biped
• Beryl Biped
– Headless 2D humanoid
•
•
•
•
Rigid torso
Jointed legs
Point feet
Rod masses of human proportion
• Evaluation Function – Move
point between feet as far right as
possible
– If used COM, just fell down
Results – Beryl Biped
After 100 generations…
Skipping, back leg never gets
In front of forward leg
Results – Beryl Biped
Another trial…
Skipping walk
Results – Beryl Biped
Another trial…
Weird walk
Results
• GA will fail to find good
optima of evaluation
function is no good
• Small changes can
restore expected behavior
• How to change evaluation
function to reward grace?
???