Gaussian Process Dynamic Models for
Human Motion
Wang, Fleet, Hertzmann
Presented by Archit Rathore
Arpit Agarwal
Masare Akshay Sunil

B
A
x
1 x
2 x
3 x
4
X x t
= f(x t-1
,A) + n x,t y t
= g(x t
,B) + n y,t y
1 y
2 y
3 y
4
Y p(X,Y,α,β,W) = p(Y|X,β,W) p(X|α) p(α) p(β) p(W)

Given the learned model Τ = {Y,X,α,β,W}, the distribution over a new sequence Y {*} and its associated latent trajectory X {*} is given by: p (Y {*} ,X {*} |T) = p (Y {*} |X {*} ,T) p (X {*} |T) p (Y {*} ,X {*} |T) = |W| M exp(-0.5 tr(K -1
Y{*}
√(2π) MD |K
Y{*}
| D
Z
Y
W 2 Z
Y
T )) p (X {*} |T) = p (x
1
(*) ) exp(-0.5 tr(K -1
X{*}
Z
X
Z
Y
Z
X
T ))
√(2π) (M-1)d |K
X{*}
| d

The skeleton used is a simplified version of the default skeleton in the CMU mocap database. The numbers in parentheses indicate the number of DOFs for the joint directly above the labeled body node in the kinematic tree.
Each pose is defined by 44 Euler angles for joints, three global (torso) pose angles, and three global
(torso) translational velocities.
The data is mean subtracted.

MAP Estimation: Minimizing the -ve log-posterior of the unknowns -ln p (X,α,β,W|Y) given by:
L = L
Y
+ L
X
+ Σ ln β j
+ 1/(2κ 2 ) tr(W 2 ) + Σ ln α j
L
Y
= D/2 ln|K
Y
|+ 0.5
tr(K
Y
-1 YW 2 Y T ) - N ln|W|
L
X
= d/2 ln|K
X
|+ 0.5tr(K -1
X
X
2:N
X T
2:N
)+ 0.5x
1
T x
1
Balanced GPDM: The L
X
term penalizes unsmooth trajectory, therefore increasing weight on L
X during optimization encourages smoothness. The weight used is D/d - the weight between dimensions of data and latent spaces.
Manually specified hyperparameters: Smooth trajectories can also be ensured by selecting α by hand by examining samples from p(X|α).
Two-stage Map estimation: First estimate hyparameters Θ={X,α,β,W} from unknown latent trajectories X, then estimate X while holding Θ fixed.

PCA Initializer Latent Coordinates from GPLVM
Dynamic Predictive Distribution by HMC
Mean Prediction Sample
Latent Coordinates from GPDM
High Confidence Latent Space Random Trajectories Longer Random Trajectories

GOLF SWING MODEL
Dynamic Predictive Distribution by HMC
Mean Prediction Sample
Latent Coordinates from GPLVM Latent Coordinates from GPDM High Confidence Latent Space
FOUR WALKER MODEL
Random Trajectories
Dynamic Predictive Distribution by HMC
Mean Prediction Sample
Latent Coordinates from GPLVM Latent Coordinates from GPDM High Confidence Latent Space Random Trajectories
Samples stay close to training data in Golf Swing Model and form Clumpy Point Clusters in Four Walker Model

Walk Synthesized by taking the mean of the Predictive Distribution
MAP Two-Stage MAP
Six Walks Synthesized by Sampling from the Predictive Distribution
B-GPDM

The technique presented in the paper is used for human tracking and synthesizing high quality poses from mocap data. The smoothed trajectories obtained from the GPDM approach can transfer the cost of high quality camera equipment to the relatively cheaper computational cost for estimating the parameters.
The prevalence of mocap data to produce realistic animations leads to the need to estimate the non linear manifold on which the high dimensional human motion data lies on.
The approach paves way for motion synthesis subject to sparse user constraints. This can also be used to estimate missing frames from data or reduce impact of noise from sensors when translating the video data to animation.