EE4-62 MLCV Lecture 15-16 Pose Estimation – Gaussian Process Tae-Kyun Kim 1 EE4-62 MLCV MS KINECT http://www.youtube.com/watch?v=p2qlHoxPioM The KINECT body pose estimation is achieved by randomised regression forest techniques. 2 EE4-62 MLCV We learn pose estimation as a regression problem, and Gaussian process as a cutting edge regression method. We see it through the case study (slide credits to): Semi-supervised Multi-valued Regression, Navaratnam, Fitzgibbon, Cipolla, ICCV07, where practical challenges addressed are 1)multi-valued regression and 2)sparsity of data. 3 EE4-62 MLCV A mapping function is learnt from the input image I to the pose vector θ , which is taken as a continuous variable. 4 Given an image (top left) , e.g. a silhouette (top right) is obtained by background subtraction techniques (http://en.wikipedia.org/wiki/Background_subtraction). The estimated 3d pose is shown at two camera angels (bottom left and right). 5 EE4-62 MLCV We attempt to map 2D image space to 3D pose space. There is inherent ambiguity in pose estimation (as an example in the above). 6 EE4-62 MLCV Eye tracking can be tackled as a regression problem, where the input is an image I and the output is a eye location. 7 EE4-62 MLCV Typical image processing steps: Given an image, a silhouette is segmented. A shape descriptor is applied to the silhouette to yield a finite dimensional vector. (Belongie and Malik, Matching with Shape Contexts, 2000) 8 EE4-62 MLCV The output is a vector of m joint angles. 9 EE4-62 MLCV 10 EE4-62 MLCV 11 EE4-62 MLCV 12 EE4-62 MLCV 13 EE4-62 MLCV 14 EE4-62 MLCV EE4-62 MLCV Gaussian Process 15 EE462 MLCV Review of Gaussian density estimation (lecture 1,2) i.i.d. (independent identical distributed) We want to find the Gaussian parameters from the given data. The problem is to find the parameters by maximising the posterior probability 2 2 𝑝 𝐱 𝜇, 𝜎 𝑝 𝜇, 𝜎 𝑝 𝜇, 𝜎 2 𝐱 = 𝑝(𝐱) 16 EE462 MLCV Review of Gaussian density estimation (lecture 1,2) 2 2 𝑝 𝐱 𝜇, 𝜎 𝑝 𝜇, 𝜎 𝑝 𝜇, 𝜎 2 𝐱 = 𝑝(𝐱) We do not have priors on the parameters and data, thus we maximise the (log) likelihood function instead. Maximum Likelihood (ML) vs Maximum A Posterior (MAP) solutions: P(X|Y)P(Y) P(X|Y) (e.g. Gaussian Process) 17 EE462 MLCV Review of polynomial curve fitting (lecture 1,2) We want to fit a polynomial curve to given data pairs (x,t). = 𝐰𝑇 𝐱 where 𝑤0 𝐰= 𝑤1 ⋮ 1 𝑥 𝐱= 𝑥 2 ⋮ The objective ftn to minimise is EE4-62 MLCV Gaussian Processes 19 EE4-62 MLCV 20 EE4-62 MLCV 𝐲~𝑁(𝐲|𝟎, 𝐊) 21 EE4-62 MLCV 𝐲(𝐱) 𝐱 22 EE4-62 MLCV Gaussian Processes for Regression 23 EE4-62 MLCV 24 EE4-62 MLCV 25 EE4-62 MLCV 26 EE4-62 MLCV 𝑡 =𝑦 𝑥 +𝜖 27 EE4-62 MLCV 𝑁×𝑁 𝑁×1 28 EE4-62 MLCV 29 30 EE4-62 MLCV 31 Gaussian Process Matlab Toolbox http://www.lce.hut.fi/research/mm/gpstuff/i nstall.shtml (try demo_regression_robust.m, demo_regression1.m) 32 EE4-62 MLCV Learning Hyperparameters 33 34 EE4-62 MLCV Automatic Relevance Determination 35 36 37 Back to the pose estimation problem 38 39 EE4-62 MLCV • It’ll never work… – is multivalued – and live in high dimensions 40 EE4-62 MLCV 41 EE4-62 MLCV 42 EE4-62 MLCV 43 EE4-62 MLCV 44 EE4-62 MLCV 45 EE4-62 MLCV 46 EE4-62 MLCV GMM : Gaussian Mixture Model GPLVM : Gaussian Process Latent Variable Model 47 EE4-62 MLCV 𝑝 𝜃𝑧 = 𝑝(𝜃, 𝑧) 𝑝(𝑧) 𝑝 𝑧 = ∫ 𝑝 𝜃, 𝑧 𝑑𝜃 48 EE4-62 MLCV 49 EE4-62 MLCV 50 EE4-62 MLCV 51 EE4-62 MLCV 52 EE4-62 MLCV 53 EE4-62 MLCV 54 EE4-62 MLCV 55 EE4-62 MLCV 56 EE4-62 MLCV 57 EE4-62 MLCV 58 EE4-62 MLCV 59 EE4-62 MLCV 60 EE4-62 MLCV t: a continuous latent variable 𝑝 𝑧, 𝜃 = ∫ 𝑝 𝑧, 𝜃 𝑡 𝑝 𝑡 𝑑𝑡 61 EE4-62 MLCV 62 EE4-62 MLCV 63 EE4-62 MLCV 64 EE4-62 MLCV 65 EE4-62 MLCV 66 EE4-62 MLCV 67 EE4-62 MLCV 68 EE4-62 MLCV 69 EE4-62 MLCV 70 EE4-62 MLCV Real-Time Human Pose Recognition in Parts from Single Depth Images (J. Shotton et al, 2011) • Key features – Depth image as input – Real-time by Random Forest, and Part-based 71 EE4-62 MLCV Progressive Search Space Reduction for Human Pose Estimation (Ferrari et al, 2008) 72 EE4-62 MLCV