Camera Tracking
and Scoring
Algorithm for
Synchronized Divers
Ramon Ortega
Spring 2014
CSCI 512
Computer Vision
Dr. William Hoff
Goal of Project

Create Prototype Program
for US Olympics
Committee

Use a computer to judge
how well 2 divers stayed
in-sync during a dive

Methods


1. Diver Segmentation
2. Diver Comparison and
Scoring
Segmentation

How: Separate each diver from the background using Snakes

What: Active Contours/Energy Splines

Challenges:

Interlaced Images
Attraction to bathing Suits and
Non-skin edges
More on Snakes



What is a snake?

A way to segment an object of interest from a background

A moving contour whose points are attracted by edge (strong
image gradients)
Snakes stop moving when energy is minimize

Bending/stretching curve = more energy

Good features = less energy
V(s) is the representation of a curve
v(s)  x(s), y(s)

The energy function of the curve has 3 terms: internal
spline energy, image external energy, and user constraint
energy
    int v( s)    img v( s)    con v(s) ds
Snake Parameters

Image energy/force options

Sigma used in Gaussian filtering aka, how image derivatives
(gradients) are calculated

Weight, w, assigned to attraction to edges (strong gradients)
corners)
 img   w  I ( x, y )

2
Internal Snake Energy Spline options

a is first order membrane energy, min energy when curve is
small

b is second order thin plate energy, min energy when curve is
smooth

 int v( s)   a ( s) v s ( s)  b ( s) v ss ( s)
2
2
2
Visualizing Image Force
Synchronicity Scoring


Real Life Scoring Criteria

the starting position, the approach and the take-off

the coordinated timing of the movements during the flight

the similarity of the angles of the entries

the comparative distance from the springboard or platform of the entry

the coordinated timing of the entries
Computer Vision Scoring Criteria (0-10)


Average of Major Axis Angle, and eccentricity difference from regionprops() function in
Matlab
𝑆𝑐𝑜𝑟𝑒 =
𝑛𝑓𝑟𝑎𝑚𝑒𝑠 1−∆𝑎𝑛𝑔𝑙𝑒/𝑚𝑎𝑥𝐴𝑛𝑔𝑙𝑒 1−∆𝑒𝑐𝑐𝑒𝑛𝑡𝑟𝑖𝑐𝑖𝑡𝑦/𝑚𝑎𝑥𝐸𝑐𝑐𝑒𝑛𝑡𝑟𝑖𝑐𝑖𝑡𝑦
+
𝑛=1
2
2
𝑛𝐹𝑟𝑎𝑚𝑒𝑠
∗10
2012 Olympics, London: Men’s Synchronized
3M Spring Board: America
Testing 2012 America
angle = 89.3310
angle2 =88.7878
currentAngDiff = 0.5432
eccentricity = 0.9729
eccentricity2 = 0.9423
currentEccenDiff =0.0306
Frame 1
Start score = 9.8167
Testing 2012 America
angle = 37.49
angle2 =27.43
currentAngDiff = 0
eccentricity = .9140
eccentricity2 = .9363
currentEccenDiff =0.0223
frame 175
Final score = 9.6909
Libraries\Videos
2012 Olympics, London: Men’s Synchronized
3M Spring Board: Russia
Testing 2012 Russia
Testing 2012 Russia
angle =86.9250
angle2 = 81.8034
currentAngDiff =5.1216
eccentricity 0.7819
eccentricity2 =0.9093
currentEccenDiff = 0.1274
Frame = 1, 2
Start score = 9.1586
Testing 2012 Russian
angle =38.5682
angle2 = 48.5188
currentAngDiff =9.9506
eccentricity 0.7562
eccentricity2 = 0.8855
currentEccenDiff = 0.1293
Frame = 103
Final score = 8.21
2012 Olympics, London: Men’s Synchronized
3M Spring Board: Mexico
Testing 2012 Mexico
angle =83.5984
angle2 = 87.2265
currentAngDiff =3.6281
eccentricity =0.9671
eccentricity2 =0.9871
currentEccenDiff = 0.0201
Start score = 9.6981
Testing 2012 Mexico
angle =25.8997
angle2 = 66.2583
currentAngDiff =40.3587
eccentricity =0.7037
eccentricity2 =0.8712
currentEccenDiff = 0.1675
Frame = 28
Final score = 9.1
Results
2012 Olympics, London: Men’s
Synchronized 3M Spring Board:
Computer
Real Life
Comments
Synch Score Judge Score
(0-10)
(0-100+)
America
9.76
86.7
Heavy Snake
Error
Russia
8.21
100
Significant
Snake Error
Mexico
9.1
65.1
Infinite
Snake Error;
Divers Not
Tracked at
All by Snakes
Conclusions

Software Competencies



Snakes track skin well
Software Flaws

Guess for where next frame’s
snake will appear is far off

Snakes get stuck on edges stronger
than a diver

Scoring is very basic
Improvements

Limit Area of a snake contour

Limit snake centroid to center
image
Questions
References
[1]
DD Morris, J. Rehg, Singularity analysis for articulated object tracking, in: Proc CVPR’98, Santa Barbara, CA,
1998,pp. 286-296.
[2]
TB Moeslund , Adrian Hilton , Volker Krüger, A survey of advances in vision-based human motion capture and
analysis, Computer Vision and Image Understanding, 2006, 104(2), pp.90-126
[3]
Haojie Li; Shouxun Lin; Yongdong Zhang; Kun Tao, "Automatic Video-based Analysis of Athlete Action," Image
Analysis and Processing, 2007. ICIAP 2007. 14th International Conference on , vol., no., pp.205,210, 10-14 Sept.
2007 doi:10.1109/ICIAP.2007.4362780 keywords: {image motion analysis;sport;video,
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4362780&isnumber=4362742
[4]
B. Rosenhahn, T. Brox, D. Cremers, and H.-P. Seidel, “A Comparison of Shape Matching Methods for Contour
Based Pose Estimation,” in Combinatorial Image Analysis, vol. 4040, R. Reulke, U. Eckardt, B. Flach, U. Knauer,
and K. Polthier, Eds. Springer Berlin Heidelberg, 2006, pp. 263–276.
[5]
Peng Guan; Weiss, A.; Balan, A.O.; Black, M.J., "Estimating human shape and pose from a single image,"
Computer Vision, 2009 IEEE 12th International Conference on, vol., no., pp.1381,1388, Sept. 29 2009-Oct. 2 2009
doi:10.1109/ICCV.2009.5459300 keywords: {Humans;Shape},
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5459300&isnumber=5459144
[6]
Ovsjanikov, M.; Bronstein, A.M.; Bronstein, M.M.; Guibas, L.J., "Shape Google: a computer vision approach to
isometry invariant shape retrieval," Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International
Conference on , vol., no., pp.320,327, Sept. 27 2009-Oct. 4 2009 doi: 10.1109/ICCVW.2009.5457682 keywords:
computer graphics} URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5457682&isnumber=5457418
[7]
Szeliski, Richard , Computer Vision: Algorithms and Applications. Springer 1st ed ,Vol. 2010
[8]
Thomas B. Moeslund, Adrian Hilton, Volker Krüger, A survey of advances in vision based human motion capture
and analysis, Computer Vision and Image Understanding, Volume 104, Issues 2–3, November–December 2006,
Pages 90-126, ISSN 1077-3142, http://dx.doi.org/10.1016/j.cviu.2006.08.002.
URL: http://www.sciencedirect.com/science/article/pii/S1077314206001263