Content Analysis and Restoration
of Marine Video
Student
:- Ken Sooknanan
Supervisor :- Professor Naomi Harte
in collaboration with Prof. Jim Wilson
(Dept. of Zoology), the Marine Institute
Ireland, and Prof. Anil Kokaram (Google)
Goals
1) Content Analysis:-
(a) Identify Nephrop burrow
complexes
(a) Complex
(b) Identify when major changes
along the seabed occur.
2) Restoration:(a) Improve Visibility by correcting
illumination degradations due to
light source.
(b) Nephrop Video
Light Footprint
(c) Rocky
(d) Muddy Video
Improving Visibility
(1) Degradations modelled as mixture
r ≤ rf
I (x)
G(x)
3.5r 2 ( x )
r > rf
I ( x)e
=
c
s s
r ( x)  ( x  c) S ( x  c);S = 1 2 ; c = cx
s2 s3
y
2
I (x)
T
(2) S, c estimated with ratio of pt. correspondences
 G2 
G1 I 2e
2
2



ln


3
.
5
r

r

2
1 
G 
3.5r12
G 2 I1e
 1
optimized alternately in Bayesian Framework
I ( x)e
G1 r1
3.5r22

3.5
r 2 ( x)
G2
G(x)
r2
c
Frame 1
c
Frame 2
cn+1=arg max p(c|G1,G2,Sn);
Sn+1=arg max p(S|G1,G2,cn+1);
(3) Footprint estimated as region where largest
percentage increase in degradation occurred
(c) A(rx) =
G2(r2)
G1(r1)
(d) A(rx) > μ. Split into
regions, footprint (red)
Results
IMAGES :-
(a) Original with
footprint in blue
(b) Our Result
(c) Seon Joo
Original Result
(d) Seon Joo Result
with our radii est.
VIDEO :-
[1] Seon Joo, et al.., “Robust radiometric calibration and vignetting correction," Pattern Analysis and Machine Intelligence, 2008
Identifying Nephrop Clusters
• Problem broken up into two parts:- identifying all burrows
- clustering
• Currently working on identification part
(a) Sample Video Frame
• Exploring the use of a Mosaic, as it:-
-
Eliminates tracking
Fixes geometric distortion.
Computations reduced to a single image
Provides a wide view of the seabed.
Easy to spot clusters.
(b) Generated Mosaic
(using 1st 50 Frames)
Algorithm overview
(1) Highlight all dark regions
- DOG
2) Segment and label DOG – Intensity based (Bayesian framework - ICM).
3) Identify and Split multiple burrows regions – shape modelling with GMMs
(a) Original
(b) Segmentation (c) Est. GMM
(d) Split Region
 f 
 0.5 
 
4) Classify based on shape, colour and shading features, f, p ( f )  e
- Cascade Classifier,
SHAPE
COLOUR
SHADDING
p(f) > (T =0.5)



2
Initial Results
(a) Original Mosaic
Results obtained from 3 Video
Sequences (10 min., 15000
Frames) Compared with
Ground Truth from:(1) Video (existing method)
(2) Mosaic
(b) Identified Burrows
Exp. Video Results
Mosaic Results
Ground
Truth
% Correct
Detected
Ground
Truth
Recall
%
Precision
%
1
50
90
50
90.0
84.0
2
63
85.7
60
82.5
88.3
3
84
88.1
83
88.1
85.5
Accomplishments
Year 1:- Attended and passed 3 courses to gain the necessary 15-credits
Year 2:- Wrote year-1 transfer report and passed viva to enter PhD roster.
Year 3:(1) Gave a presentation in Google, San Francisco on 23rd Jan. 2012
(2) Published a paper [1] in SPIE conference, San Francisco Jan. 2012
(3) Submitted a paper [2] on Burrow Detection algorithm in ICIP 2012.
(4) Submitted an abstract [3] on Mosaicing Algorithm in Oceans conference 2012.
(5) Established good collaboration with the Marine Institute Galway
- keep in contact with Jennifer Doyle approximately once every month.
[1] Sooknanan, et al.., “Improving Underwater Visibility using Vignetting correction," in proceedings of SPIE, 2012
[2] “A Bayesian Framework for Detection of Nephrop Burrows for Seabed Video Analysis,” ICIP 2012
[3] “A Bayesian Framework for Mosaic Creation of the Seabed from Underwater Video For Nephrop Burrow Detection”, Oceans, 2012
Future Work
(1) Present work in Study Group on Nephrops Survey (SGNEPS) Meeting in
Italy in 8th March 2012.
(2) Do more testing with Burrow detection algorithm
(3) Move onto the Burrow clustering part of the problem
(4) Move onto detecting when major changes in seabed type occur
(5) Write papers on (3) and (4)
(6) Write up Thesis.