Content Analysis and Restoration of Marine Video Student :- Ken Sooknanan Supervisor :- Professor Naomi Harte in collaboration with Prof. Jim Wilson (Dept. of Zoology), the Marine Institute Ireland, and Prof. Anil Kokaram (Google) Goals 1) Content Analysis:- (a) Identify Nephrop burrow complexes (a) Complex (b) Identify when major changes along the seabed occur. 2) Restoration:(a) Improve Visibility by correcting illumination degradations due to light source. (b) Nephrop Video Light Footprint (c) Rocky (d) Muddy Video Improving Visibility (1) Degradations modelled as mixture r ≤ rf I (x) G(x) 3.5r 2 ( x ) r > rf I ( x)e = c s s r ( x) ( x c) S ( x c);S = 1 2 ; c = cx s2 s3 y 2 I (x) T (2) S, c estimated with ratio of pt. correspondences G2 G1 I 2e 2 2 ln 3 . 5 r r 2 1 G 3.5r12 G 2 I1e 1 optimized alternately in Bayesian Framework I ( x)e G1 r1 3.5r22 3.5 r 2 ( x) G2 G(x) r2 c Frame 1 c Frame 2 cn+1=arg max p(c|G1,G2,Sn); Sn+1=arg max p(S|G1,G2,cn+1); (3) Footprint estimated as region where largest percentage increase in degradation occurred (c) A(rx) = G2(r2) G1(r1) (d) A(rx) > μ. Split into regions, footprint (red) Results IMAGES :- (a) Original with footprint in blue (b) Our Result (c) Seon Joo Original Result (d) Seon Joo Result with our radii est. VIDEO :-  Seon Joo, et al.., “Robust radiometric calibration and vignetting correction," Pattern Analysis and Machine Intelligence, 2008 Identifying Nephrop Clusters • Problem broken up into two parts:- identifying all burrows - clustering • Currently working on identification part (a) Sample Video Frame • Exploring the use of a Mosaic, as it:- - Eliminates tracking Fixes geometric distortion. Computations reduced to a single image Provides a wide view of the seabed. Easy to spot clusters. (b) Generated Mosaic (using 1st 50 Frames) Algorithm overview (1) Highlight all dark regions - DOG 2) Segment and label DOG – Intensity based (Bayesian framework - ICM). 3) Identify and Split multiple burrows regions – shape modelling with GMMs (a) Original (b) Segmentation (c) Est. GMM (d) Split Region f 0.5 4) Classify based on shape, colour and shading features, f, p ( f ) e - Cascade Classifier, SHAPE COLOUR SHADDING p(f) > (T =0.5) 2 Initial Results (a) Original Mosaic Results obtained from 3 Video Sequences (10 min., 15000 Frames) Compared with Ground Truth from:(1) Video (existing method) (2) Mosaic (b) Identified Burrows Exp. Video Results Mosaic Results Ground Truth % Correct Detected Ground Truth Recall % Precision % 1 50 90 50 90.0 84.0 2 63 85.7 60 82.5 88.3 3 84 88.1 83 88.1 85.5 Accomplishments Year 1:- Attended and passed 3 courses to gain the necessary 15-credits Year 2:- Wrote year-1 transfer report and passed viva to enter PhD roster. Year 3:(1) Gave a presentation in Google, San Francisco on 23rd Jan. 2012 (2) Published a paper  in SPIE conference, San Francisco Jan. 2012 (3) Submitted a paper  on Burrow Detection algorithm in ICIP 2012. (4) Submitted an abstract  on Mosaicing Algorithm in Oceans conference 2012. (5) Established good collaboration with the Marine Institute Galway - keep in contact with Jennifer Doyle approximately once every month.  Sooknanan, et al.., “Improving Underwater Visibility using Vignetting correction," in proceedings of SPIE, 2012  “A Bayesian Framework for Detection of Nephrop Burrows for Seabed Video Analysis,” ICIP 2012  “A Bayesian Framework for Mosaic Creation of the Seabed from Underwater Video For Nephrop Burrow Detection”, Oceans, 2012 Future Work (1) Present work in Study Group on Nephrops Survey (SGNEPS) Meeting in Italy in 8th March 2012. (2) Do more testing with Burrow detection algorithm (3) Move onto the Burrow clustering part of the problem (4) Move onto detecting when major changes in seabed type occur (5) Write papers on (3) and (4) (6) Write up Thesis.