Albert Parker
Department of Mathematical Sciences
Montana State University
March 30, 2006

Joint work with:
• Austin Roorda, School of Optometry, University of California, Berkeley
• Curt Vogel and Qiang Yang, Department of Mathematical
Sciences, Montana State University
• David Arathorn, Center for Computational Biology, Montana
State University
• Scott Stevenson, College of Optometry, University of Houston
1

Outline
• Show and Tell (Results)
– Estimating Eye Motion from AOSLO data
– Image Dewarping and Registration
• A Map-Seeking Circuit (MSC) estimates the motion
2

Results: Estimated Motion from AOSLO Data
200
100
0
−100
−200
−300
0 50 100 150 200 250 raw filtered
300
200
100
0
−100
−200
0 50 100 150 time
200 250 raw filtered
300
3

Image Preprocessing
1. Smooth the data with a Gaussian kernel to reduce the effect of noise and and amplitude variation.
Raw Smoothed
250
300
350
400
450
50
100
150
200
250
300
350
400
450
50
100
150
200
50 100 150 200 250 300 350 400 450 500 50 100 150 200 250 300 350 400 450 500
4

2. Break the smoothed image into one or more “channels”.
250
300
350
400
450
50
100
150
200
High
50 100 150 200 250 300 350 400 450 500
250
300
350
400
450
50
100
150
200
Medium
50 100 150 200 250 300 350 400 450 500
Low
250
300
350
400
450
50
100
150
200
50 100 150 200 250 300 350 400 450 500
5

3. Determine the eye motion across “patches”.
We can reliably calculate 16 - 128 motion estimates per frame, which yields 480 - 3840 estimates per second.
250
300
350
400
450
50
100
150
200
50 100 150 200 250 300 350 400 450 500
250
300
350
400
450
50
100
150
200
50 100 150 200 250 300 350 400 450 500
6

Image Registration
• Dewarp each frame of the AOSLO video
• Add each dewarped frame to create a denoised retinal image
Image Frame=31
250
300
350
400
100
150
200
450
500
550
100 150 200 250 300 350 400 450 500 550
7

Image Montage
• Dewarp each frame of the AOSLO video
• Add dewarped frames to create a montage
Image Frame=25
400
600
800
1000
1200
1400
1600
1800
200 400 600 800 1000 1200 1400 1600 1800 2000
8

MAP-SEEKING CIRCUIT ALGORITHM (MSC)
Model images E , E dence between E
0 as vectors in Hilbert spaces tively. Given transformation T : H → H
0 and E
0
H , H
0
, respec-
, define the corresponassociated with the transformation T to be the inner product h T ( E ) , E
0 i
H 0
.
Goal: Find T which maximizes correspondence from linear transformations of form
T = T i
( L )
L
◦ · · · ◦ T i
(2)
2
◦ T i
(1)
1
, where for each “layer” ` between 1 and L , we have i
`
∈ { 1 , 2 , . . . , n
`
} .
For example, we can let T i
(1)
1 let T i
(2)
2 be some vertical translation and be some horizontal translation of the image E .
9

ADVANTAGE of MSC over Cross-Correlation: MSC can include other transformations such as rotations, dilations, shear, compression, ....
SYNOPSIS: A Map-Seeking Circuit finds a solution to the discrete optimization problem
( i
∗
1
, . . . , i
∗
L
) = arg max
1 ≤ i
`
≤ n
`
¿
T i
( L )
L
◦ · · · ◦ T i
(2)
2
◦ T i
(1)
1
( E ) , E
0
À
H 0

MSC KEY IDEA (which makes it fast)
Imbed the discrete problem in continuous constrained optimization problem. Maximize multilinear form
M ( where T x ( ` ) x
(1)
=
, . . . , x
( L )
) = h T x ( L )
P n
` i =1 x
( ` ) i
T i
( ` )
◦ · · · ◦ T x (1)
( E ) , E
0 i
H 0
,
SIMPLIFYING PROPERTY: Components of ∇ x
M can be computed quickly and relatively cheaply via the inner product
∂M
∂x
( ` ) i
= h T i
( ` )
`
◦ T x ( ` − 1)
◦ · · · ◦ T x (1)
( E ) , T
0 x ( ` +1)
◦ · · · ◦ T
0 x ( L )
( E
0
) i
H 0
MSC is an iterative algorithm which uses this gradient simplification to maximize the correspondence h T ( E ) , E
0 i
H 0
.
10

11

COMPUTATIONAL COST
• Computational complexity of each MSC iteration is on the order of the sum n
1
+ n
2
+ . . .
+ n
L
, where n
` is the number of transformations in layer is the number of layers.
` , and L
• The complexity of an exhaustive search is the product n
1 n
2
· · · n
L
.
Convergence tends to be fast and solutions tend to be robust if the data has a “sparse encoding”. Image data pre-processing can provide such a sparse encoding.
12

CONCLUSIONS
Using MSC, we can
• Estimate eye motion from AOSLO videos.
• Identify very general transformations between AOSLO video frames, including translations, rotation, shear and compression.
• Register AOSLO video frames to create de-noised retinal images and montages
13