Digital Image Processing 5
Assessed Laboratory Exercise II
Image Correlation
The objective of this laboratory exercise is to construct a set of procedures in Matlab
that will input a stereo-pair of images and return a dense correspondence map. The
approach will be based on matching levels of a Laplacian-of Gaussians Pyramid (code
supplied), of reduction factor 1.5, and use of statistical correlation.
1. Investigate using the supplied Matlab code to implement a LOG pyramid by
reading the stereo-pair of images located at:
"http://www.dcs.gla.ac.uk/~sumitha/stereopair" into the Matlab environment
and constructing a LOG pyramid from each of these for 6 levels. Write a
routine to take each level of each pyramid and construct a single image
containing these.
2. Construct a routine that will correlate two image patches and return a
correlation score as below.
Correlation between the reference window situated at (xL,yL) in the left bandpass image and the search window situated at a trial match point (xR,yR) in the
right band-pass images is calculated as follows:
 LR xL , y L , xR , y R  
C LR  xL , y L , xR , y R 
 L2 xL , y L  R2  xR , y R 
The above formulation assumes that the input image have been pre-filtered
with a LoG filter if scale σs (not to be confused with the σ, standard deviation,
of the image patch signal) to produce zero mean images BL & BR.
The windowed covariance is computed as follows:
CLR ( xL , yL , xR , yR ) 
U
V
  G(u, v)  B ( x
L
u U v V
L
 u, yL  v)  BR ( xR  u, yR  v)
The local Gaussian windowed standard deviations are computed similarly:
 ( xL , yL ) 
2
L
 ( xR , yR ) 
2
R
U
V
  G(u, v)  B ( x
u U v V
U
L
 u, yL  v)  BL ( xL  u, yL  v)
R
 u, yR  v)  BR ( xR  u, yR  v)
L
V
  G(u, v)  B ( x
u U v V
R
Note: that that to maintain stability the Gaussian window must have a scale
factor σG ≥ σs i.e. have a standard deviation (blur factor) equal or greater than
that of the LoG filter and that the σG kernel should be passed in as a parameter
along and the size of the σG kernel must also equal the correlation window.
This kernel is constructed using fspecial. A factor of 1.3 has been found to
work well in practice (Urquhart, 97, Jin 89). Note also that the elements of this
Gaussian window must sum to 1.0. This is achieved by summing the elements
1
of the 2D Gaussian kernel supplied by fspecial and then dividing each kernel
element by this sum.
Call the above routines within a single procedure to return the correlation
score between the patches, passing in the σG kernel. Test this procedure using
the identical image patches, identical but inverse (one patch negated) patches
and two entirely different patches (the same patch transposed will usually do)
to get the expected correlations scores of 1, -1, & ~0.
3. Write a routine to input two images and apply the correlation routine at 5
locations in the right image centred on each pixel (avoiding border pixels) of
the left image, setting the correlation patch size to a YxY pixels and also
inputting σG as a parameter. The routine should return 5 correlation score
images (this could be achieved by writing a routine to correlate the same patch
location in each image and then shifting the right image) and the correlation
scores should be clipped to be ≥ 0.0.
Test the procedure using identical images and shifted versions to ensure the
expected values are being returned.
4. Now write a procedure to compute the sub-pixel disparity the horizontal and
vertical offsets, given by fitting a 3 point quadratic function horizontally and
vertically at each of the trial match locations:
•
•
•
•
•
•
•
•
•
•
•
•
•
Fit a quadratic function to the scores.
Correlation, cor = aX2 + bX + c
Find the turning point 2aX + b = 0
Best match at the turning point.
Find the roots a & b from:
cor(x,y) = c
c0 = cor(x-1,y)-cor(x,y) = a-b
c1 = cor(x+1,y)-cor(x,y) = a+b
a = (co + c1)/2, b= (c1-c0)/2
dx = -b/2.a
a = 0, flat surface, no refinement possible
a < 0, near local maximum
a > 0, near local minimum
This function should return 3 images: a horizontal disparity map, a vertical
disparity map and a confidence map (the value of aX2 + bX + c for the
estimated value of X)
Test this procedure on the supplied stereo pair of images for levels 0..5 of the
pyramid and for σG = 1.3 & 2.0 and view the recovered disparity maps both as
images and also as surfaces. Display histograms of all of these images and
comment on the effect of σG.
2