George Fisher
Brian Welsch
Space Sciences Lab, UC Berkeley
Purpose: Given two images separated in time, determine an estimate for a
2-d velocity field by finding the shifts that maximize a “localized” crosscorrelation function between the two images. The original concept was published by November & Simon (ApJ 333, 427, 1988).
Our technique: Use FFTs to compute the cross-correlation function, and use a 2 nd -order accurate Taylor expansion of the 2-d images to locate the peak of the cross-correlation function to sub-pixel accuracy. The technique does not
a-priori satisfy any physical laws.
Code: Written in C, uses the FFTW3 library (“Fastest Fourier Transform in the West”). Is designed for easy usage within an IDL or GDL session. Latest version (1.01) very fast (4096 x 4096 images in about 6 minutes).

f
1 v x v y f
2

1
2
FLCT
x
y

x

To construct a 2D velocity field that connects two images I
1
(x,y) and
I
2
(x,y) taken at two different times t
1 and t
2
, one must start from some given location within both images, compute a velocity vector, and then repeat the calculation while varying that location over all pixel positions.
Something must be done to enhance the importance of those parts of the two images that are near the selected pixel, and to de-emphasize the parts of the image that are far away from it. FLCT does this localization by multiplying each of the two images by a gaussian of width σ, centered at pixel location (x
S
1 and S
2 i
,y j
). We denote the resulting images as “sub-images”
. The expressions for S
1 and S
2 are:
S
1 i , j
( x , y )

I
1
( x , y ) e

(( x
 x i
)
2 
( y
 y j
)
2
) /
 2
S
2 i , j
( x , y )

I
2
( x , y ) e

(( x
 x i
)
2 
( y
 y j
)
2
) /
 2
.
( 1 )
The quantity σ is a free parameter in FLCT, and its optimal value changes depending on the nature of the image and the size scales present in the velocity field, as illustrated later in this poster.
5


The cross-correlation function of sub-image 1 with sub-image 2 is defined by
C i , j
(
 x ,
 y )

 dxdy S
1 i , j *
(
 x ,
 y ) S
2 i , j
(
 x
 x ,
 y
 y ).
(2)
We want to find, for each pair of sub-images S
1 position x i
,y j
, the shifts and S amplitude of the shifts, divided by the time δt =t
2
-t
1
2 centered at
δx and δy that maximize C(δx,δy). The between images 1 and 2 defines the velocity determined by FLCT: v x
= δx/δt, and v y
= δy/δt.
FLCT uses the convolution theorem to compute C( δx,δy) using Fourier transforms. If we write F(S
1
)= s
1
(k x
,k y
) and, F(S
2
) = s
2
(k x
,k y
), where F denotes Fourier transform, then the above equation can be written
C i , j
(
 x ,
 y )

F

1
( s
1
* s
2
), ( 3 ) where F -1 denotes the inverse Fourier Transform. The name “FLCT” stems from this approach: Fourier Local Correlation Tracking.
6

As a practical matter, we find the peak of f( δx,δy) ≡ |C(δx,δy)|, rather than
C( δx,δy) so that the operation doesn’t involve complex arithmetic. For notational simplicity, we henceforth use x,y for δx,δy in the following discussion.
We employ a curve-fitting approach to find the peak in f(x,y) that was inspired by that of Chae (2004, LCT code written in IDL). First, since the images and sub-images are computed at discrete points in space, we first identify the pixel coordinates (x m
,y n
) of the largest value of f, denoting the largest value of f as f(x m
,y n
). Note that (x m
,y n
) may not be equal to
(x i
,y j
), if the location of the peak of f(x,y) has shifted by more than a pixel in x or y. To find the peak to sub-pixel resolution, we then Taylor-expand f(x,y) to 2 nd order about the (x m
,y n
) location, denoting the expansion as f t
(x,y): f t
( x , y )
 f ( x m
, y n
)

 f
 x
( x
 x m
)

 f
 y
( y
 y n
)

 2 f
 x
 y
( x
 x m
)( y
 y n
)

1
2
 2 f
 x
2
( x
 x m
)
2 
1
2
 2
 y
2 f
( y
 y n
)
2
( 4 ) where the partial deriviatives are evaluated at the point (x m
,y n
).
7

At the peak, we require that the x and y partial derivatives of the Taylor expanded function f t
(x,y) be 0. These conditions result in a pair of linear equations which allow us to solve for the location (x max
,y max
) of the peak: x max y max
 x m



 2
 y
2 f
 y n



 2
 x 2 f
 f
 x

 2 f
 x
 y

 f y

 2 f
 x
 y
 f
 y

 /




 2 f
 x
 y


2

 2
 x
2 f
 f
 x

 /




 2 f
 x
 y


2

 2
 x 2 f
 2
 y
2 f
 2
 y 2 f




( 5 a )
( 5 b )
To evaluate the partial derivatives, we use standard 2 nd order finite difference expressions assuming a uniform grid in the sub-images:
 f
 x
 2
 x
2 f

( f m

1 , n


( f m

1 , n
 f m

1 , n
) f m

1 , n
/( 2
 x )

2 f m , n
)
;
/(

 f
 y

( f m , n

1
 f m , n

1
) /( 2
 y ) x
2
) ;
 2
 y
2 f

( f m , n

1
 f m , n

1

2 f m , n
) /(
 y
2
)
 2 f
 x
 y

( f m

1 , n

1
 f m

1 , n

1
 f m

1 , n

1
 f m

1 , n

1
) /( 4
 x
 y ) .
( 6 )
The total shift in x,y that corresponds to the peak in f is then given by
δx=(x m
-x i
)+(x max
-x m
), and δy=(y n
-y j
)+(y max
-y n
). Equations (5) and (6) thus provide a simple and accurate method for finding the peak of the crosscorrelation function to sub-pixel resolution.
8

Given 2 images I
1 and I
2 taken at times t
1 and t
2
=t
1
+ δt:
First, pick a value of σ, based on image properties and any a-priori knowledge of velocity properties
Then, for each pixel location in I
1 and I
2
, repeat until done:
1. Compute sub-images S
1 and S
2
.
2. Compute Fourier transforms of S
1 and S
2
, s
1 and s
2
.
3. Compute inverse Fourier transform of s
1
* s
2
. This is
C i,j .
4. Compute the absolute value of C i,j , |Ci,j|.
5. Compute the shifts δx, δy that maximize |C i,j |.
6. Compute velocities v x
= δx/δt, v y
= δy/δt.
7. Move to next pixel.
9

The FLCT code, which is currently called flct , was initially written in IDL, but has since been re-written in C for portability and speed. FLCT also uses its own endian-independent binary input format for the images and for the output velocity arrays. FLCT uses the FFTW library (version 3) for computing the Fast-Fourier Transforms needed to compute the crosscorrelation functions. We have written IDL procedures which write the two images flct needs into an input file, and which read the flct output file, consisting of arrays of v x
,v y
, and a mask array, v m
. flct was designed to be easy to run from within an IDL session.
To compile the FLCT code, the only external library needed is the FFTW3 library. To download a copy of the FLCT source code and compilation instructions, along with IDL i/o procedures, and pre-compiled binary executables for Linux (x86 and x86_64), Solaris, MS Windows, and OSX) go to http://solarmuri.ssl.berkeley.edu/overview/publicdownloads/software.html
.
Be sure to get the C version (not IDL version), and get version 1.01 or later.
10


Performing quantitative tests of correlation-tracking algorithms requires the ability to shift image data with sub-pixel resolution very accurately. Here, we outline the technique we have adopted for doing this.
How does one evaluate a function f(x,y) at points x,y that are slightly shifted from the points on which the data are sampled? Our approach is to note that the problem can be written as a convolution between the original function f(x,y) and a delta function: f ( x
  x , y
  y )

 dxdy f ( x , y )

( x
  x , y
  y ) (1)
Recognizing this, one can use the convolution theorem to write this equation in Fourier space: where
˜
( x
  x , y
  y )
 ˜
( k x
, k y

*
(
 x ,
 y )( k x
, k y
) (2)

*
(
 x ,
 y )( k x
, k y
)
 exp[ ik x
 x ] exp[ ik y
 y ] (3)




If δ x
, δ y are the same values for the entire image , i.e.
a uniform shift, then one can perform a simple inverse FFT to compute values for the shifted image: f ( x
  x , y
  y )

F

1
[ ˜ ( k x
, k y

*
(
 x ,
 y )( k x
, k y
)] (4)
This shifting technique is very fast – just a few seconds for 4096 2 images.
δx=50.75
δy=-100.25


If δx,δy are themselves functions of image location, then one cannot find the warped image values with a simple inverse FFT, because the
Fourier transform of the needed delta function is different for each point in the image. However, one can still write down the solution in closed form: f ( x i
  x i , j
, y j
  y i , j
)

  k x k y
˜
( k x
, k y
) exp[ ik x
( x i
  x i , j
)] exp[ ik y
( y j
  y i , j
)] (5)
The double sum must be performed for each spatial point of the image. This is very accurate, but for images larger than roughly
512 2 , is quite slow for typical contemporary processors.
Why do these FFT-based techniques shift and warp images more accurately than techniques based on the usage of more conventional (e.g. cubic convolution or B-spline) interpolation methods? The answer appears to lie in the destruction of phase relationships between small and large-scales in the interpolation techniques that focus on local behavior near a point.

Warp:

• In IDL, the function shift_frac2d.pro is provided as a part of the FLCT software, and uses the same syntax as the shift function in IDL: f2=shift_frac2d(f1,delx,dely). It will perform uniform shifts or warping depending on the rank of delx and dely.
• In C, we have written functions shift_frac2d and warp_frac2d which are much faster than the IDL function. The code will be publicly posted, and is currently available upon request (and has already been given to the HMI team).