Focal plane image restoration using the 2D4 algorithm
Yong Wu and Mark A. Neifeld
Electrical & Computer Engineering Department, Optical Sciences Center, The University of Arizona, Tucson, AZ 85721
yongwu@ece.arizona.edu; neifeld@ece.arizona.edu,(520) 621-6102(Tel), (520)621-6102(Fax)
Abstract: A highly parallel image restoration algorithm is described and compared with several
other popular methods including the Wiener filter, projection onto convex sets, and the Viterbi
algorithm. Parallel focal plane implementations are also discussed.
2000 Optical Society of America
OCIS codes: (100.3010) Image reconstruction-restoration; (100.6640) Superresolution
1. Introduction
Image restoration is an important component within many imaging applications. Imagery is often corrupted via
passage through a distorting channel (e.g., a turbulent atmosphere). The restoration problem involves inverting this
distortion to arrive at an estimate of the original object. A typical image distortion arises from propagation through
a lowpass optical channel. Detector noise produces additional image degradation and often results in a poorly
conditioned inverse problem. Using knowledge of the corrupting channel and noise statistics together with some
prior information concerning the object itself, numerous techniques have been described for restoring the lost high
spatial-frequency information. These methods range from traditional linear approaches (e.g., the Wiener filter) to
iterative procedures that incorporate various object-space constraints (e.g., projection onto convex sets) [1,2].
Recently, a communication-theoretic method was described in which a finite-alphabet constraint is imposed on the
object space. Such a constraint makes possible a Maximum-Likelihood (ML) solution to the image restoration
problem and a modified Viterbi algorithm (VA) was proposed as a method for approximating this ML performance
[3,4]. While offering exceptional restoration performance, the proposed VA solution suffers from a severe
complexity burden and only mild blur can be tolerated at reasonable computational loads. In this paper we describe a
new communication-theoretic image restoration algorithm. Our new algorithm exploits the finite alphabet-constraint
to achieve performance comparable to the VA; however, its highly parallel and locally connected nature facilitates
an efficient focal plane implementation.
2. Description of 2D4 Algorithm
2.1 Problem setup
The power of digital communication theory derives from the finite-alphabet constraint associated with the
transmitted data. Many imaging applications are reasonably modeled using an analogous constraint. For example,
images of text can generally be assumed to have derived from a binary-valued pixel alphabet. We will assume
throughout this paper that the objects of interest are indeed two-dimensional binary-valued arrays. The corrupting
channel is taken to be a linear, shift-invariant, lowpass filter followed by additive white Gaussian noise. The discrete
l L
m L
channel point spread function (PSF) will be given by {h(i,j)} and is normalized so that  
h(i, j )  1 where
l  L
m l
the PSF is assumed to have (2L+1) pixel support in each dimension. The resulting image measured on the CCD is
given by
g (i, j )  l  L m L f (i l , j  l )h(l , m)  n(i, j )
l L
m L
(1)
where {f(i,j)} represents the unknown object pixel values and {n(i,j)} represents the AWGN term with zero mean
and variance  2 .
2.2 Algorithm Overview
In the presence of a finite-alphabet constraint, the maximum-likelihood image restoration algorithm is conceptually
straightforward. The ML procedure will simply compare the measured image g(i,j) with all possible corrupted 2D
arrays {gˆ  h * fˆ } , selecting the one that maximizes the pagewise likelihood p[ g (i, j ) | fˆ (i, j )] . Such a task is
computationally unreasonable however, requiring 2 N
2
1
pagewise metric (e.g., likelihood) evaluations. Recent
efforts to apply ML detection to 2D data arrays have been motivated by turbo-decoding and the associated iterative,
likelihood-based decoding paradigm [5]. We apply this so-called two-dimensional distributed data detection (2D4)
algorithm to the problem of binary-valued image restoration. The 2D4 algorithm is an iterative algorithm that uses
local connectivity to manage the computational burden of ML detection. Through iterative processing on softdecision (i.e., likelihood) information, 2D4 provides a mechanism through which distant pixels can influence one
another. The algorithm is based on estimating the conditional probability p[g(i,j)|f(i,j)] using only neighborhood
data N(i,j):
p[ g (i, j ) | f (i, j )]  E{g (i, j ) | f (i, j ), N (i, j )}
  N (i , j ) p[ g (i, j ) | f (i, j ), N (i, j )]P[ N (i, j )]


  N (i , j ) p[ g (i, j ) | f (i, j ), N (i, j )]  P[ f (l , m)] 
 f ( l , m )N ( i , j )

(2)
where   {0,1}( 2 L 1)( 2 L 1) 1 is the binary-valued space of all possible neighborhood patterns. The resulting algorithm
can be defined by the update rules:
LU( k ) [ f (i, j )] 
C
N ( i , j )
ij
[ f (i, j ), N (i, j )]
L
( k 1)
(3)
[ f (l , m)]
f ( l , m )N ( i , j )
(4)
L( k ) [ f (i, j )]  (1   ) L( k 1) [ f (i, j )]  LU( k ) [ f (i, j )]
where L is the likelihood estimate at the kth iteration, Cij[f(i,j),N(i,j)] are a set of coefficients that characterize the
channel and  is a relaxation parameter that determines how much likelihood information from the current iteration
should be incoporated into the total likelihood computation.
(k)
3. Simulation result
Our simulation models an incoherent optical 4f system with a square Fourier plane aperture whose width is 1/W.
The resulting PSF is given by
(5)
h( x, y )  sin c( x / W ) sin c( y / W ) / W
where W measures the degree of blur experienced by the original image. h(x, y) is sampled and truncated to 3X3
support. The original image is assumed to comprise a 2D array of random, independent pixels with equal
probabilities of 0 and 1. The relaxation parameter  is set to 0.3.
In Table 1, we summarize the minimum SNR required for several algorithms to achieve BER=1e-3. According
to this data Threshold and Wiener filter algorithms are not capable of superresolution; while, POCS and VA are
seen to leverage the finite-alphabet constraint to achieve resolution beyond the classical limit [6].
Table 1. minimum SNR required to achieve BER=1e-3 (Unit: dB)
W
Threshold
Wiener Filter
Projection on convex sets
RCVA
2D4 algorithm
0.6
11.1
10.9
11.1
10.8
11.1
1.0
15
13.5
13.6
13.1
13.1
1.3
25.9
17.2
17.2
1.5
NA
21.2
20.7
17.7
17.9
1.8
NA
31.3
28.2
23.5
2.0
NA
NA
NA
24.6
35.3
As another example, Figure 1 depicts the BER performance of the 2D4 algorithm as a function of SNR for W
=1.5 (SNR=10log(  2 /0.52) ). From this figure we notice that the 2D4 algorithm outperforms POCS, the Wiener
filter and the Threshold algorithm.
Fig. 1 The performance of 2D4 algorithm compared to other popular algorithms for W/  =1.5 case.
4. Focal Plane Implementation of 2D4
From equations (3) and (4) we observe that each pixel communicates only with its neighboring pixels and that each
pixel on the page executes an identical computation. Likelihood updates on all pixels can therefore be performed in
parallel. A locally connected focal plane implementation is possible and an example processing array is shown in
Fig. 2.
Pixel processor
Neighborhood connections
Fig. 2 locally connected pixel processing used to implement 2D4 algorithm
The array comprises an array of pixel processors. Each pixel processor unit will consist of a detector, memory,
and ALU. The detector is used to measure the received image pixel. Memory is used to store the 3X3 h(i,j)
coefficients, likelihood information collected from neighbors, and the likelihood information computed in last
iteration. The ALU is used to calculate likelihood L[f(i,j)] updates at the current iteration. During each
computational cycle, each pixel processor collects likelihoods from its neighboring and then searches 2 8 possible
possible neighborhood configurations to calculate equations (3) and (4). After the calculated likelihood is stored in
memory, the whole computational cycle ends. All pixels in the image can finish one iteration in parallel during one
computational cycle.
5. Reference
[1] Jae S. Lim, Two-dimensional Signal and Image Processing (Prentice-Hall, 1990), Chap. 9.
[2] D.C. Youla and H. Webb, “Image Restoration by the Method of Convex Projections: Part 1-Theory,” IEEE transactions on medical imaging,
Vol. MI-1, No.2 81-94 (Oct. 1982).
[3] Casey Miller, Bobby R. Hunt, Michael W. Marcellin and Mark A. Neifeld, “Image restoration with the Viterbi algorithm,” J. Opt. Soc. Am.
A, Vol. 17, No.2 (Feb. 2000).
[4] Mark A. Neifeld, Ruozhong Xuan and Michael W. Marcellin, “Communication theoretic image restoration for binary-valued imagery,”
Applied Optics, Vol. 39, No.2 (Jan. 2000).
[5] Xiaopeng Chen, Keith M. Chugg and Mark A. Neifeld, “Near-Optimal Parallel Distributed Data Detection for Page-Oriented Optical
Memories,” IEEE Journal of Selected topics in Quantum Electronics, Vol. 4, No.5, 866-879 (Sep./Oct. 2000).
[6] H. C. Andrews and B. R. Hunt, Digital Image Restorations (Prentice-Hall, 1977), Chap. 8.