DEBLURRING AND SPARSE UNMIXING OF HYPERSPECTRAL
IMAGES USING MULTIPLE POINT SPREAD FUNCTIONS
SEBASTIAN BERISHA∗ , JAMES G. NAGY† , AND ROBERT J. PLEMMONS‡
Abstract. This paper is concerned with deblurring and spectral analysis of ground-based astronomical images of space objects. A numerical approach is provided for deblurring and sparse
unmixing of ground-based hyperspectral images (HSI) of objects taken through atmospheric turbulence. Hyperspectral imaging systems capture a 3D datacube (tensor) containing: 2D spatial
information, and 1D spectral information at each spatial location. Pixel intensities vary with wavelength bands providing a spectral trace of intensity values, and generating a spatial map of spectral
variation (spectral signatures of materials). The deblurring and spectral unmixing problem is quite
challenging since the point spread function (PSF) depends on the imaging system as well as the
seeing conditions and is wavelength varying. We show how to efficiently construct an optimal Kronecker product-based preconditioner, and provide numerical methods for estimating the multiple
PSFs using spectral data from an isolated (guide) star for joint deblurring and sparse unmixing the
HSI datasets in order to spectrally analyze the image objects. The methods are illustrated with
numerical experiments on a commonly used test example, a simulated HSI of the Hubble Space
Telescope satellite.
Key words. image deblurring, hyperspectral imaging, preconditioning, least squares, ADMM,
Kronecker product
AMS Subject Classifications: 65F20, 65F30
1. Introduction. Information about the material composition of an object is
contained most unequivocally in the spectral profiles of the brightness at the different
surface pixels of the object. Thus by acquiring the surface brightness distribution in
narrow spectral channels, as in hyperspectral image (HSI) data cubes, and by performing spectral unmixing on such data cubes, one can infer material identities as
functions of position on the object surface [1]. Spectral unmixing involves the computation of the fractional contribution of elementary spectra, called endmembers. By
assuming the measured spectrum of each mixed pixel in an HSI is a linear combination
of spectral signatures of endmembers, the underlying image model can be formulated
as a linear mixture of endmembers with nonnegative and sparse coefficients
G = XM + N ,
where M ∈ RNm ×Nw represents a spectral library containing Nm spectral signatures
of endmembers with Nw spectral bands or wavelengths, G ∈ RNp ×Nw is the observed
data matrix (each row contains the observed spectrum of a given pixel, and we use Np
to denote the number of pixels in each image), and X ∈ RNp ×Nm contains the fractional abundances of endmembers (each column contains the fractional abundances
of a given endmember). Here, we assume X is a sparse matrix and N ∈ RNp ×Nw
is a matrix representing errors or noise affecting the measurements at each spectral
∗ Department of Mathematics and Computer Science, Emory University.
Email: sberish@emory.edu.
† Department of Mathematics and Computer Science,
Emory University.
Email:
nagy@mathcs.emory.edu. Research supported in part by the AFOSR under grant FA9550-09-1-0487,
and by the US National Science Foundation under grant no. DMS-1115627.
‡ Department of Computer Science, Wake Forest University, Winston-Salem, NC, USA. Email:
plemmons@wfu.edu. His work was supported by grant no. FA9550-11-1-0194 from the US Air Force
Office of Scientific Research and by contract no. HM1582-10-C-0011 from the US National GeospatialIntelligence Agency.
1
2
S. BERISHA, J. NAGY AND R. PLEMMONS
band or wavelength; see, e.g. [1, 14]. If we assume that the data at each wavelength
has been degraded by a blurring operator, H, then the problem takes the form
G = HXM + N .
Given a spectral library of the endmembers, M , and assuming that we have computed
a priori the parameters defining the blurring operator H, the goal becomes to compute
the nonnegative and sparse matrix of fractional abundances, X.
A major challenge for deblurring hyperspectral images is that of estimating the
overall blurring operator H, taking into account the fact that the blurring operator
point spread function (PSF) can vary over the images in the HSI datacube. That
is, the blurring can be wavelength dependent and depend on the imaging system
diffraction blur as well as the effects of atmospheric turbulence blur on the arriving wavefront, see e.g., [5, 9, 10, 12]. We point especially to recent development of
the hyperspectral Multi Unit Spectroscopic Explorer (MUSE) system installed on the
Very Large Telescope (VLT) being deployed by the European Southern Observatory
(ESO) at the Paranal Observatory in Chile. The MUSE system will collect up to
4, 000 bands, and research is ongoing to develop methods for estimation of the wavelength dependent PSFs for deblurring the resulting HSI datacube for ground-based
astrophysical observations, see e.g. [9, 12]. In particular, Soules, et al. [10] consider
the restoration of hyperspectral astronomical data with spectrally varying blur, but
assume that the spectrally varying PSF has already been provided by other means,
and defer the PSF estimation to a later time. In this paper we consider the estimation of a Moffat function parameterization of the PSF as a function of wavelength
and derive a numerical scheme for deblurring and unmixing the HSI datacube using
the estimated PSFs. Moffat function parameterizations capture the diffraction blur of
the telescope system as well as the blur resulting from imaging through atmospheric
turbulence, and have been used quite effectively by astronomers for estimating PSFs;
see, for example, the survey by Soulez et al. [10].
This paper is outlined as follows. In Section 2 we review a numerical approach for
deblurring and sparse unmixing of HSI datacubes for the special case of a homogeneous
PSF across the wavelength, based on work by Zhao et al. [14]. Section 3 concerns
estimating the wavelength dependent PSFs to model the blurring effects of imaging
through the atmosphere, and application of a numerical scheme for this multiple PSF
case using a preconditioned alternating direction method of multipliers. We show
how to efficiently construct an optimal Kronecker product-based preconditioner to
accelerate convergence. In order to illustrate the use of our method, some numerical
experiments on a commonly used test example, a simulated hyperspectral image of
the Hubble Space Telescope satellite, are reported in Section 4. Some concluding
comments are provided in Section 5.
2. Numerical Scheme for the Single PSF Case. In this section we describe
and expand upon the numerical scheme used in [14] for solving the hyperspectral
image deblurring and unmixing problem by using the Alternating Direction Method
of Multipliers (ADMM) in the single PSF case. Here, it is assumed that the blurring
operator H is defined by a single PSF and each column of XM is blurred by the
same H. The authors in [14] have presented a total variation (TV) regularization
method for solving the deblurring and sparse hyperspectral unmixing problem, which
takes the form
1
min ||HXM − G||2F + µ1 ||X||1 + µ2 T V (X)
X≥0 2
3
HYPERSPECTRAL IMAGING WITH MULTIPLE PSF
where H ∈ RNp ×Np is a blurring matrix constructed from a single Gaussian function
assuming periodic boundary conditions, and µ1 , µ2 are two regularization terms used
to control the importance of the sparsity and the total variation terms. Numerical
schemes for both isotropic and anisotropic total variation are presented in [14]. For
isotropic TV, the above problem can be rewritten as
Np Nm
X
X
1
||Wij ||2
min ||HXM − G||2F + µ1 ||V ||1 + µ2
2
i=1 j=1
(2.1)
subject to
Dh X = W (1) ,
Dv X = W (2) ,
V = X,
V ∈ K = {V ∈ RNp ×Nm , V ≥ 0} ,
where
h
i
(1)
(2)
Wi,j = Wi,j
, Wi,j ∈ R1×2 ,
(1)
Wi,j = Di,h xj ,
(2)
Wi,j = Di,v xj ,
1 ≤ i ≤ Np ,
1 ≤ j ≤ Nm .
Here, the matrices Dh and Dv represent the first order difference matrices in the
horizontal and vertical direction, respectively. The authors in [14] solve the above
problem using an alternating direction method. The problem in (2.1) can be decoupled
by letting
f1 (X) =
1
||HXM − G||2F
2
and
f2 (Z) = XK (V ) + µ2
Np Nm
X
X
||Wij ||2 + µ1 ||V ||1
i=1 j=1
where
 (1) 
W
0 if V ∈ K
Z = W (2)  , XK =
.
∞ otherwise
V
The constraints are expressed as


 (1) 
Dh
W
BX + CZ =  Dv  X − I3Np ×3Np W (2)  = 03Np ×Nm ,
INp ×Np
V
where we use I and 0 to denote, respectively, an identity matrix and matrix of all
zeros (subscripts on these matrices define their dimensions, and may be omitted later
in the paper, if dimensions are clear from the context).
Furthermore, by attaching the Lagrange multiplier to the linear constraints the
augmented Lagrangian function of (2.1) is written as
β
L(X, Z, Λ) = f1 (X) + f2 (Z)+ < Λ, BX + CZ > + ||BX + CZ||2F ,
2
4
S. BERISHA, J. NAGY AND R. PLEMMONS
where
 (1) 
Λ
Λ = Λ(2)  ∈ R3Np ×Nm ,
Λ(3)
β > 0 is the penalty parameter for violating the linear constraints, and < ·, · > is the
sum of the entries of the Hadamard product. With this formulation the hyperspectral
unmixing and deblurring problem is solved using an alternating direction method
consisting of solving 3 subproblems at each iteration k:

 Step 1: Xk+1 ← arg min L(X, Zk , Λk )
Step 2: Zk+1 ← arg min L(Xk+1 , Z, Λk )
.

Step 3: Λk+1 ← Λk + β(BXk+1 + CZk+1 )
The X-subproblem, or Step 1, consists of solving
β
1
Xk+1 ∈ arg min{ ||HXM − G||2F + < Λ, BX + CZ > + ||BX + CZ||2F }. (2.2)
2
2
X
The above subproblem is the solution of the classical Sylvester matrix equation
H T HXM M T + βB T BX = H T GM T − βB T CZk − B T Λk .
(2.3)
Similar alternating minimization schemes have been used for solving the hyperspectral unmixing problem in [3] and [6]. However, the key step of the alternating
minimization scheme presented in [14] consists in transforming the matrix equation
(2.3) to a linear system which has a closed-form solution. In particular the authors
in [14] reformulate the Sylvester matrix equation in (2.3) as
(M M T ⊗ H T H + βI ⊗ B T B)x = ĝ ,
where
x = vec(X) = vec( x1
···
xn


x1


) =  ...  ,
xn
xi = ith column of X ,
and similarly, ĝ = vec(H T GM T − βB T CZk − B T Λk ). Let M = U ΣV T be the
singular value decomposition of M , and let H = F ∗ ΓF and B T B = F ∗ ΨF be
the Fourier decomposition of H and B T B, respectively. Here, we assume spatially
invariant blur with periodic boundary conditions. The above linear system takes the
form
(U ⊗ F ∗ )(ΣΣT ⊗ Γ2 + I ⊗ Ψ2 )(U T ⊗ F )x = ĝ
and thus a direct solution is given by
x = (U ⊗ F ∗ )(ΣΣT ⊗ Γ2 + I ⊗ Ψ2 )−1 (U T ⊗ F )ĝ.
For a detailed description of the solution of Steps 2 and 3 of the alternating minimization approach see [14].
HYPERSPECTRAL IMAGING WITH MULTIPLE PSF
5
3. Numerical Scheme for the Multiple PSF Case. In this section we provide the problem formulation for the general case where each column of the matrix
XM is generally blurred by a different blurring operator. In particular, the deblurring and hyperspectral unmixing problem with multiple PSFs takes the form


 

g1
H1 0 · · ·
0
XM e1

 0 H2 · · ·

 
0 

  XM e2   g2 
(3.1)
= . 
 ..



.
.
.
..
..
..  
 .
  .. 
0
···
0
0
HNw
XM eNw
gNw
where ei ∈ RNw is the ith unit vector, gi is the ith column of the observed matrix
G, and each blurring matrix Hi is defined by different PSFs that vary with wavelength. For example, in astronomical imaging, a blurring operator H for a particular
wavelength, λ, might be accurately modeled with a circular Moffat function
−α2
i2 + j 2
α2 − 1
(3.2)
1+
PSF(α0 , α1 , α2 , λ) =
π(α0 + α1 λ)
(α0 + α1 λ)2
Moffat functions are widely used to paramaterize PSFs in astronomical imaging. The
parameters α0 , α1 and α2 are the Moffat function shape parameters for the associated
PSF which are to be estimated from the data, see, e.g. [9].
Notice that problem (3.1) can be rewritten as

 

H1 XM e1
g1
 H2 XM e2   g2 

 


 =  ..  .
..



.
. 
HNw XM eNw
gNw
By utilizing Kronecker product properties the

(eT1 M T ⊗ H1 )x
 (eT2 M T ⊗ H2 )x


..

.
above equation can be reformulated as
 

g1
  g2 
 

 =  ..  ,
  . 
(eTNw M T ⊗ HNw )x
gNw
where x = vec(X). Thus, the multiple PSF hyperspectral image deblurring and
unmixing problem can be formulated as
Hx = g
where
mT1 ⊗ H1
 mT2 ⊗ H2

H=
..

.

mTNw ⊗ HNw



,


x = vec(X),
g1
g2
..
.





g=
.


gNw
Hence, the X-subproblem (2.2) for the multiple PSF formulation takes the form
1
β
2
2
Xk+1 ∈ arg min
||Hx − g||F + < Λ, BX + CZ > + ||BX + CZ||F .
2
2
X
6
S. BERISHA, J. NAGY AND R. PLEMMONS
That is, using Kronecker product properties and by applying the vectorizing operator, vec(·), Xk+1 is a solution of the
1
β
2
T
2
min kHx − gkF +vec(Λ) ((I ⊗B) x + vec (CZ)) + k (I ⊗B) x + vec(CZ)kF ,
x
2
2
where x = vec(X). Now, if we set the gradient of the augmented Lagrangian for the
X-subproblem to 0 then we obtain
HT Hx + β I ⊗ B T B x = HT g − β I ⊗ B T vec(CZk ) − vec(B T Λk ).
The above equation can be rewritten as
(HT H + βI ⊗ B T B)x = ĝ ,
where ĝ = HT g − β I ⊗ B T vec(CZk ) − vec(B T Λk ). Notice that

mT1 ⊗ H1
T

 m2 ⊗ H2
T
T
T
T
H H = m1 ⊗ H1 m2 ⊗ H2 · · · mNw ⊗ HNw 
..

.
mTNw ⊗ HNw
(3.3)





T
= m1 mT1 ⊗ H1T H1 + m2 mT2 ⊗ H2T H2 + · · · + mNw mTNw ⊗ HN
HNw .
w
Using the decompositions HiT Hi = F ∗ Γ2i F and B T B = F ∗ Ψ2 F equation (3.3)
takes the form
(m1 mT1 ⊗F ∗ Γ21 F +m2 mT2 ⊗F ∗ Γ22 F +· · ·+mNw mTNw ⊗F ∗ Γ2Nw F +I ⊗F ∗ Ψ2 F )x = ĝ.
Thus, the X-subproblem to be solved for the multiple PSF case is
(I ⊗ F ∗ )(m1 mT1 ⊗ Γ21 + m2 mT2 ⊗ Γ22 + · · · + mNw mTNw ⊗ Γ2Nw + I ⊗ Ψ2 )(I ⊗ F )x = ĝ.
Notice that the middle part of the coefficient matrix in the above linear system is
not diagonal as in the single PSF case, and thus there is not an explicit solution of the
X-subproblem for multiple PFSs. However, the coefficient matrix for multiple PSFs
is symmetric positive definite (spd) and thus we use the conjugate gradient method
to solve the above subproblem. Construction of a preconditioner is described next.
4. Conjugate Gradient Preconditioner. The X-subproblem involves the coefficient matrix
(I ⊗ F ∗ )(m1 mT1 ⊗ Γ21 + m2 mT2 ⊗ Γ22 + · · · + mNw mTNw ⊗ Γ2Nw + I ⊗ Ψ2 )(I ⊗ F ) .
Our goal is to approximate m1 mT1 ⊗ Γ21 + · · · + mNw mTNw ⊗ Γ2Nw by one Kronecker
product
A⊗D,
where D is a diagonal matrix, and A is spd. If we can find such an approximation,
then we can compute A = U ΣU T , and
(I ⊗ F ∗ )(m1 mT1 ⊗ Γ21 + m2 mT2 ⊗ Γ22 + · · · + mNw mTNw ⊗ Γ2Nw + I ⊗ Ψ2 )(I ⊗ F )
≈ (I ⊗ F ∗ )(A ⊗ D + I ⊗ Ψ2 )(I ⊗ F )
= (I ⊗ F ∗ )(U ΣU T ⊗ D + I ⊗ Ψ2 )(I ⊗ F )
= (U ⊗ F ∗ )(Σ ⊗ D + I ⊗ Ψ2 )(U T ⊗ F ) .
7
HYPERSPECTRAL IMAGING WITH MULTIPLE PSF
4.1. Kronecker Product Approximation. One, very simple approximation,
can be obtained by replacing each Γj by a single matrix, such as by an average of all
diagonal matrices, Γavg . In this case we use
A = M M T and D = Γ2avg .
However, it is difficult to determine the quality of using such a simple approach.
Therefore, we seek a different and possibly “optimal” approximation. Consider


mT1 ⊗ Γ1


..
m1 mT1 ⊗Γ21 + · · · + mNw mTNw ⊗Γ2Nw = m1 ⊗Γ1 · · · mNw ⊗ΓNw 

.
mTNw ⊗ ΓNw
= CC T ,
where
C = m1 ⊗ Γ1

m11 Γ1
 m21 Γ1

=
..

.
m2 ⊗ Γ2
···
mNw ⊗ ΓNw
m12 Γ2
m22 Γ2
···
···
mNm 1 Γ1
mNm 2 Γ2
···

m1Nw ΓNw
m2Nw ΓNw 

.

mNm Nw ΓNw
Notice that if we can approximate C with C ≈ M̂ ⊗ Γ̂, where Γ̂ is diagonal,
then CC T ≈ M̂ M̂ T ⊗ Γ̂2 . For example, one such simple approximation, as discussed
above, is
M̂ = M ,
Γ̂ = Γavg .
Instead of using this approach, we show how to find an “optimal” M̂ and Γ̂ that
minimizes
kC − M̂ ⊗ Γ̂kF .
Using ideas from Van Loan and Pitsianis [11], we can find the approximation
C ≈ M̂ ⊗ Γ̂ by transforming C to “tilde” space; the optimal Kronecker product
approximation of a matrix is obtained by using the optimal rank-1 approximation of
the transformed matrix. In our case, the tilde transformation of C is given by

 

vec(m11 Γ1 )T
m11 vec(Γ1 )T
 vec(m21 Γ1 )T   m21 vec(Γ1 )T 
 


..
..

 





.
.




 vec(mNm 1 Γ1 )T   mNm 1 vec(Γ1 )T 




 vec(m12 Γ2 )T   m12 vec(Γ2 )T  





m1 vec(Γ1 )T
 vec(m22 Γ2 )T   m22 vec(Γ2 )T 

 
 

 
  m2 vec(Γ2 )T 
..
..

=
=
.
.
C̃ = 

..

 
 

T
T
.
 vec(mNm 2 Γ2 )   mNm 2 vec(Γ2 ) 
 


T
m
vec(Γ
)

 

..
..
Nw
Nw

 

.
.

 

 vec(m1N ΓN )T   m1N vec(ΓN )T 
w
w
w
w

 

 vec(m2N ΓN )T   m2N vec(ΓN )T 
w
w
w
w

 


 

..
..

 

.
.
vec(mNm Nw ΓNw )T
mNm Nw vec(ΓNw )T
8
S. BERISHA, J. NAGY AND R. PLEMMONS

m1
 0

= .
 ..
0
0
m2
···
···
..
.
0
···
0
0
..
.

vec(Γ1 )T
vec(Γ2 )T
..
.





,



mNw
vec(ΓNw )T
where mij is the ith entry in vector mj , for i = 1, · · · , Nm and j = 1, · · · , Nw . To
find the optimal Kronecker product approximation of C, first observe (see, e.g., [11])
that
kC − M̂ ⊗ Γ̂kF = kC̃ − vec(M̂ )vec(Γ̂)T k2 ,
and thus the optimal Kronecker product approximation problem is equivalent to an
optimal rank-1 approximation of C̃. By the Eckhart-Young theorem (see, e.g., [2]),
the best rank-1 approximation is obtained from the largest singular value and corresponding singular vectors of C̃; that is,
C̃ ≈ σ˜1 u˜1 v˜1 T .
The matrices M̂ and Γ̂ are then constructed from σ˜1 , u˜1 , and v˜1 ; specifically,
p
p
vec(M̂ ) = σ˜1 ũ1 , vec(Γ̂) = σ̃1 v˜1 .
4.2. Computing the largest singular triplet of C̃. The matrix C̃ can be
quite large, Nm Nw ×Np2 , so it is important to exploit its structure in order to efficiently
compute the largest singular triplet; simply exploiting sparsity is not enough. To
describe how we do this, we first define matrices




vec(Γ1 )T
m1
0 ···
0
 vec(Γ2 )T 
 0 m2 · · ·
0 




M= .
,
..  and T = 
..
..



 ..
.
.
.
T
0
0 · · · mNm
vec(ΓNw )
so that C̃ = MT . Now notice that since each Γj is an Np × Np diagonal matrix, then
at most Np columns of T are nonzero. Thus, there is a permutation matrix P such
that
TP = T 0
⇔ T = T 0 PT ,
where 0 is an Nw × Np2 − Np matrix of all zeros, and
γ1T

 γT
 2
T =
 ..
 .






T
γN
w
is an Nw × Np matrix, where γj is a vector containing the diagonal elements of Γj .
Next, observe that the structure of M allows us to efficiently compute a thin QR
decomposition,
M = QR ,
HYPERSPECTRAL IMAGING WITH MULTIPLE PSF
9
where R = diag(km1 k2 , km2 k2 , . . . kmNw k2 ). Thus, we can now rewrite C̃ as
C̃ = MT
= QR T 0 P T
= Q RT 0 PT .
Notice that RT is an Nw × Np matrix, which is relatively small compared to the
Nm Nw × Np2 matrix C̃. In addition, because Q and P are orthogonal matrices, to
compute the largest singular triplet of C̃, we need only compute the largest singular
triplet of RT . That is, if we denote the largest singular triplet of RT as (ut , σt , vt ),
and the largest singular triplet of C̃ as (ũ1 , σ̃1 , ṽ1 ), then
vt
σ̃1 = σt , ũ1 = Qut and ṽ1 = P
,
0
where 0 is an (Np2 − Np ) × 1 vector of all zeros. It is also important to notice that
√
the zero structure of ṽ1 implies that if vec(Γ̂) = σ̃1 ṽ1 , then Γ̂ is a diagonal matrix
– precisely what we need for our preconditioner.
4.3. Approximation Quality of the Preconditioner. In this subsection, we
consider the approximation quality of the preconditioner. It is difficult to give a
precise analytical result on the quality of the approximation, but it is possible to
get a rough bound on kC − M̂ ⊗ Γ̂k2F . The size of this norm depends on how the
PSFs vary with wavelength. Specifically, if M̂ ⊗ Γ̂ is the matrix that minimizes the
Frobenius norm, then
kC − M̂ ⊗ Γ̂k2F ≤ kC − M ⊗ Γavg k2F
=
Nm X
Nw
X
|mij |2 kEj k2F
i=1 j=1
≤ Nm
Nw
X
kEj k2F ,
j=1
where Ej = Γj − Γavg , and the last inequality results because, without loss of generality, we can assume 0 ≤ mij ≤ 1 (see Fig. 5.3). Thus, if Γ1 ≈ · · · ≈ ΓNw (that is,
the PSFs are approximately equal), then we obtain a small approximation error with
our preconditioner.
Because of the high nonlinearity of the PSFs, we have not been able to refine this
bound any further. However, we can say that it is known that all the wavelengthdependent exact PSFs for hyperspectral imaging through atmospheric turbulence are
scaled versions of a base PSF associated with the optical path difference function [8]
for the arriving light wavefront phase. Also, the blurring effects of the PSFs become
essentially much less at longer wavelengths, and there the scaling factor begins to
approach one, so that the PSFs become almost identical, see e.g., [5]. The wavelength
where this begins to occur is of course problem dependent and depends upon the level
of turbulence.
We note that, when evaluating the quality of preconditioners, it is often useful
to look experimentally at the singular values of the original matrix and the preconditioned system. However, this can only be done for very small problems. Figure 4.1
10
S. BERISHA, J. NAGY AND R. PLEMMONS
shows the singular values for 10 preconditioned and non-preconditioned systems. We
used PSFs of size 10 × 10 and vary the number of wavelengths from 9 to 99 with a
10nm step size. Note that the singular values of the preconditioned system tend to
cluster more towards 1 compared to the non-preconditioned system. We also noticed
that the singular values of the preconditioned system show a tendency to move away
from 0. The behavior of the singular values is similar as the number of wavelengths
varies. Even though there is not a tight clustering of the singular values around 1,
as one would expect from an extremely effective preconditioner, our numerical results show that our Kronecker product-based preconditioner significantly reduces the
number of necessary iterations for the convergence of the conjugate gradient method.
Fig. 4.1. Singular values for the non-preconditoned (left) and preconditioned (right) systems
with varying number of wavelengths.
5. Numerical results. In this section we test the proposed numerical scheme
for the deblurring and unmixing model using single and multiple PSFs. Simulated hyperspectral data are used to evaluate the proposed method. We compare the multiple
PSF approach with the single PSF approach. The quality of the estimated fractional
abundances of endmembers is evaluated by using the relative error defined by:
||Xtrue − X||2
,
||Xtrue ||2
where Xtrue is the matrix of true fractional abundances of endmembers and X is
the computed fractional abundances of endmembers by the proposed method. In all
experiments, we used the circular Moffat functions defined in equation (3.2) to model
the PSFs, with α0 = 2.42, α1 = −0.001, and α2 = 2.66.
We consider a simulated hyperspectral image of the Hubble Space Telescope,
which is also used for testing in [14]. Similar data was also used in [7] and [13]. The
signatures cover a band of spectra from 400nm to 2500nm. We use 99 evenly distributed sampling points, leading to a hyperspectral datacube of size 128 × 128 × 99.
Six materials typical to those associated with satellites are used, Hubble aluminum,
Hubble glue, Hubble honeycomb top, Hubble honeycomb side, solar cell, and rubber
edge. The synthetic map of the satellite image is shown in Figure 5.1. The hyperspectral datacube is blurred by multiple circular Moffat point spread functions (see
Figure 5.2), i.e. each column of G is blurred by a different circular Moffat function
corresponding to a particular wavelength. Note that as the wavelength, λ, increases
there is less blurring present in those columns compared to the blurring in the columns
HYPERSPECTRAL IMAGING WITH MULTIPLE PSF
11
observed at shorter wavelengths, as expected e.g. [4]. Gaussian white noise in the level
of 30dB is also added to the datacube.
In the experiments for the numerical scheme with multiple PSFs we use all the
PSFs for the reconstruction of the fractional abundances. In the single PSF scheme
we use an average of all the PSFs for reconstruction. That is, the columns of G
are blurred with different PSFs in both cases and we use one PSF representing the
average of all PSFs for reconstruction in the single PSF numerical scheme. The plot of
relative errors is shown in Figure 5.4. One can observe that the use of multiple PSFs
provides lower relative reconstruction errors compared to the relative errors obtained
by the single PSF case. It is a known fact that the blurring varies with different
wavelengths. Figure 5.4 shows that by taking this fact into account we can achieve
much lower relative errors in the reconstruction of the fractional abundances.
The following values are used for the parameters in the alternating minimization
scheme: β = 10−2 , µ1 = 10−1 , µ2 = 5 · 10−4 . The convergence of the alternating
direction method is theoretically guaranteed as long as the penalty parameter β is
positive, see e.g. [14]. We note that the conjugate gradient method required 1, 000
iterations to solve each X-subproblem (for multiple PSFs) at the same accuracy
level as the single PSF method. By using the preconditioner (presented in Section
3.1) we were able to reduce the number of iterations required for convergence to
20. In Figure 5.5 we show the relative residual norms for the first 95 iterations
of preconditioned conjugate gradient (PCG) and conjugate gradient (CG) without
preconditioning. It is clear from this figure that the relative residual norms for PCG
decrease very quickly until they reach the default tolerance level of 10−6 whereas for
CG the relative residual norms decrease very slowly. Figures 5.6 and 5.7 show the
reconstructed columns of X for both the single PSF and multiple PSF methods.
The reconstructed spectral signatures for the various materials are shown in Figure 5.8. This figure shows clearly that the reconstructed spectra using multiple PSFs
are much better approximations to the true spectra than when using a single PSF. As
with the results shown in Figure 5.4, the single PSF is obtained averaging all PSFs.
Figure 5.9 is essentially the same as Figure 5.8, except we also include the spectral
signatures of the observed (blurred) data.
Fig. 5.1. Synthetic map representation of the hyperspectral satellite image. The false colors
are used to denote different materials, which are defined in Table 5.1.
12
S. BERISHA, J. NAGY AND R. PLEMMONS
Table 5.1
Materials, corresponding colors (see Fig. 5.1), fractional abundances of constituent endmembers, and fractional abundances of the materials used for the Hubble satellite simulation.
Material
1
Color
light gray
Percent
Constituent Endmembers
Em. 1 (100)
Percent
Fractional Abundance
11
2
green
Em. 2 (70), Em. 9 (30)
18
3
red
Em. 3 (100)
4
4
dark gray
Em. 4 (60), Em. 10 (40)
19
5
brown
Em. 5 (100)
7
6
gold
Em. 6 (40), Em. 11 (30),
32
Em. 12 (30)
7
blue
Em. 7 (100)
3
8
white
Em. 8 (100)
6
HYPERSPECTRAL IMAGING WITH MULTIPLE PSF
13
Fig. 5.2. First column: the true columns of G observed at different wavelengths (from top to
bottom: 400nm, 1107.1nm, 1814.3nm, 2500nm). Second column: the circular Moffat PSFs used to
blur the columns of G at different wavelengths. Third column: the corresponding blurred and noisy
columns of G blurred with different Moffat PSFs corresponding to different wavelengths.
14
S. BERISHA, J. NAGY AND R. PLEMMONS
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0
0
20
40
60
80
0
0
100
20
40
60
80
0
0
100
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0
0
20
40
60
80
0
0
100
20
40
60
80
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
20
40
60
80
100
0
0
0
0
100
20
40
60
80
20
40
60
80
100
20
40
60
80
100
100
Fig. 5.3. Spectral signatures of eight materials assigned to the simulated Hubble Telescope model.
1
0.9
Relative Errors
0.8
Multiple PSFs
One PSF
0.7
0.6
0.5
0.4
0.3
0.2
0
50
100
Iterations
150
200
Fig. 5.4. Relative errors for the computed fractional abundances using a single PSF and multiple PSFs.
HYPERSPECTRAL IMAGING WITH MULTIPLE PSF
15
0
10
−1
10
Relative residual norms
−2
10
−3
10
−4
10
−5
10
−6
10
PCG
CG
−7
10
0
20
40
60
80
100
Iterations
Fig. 5.5. Relative residual norms for the first 95 iterations of PCG and CG.
Fig. 5.6. Fractional abundances for materials 1 to 4 (the first 4 materials in Table 5.1). First
column: true fractional abundances; second column: the estimated fractional abundances using the
single PSF approach; third column: the estimated fractional abundances using the multiple PSF
approach.
16
S. BERISHA, J. NAGY AND R. PLEMMONS
Fig. 5.7. Fractional abundances for materials 5 to 8 (the last 4 materials in Table 5.1). First
column: true fractional abundances; second column: the estimated fractional abundances using the
single PSF approach; third column: the estimated fractional abundances using the multiple PSF
approach.
17
HYPERSPECTRAL IMAGING WITH MULTIPLE PSF
1
0.8
1
1
0.9
0.9
0.8
0.8
0.6
0.7
0.7
0.6
0.4
0.6
0.5
0.2
0
0
0.5
20
40
60
80
100
0.4
0.4
0
1
1.2
0.9
1.1
20
40
60
80
100
0
20
40
60
80
100
20
40
60
80
100
1.3
1.2
1.1
1
0.8
1
0.9
0.9
0.8
0.8
0.7
0.6
0.7
0.7
0.6
0.5
0.4
0
0.6
20
40
60
80
100
0.5
0.5
0
20
40
60
1
1
0.9
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0
20
40
60
80
100
0.4
0
80
20
100
40
0.4
0
60
80
100
Fig. 5.8. The true material spectral signatures (blue − and +), the computed spectral signatures
using the multiple PSF numerical approach (red −− and o) , and the computed spectral signatures
using the single PSF approach (magenta : and ). Note that the y-axis has not been scaled in order
to show more clearly the differences between spectral signatures in the three cases.
18
S. BERISHA, J. NAGY AND R. PLEMMONS
1
0.8
1
1
0.9
0.9
0.8
0.8
0.6
0.7
0.7
0.6
0.4
0.6
0.5
0.2
0
0
0.5
20
40
60
80
100
0.4
0.4
0
1
1.2
0.9
1.1
20
40
60
80
100
0
20
40
60
80
100
20
40
60
80
100
1.3
1.2
1.1
1
0.8
1
0.9
0.9
0.8
0.8
0.7
0.6
0.7
0.7
0.6
0.5
0.4
0
0.6
20
40
60
80
100
0.5
0.5
0
20
40
60
1
1
0.9
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0
20
40
60
80
100
0.4
0
80
20
100
40
0.4
0
60
80
100
Fig. 5.9. The true material spectral signatures (blue − and +), the computed spectral signatures
using the multiple PSF numerical approach (red −− and o), the computed spectral signatures using
the single PSF approach (magenta : and ), and the original observed blurred and noisy material
spectral signatures (black and ). Note that the y-axis has not been scaled in order to show more
clearly the differences between spectral signatures in the four cases.
HYPERSPECTRAL IMAGING WITH MULTIPLE PSF
19
6. Conclusions. We have presented a numerical approach, based on the ADMM
method, for deblurring and sparse unmixing of ground-based hyperspectral images of
objects taken through the atmosphere at multiple wavelengths with narrow spectral
channels. Because the PSFs, which define the blurring operations, depend on the
imaging system as well as the seeing conditions and is wavelength dependent, the
reconstruction process is computationally intensive. In particular, we found it important to use a preconditioned conjugate gradient method to solve a large-scale linear
system needed at each ADMM iteration. We showed how to efficiently construct an
optimal Kronecker product-based preconditioner, and provided numerical experiments
to illustrate the effectiveness of our approach. In particular, we illustrated that much
better accuracy can be obtained by using the multiple, wavelength dependent PSFs,
and we showed that our preconditioner is quite effective in significantly reducing the
number of conjugate gradient iterations.
REFERENCES
[1] M. T. Eismann, Hyperspectral Remote Sensing, SPIE Press, 2012.
[2] G. H. Golub and C. F. Van Loan, Matrix Computations, 4th Ed., Johns Hopkins University
Press, Baltimore, 2013.
[3] M.-D. Iordache, J. M. Bioucas-Dias, and A. Plaza, Total variation spatial regularization
for sparse hyperspectral unmixing, Geoscience and Remote Sensing, IEEE Transactions on,
50 (2012), pp. 4484–4502.
[4] S. M. Jefferies and M. Hart, Deconvolution from wavefront sensing using the frozen flow
hypothesis, Optics express, 19 (2011), pp. 1975–1984.
[5] A. J. Lambert and G. Nichols, Wavelength diversity in restoration from atmospheric turbulence effected surveillance imagery, in Frontiers in Optics 2009, Laser Science XXV, Fall
2009 OSA Optics & Photonics Technical Digest, OSA Technical Digest (CD).
[6] C. Li, T. Sun, K. F. Kelly, and Y. Zhang, A compressive sensing and unmixing scheme
for hyperspectral data processing, Image Processing, IEEE Transactions on, 21 (2012),
pp. 1200–1210.
[7] F. Li, M. K. Ng, and R. J. Plemmons, Coupled segmentation and denoising/deblurring models
for hyperspectral material identification, Numerical Linear Algebra with Applications, 19
(2012), pp. 153–173.
[8] M. C. Roggemann and B. M. Welsh, Imaging Through Turbulence, CRC press, 1996.
[9] D. Serre, E. Villeneuve, H. Carfantan, L. Jolissaint, V. Mazet, S. Bourguignon, and
A. Jarno, Modeling the spatial PSF at the VLT focal plane for MUSE WFM data analysis
purpose, in SPIE Astronomical Telescopes and Instrumentation: Observational Frontiers
of Astronomy for the New Decade, International Society for Optics and Photonics, 2010,
pp. 773649–773649.
[10] F. Soulez, E. Thiebaut, and L. Denis, Restoration of hyperspectral astronomical data with
spectrally varying blur, European Astronomical Society Publications Series, 59 (2013),
pp. 403–416.
[11] C. F. Van Loan and N. P. Pitsianis, Approximation with Kronecker products, in Linear
Algebra for Large Scale and Real Time Applications, M. S. Moonen and G. H. Golub, eds.,
Kluwer Publications, 1993, pp. 293–314.
[12] E. Villeneuve, H. Carfantan, and D. Serre, PSF estimation of hyperspectral data acquisition system for ground-based astrophysical observations, in Hyperspectral Image and
Signal Processing: Evolution in Remote Sensing (WHISPERS), 2011 3rd Workshop on,
IEEE, 2011, pp. 1–4.
[13] Q. Zhang, H. Wang, R. Plemmons, and V. Pauca, Tensor methods for hyperspectral data
analysis: a space object material identification study, Journal of the Optical Society of
America A, 25 (2008), pp. 3001–3012.
[14] X.-L. Zhao, F. Wang, T.-Z. Huang, M. K. Ng, and R. J. Plemmons, Deblurring and sparse
unmixing for hyperspectral images, Geoscience and Remote Sensing, IEEE Transactions
on, 51 (2013), pp. 4045–4058.