Sub-pixel mapping of abundance maps by contour minimization ;
application to hyperspectral imaging and to binary image interpolation.
C. Louchet
MAPMO, UMR CNRS 7349 & Fédération Denis Poisson FR CNRS 2964
Université d’Orléans
cecile.louchet@univ-orleans.fr
Résumé
Nous proposons une méthode d’estimation d’une image
d’étiquettes à partir de la proportion de chaque étiquette
sur des blocs disjoints. Nous l’appliquons à l’interprétation de cartes d’abondance données par le démélange
d’images hyperspectrales et au zoom d’images binaires.
Mots Clef
Cartographie sous-pixellique, cartes d’abondance, imagerie hyperspectrale, zoom d’image binaire, problème inverse
régularisé.
Abstract
We propose a method that estimates a label image from the
proportion of the labels on disjointed blocks. We apply it to
the interpretation of abundance maps provided by hyperspectral image unmixing and to binary image zooming.
Keywords
Sub-pixel mapping, abundance maps, hyperspectral imaging, binary image zoom, regularized inverse problem.
1
Introduction
In hyperspectral imaging, hundreds of images are acquired
of the same scene, each corresponding to a narrow range
of spectrum, sometimes at the price of a low spatial resolution. The composition analysis of the scene requires
the knowledge of the chemical pure elements (any chemical material as soil, grass, concrete...) present in the scene
along with their spectra. From this information, spectral
unmixing [1] builds abundance maps of the scene : each
pixel is assigned with the proportion of each pure element.
These abundance maps are very informative but in the case
where the pure elements are physically separated in the
scene (no transparency) and where the spatial resolution
is low, the precise location of each pure element inside a
pixel remains unknown and is interesting to guess from the
abundances. Furthermore, if the precise location of each
element is well estimated, the visualization of the abundance can be done through a unique image, instead of many
images (the number of pure elements) as it is usually done.
In this paper, we propose a method that estimates a label
image from its abundance maps on disjointed blocks. In
Section 2, we show that the operator that computes the
abundances of a label image on its blocks is highly noninvertible, and propose to regularize the associated inverse
problem by selecting the most regular image (the image
that minimizes the contours length) obeying to the data
constraint. In Section 3, we propose a simple simulated annealing algorithm able to solve the minimization problem.
In Section 4, we apply the method to spectral abundance
interpretation and to binary image reconstruction from a
zoomed-out version, before concluding.
2
The variational model
We aim at recovering a label image from the proportion
of its labels on each block. We focus on images defined
on a finite set Ω divided into a partition of NB blocks
(Bn )0≤n≤NB −1 , with values in a finite set of labels L =
{0, . . . , L − 1}. For each image u ∈ LΩ and each 0 ≤ n ≤
n
) where each
NB −1, we consider the L-uple (θ0n , . . . , θL−1
n
θl = #{x ∈ Bn : u(x) = l}/#Bn is the proportion of
label l in the block Bn of u. We denote by A : LΩ →
(∆L−1 )NB the operator that maps an image u to these NB
elements of the (L − 1)-dimensional simplex. The aim is to
invert A, but A is highly non-injective because many label
configurations on a block can have the same quantity of
each label (for each block, this number of configurations
is the multinomial coefficient #Bn !/(k0 ! . . . kL−1 !) with
kl = θl #Bn ). Among all these configurations we want to
favor those with short contours, measured by the Potts potential
X
∀u ∈ LΩ , E(u) =
1u(x)6=u(y) ,
x∼y
where ∼ is a neighborhood system, typically the 4-th or
the 8-th order neighborhood system, with specific boundary conditions. Finally, given a set v of abundance maps,
we propose to solve the constrained minimization problem
minimize E(u) among all u such that Au = v.
(P)
The analysis of Problem (P) is difficult. Firstly, the existence of a solution is guaranteed, but the uniqueness is not.
Indeed, if L = 2 (binary image) and Ω = {0, . . . , 2NB −1}
with Bn = {2n, 2n + 1}, if v contains only (0.5, 0.5) proportions of the labels, then (P) has 2 solutions (the one beginning with 0, the other beginning with 1). Secondly, the
solutions are very sensitive to the boundary conditions of
the neighborhood system. For example, the previous problem with NB odd and periodic boundary conditions has
many more than 2 solutions. Here all solutions to the problem will be considered acceptable.
But the formulation of Problem (P) raises interesting theoretical questions. Indeed, it is straightforward to generalize
it to a continuous domain Ω. In this case, the analysis of its
solutions (e.g. the regularity of the contours, the angles at
triple points, the number of connected components, symmetry breakings, ...) is far beyond the scope of this paper
but is probably linked to the Mumford-Shah model [3].
3
Simulated annealing algorithm
Problem (P) is a combinatorial problem that we solve using
a simulated annealing algorithm. The solution is initialized
with any image u0 in the constraint, i.e. satisfying Au0 =
v. For example, if vn = (θl )0≤l≤L−1 and if the pixels of
Bn are numbered, u0 can be assigned label 0 on the k0 first
pixels of Bn , label 1 on the k1 following pixels, ..., label
L − 1 on the kL−1 last pixels, or any permutation of this
order, where for each l, kl = #Bn θl .
From the image uN at iteration N , we propose a new image
ũ by choosing a random (uniform distribution) block in
uN , and 2 random pixels inside this block, and exchanging their labels. This proposal image has the advantage of
being easy to implement and of remaining in the constraint.
Furthermore, the underlying graph structure is connected,
as any permutation is a composition of transpositions. If
∆E := E(ũ) − E(uN ) < 0, the proposal image is accepted, i.e. uN +1 is set to ũ. Otherwise, it is accepted
with probability exp(−∆E/T N ), and rejected otherwise
(and uN +1 = uN ). Here the temperature cooling scheme
(T N )N ∈N is slowly decreasing toward 0. The theory of simulated annealing states that taking (T N )N ∈N sufficiently
slow allows the sequence uN to converge in distribution to
a uniform distribution on the solutions of (P).
4
Experiments and discussion
Application 1. Four small abundance maps were synthesized such that the sum of the superposed maps was a
constant image equal to 1. Each pixel of abundance maps
is considered as a z × z block (in our experiment z = 32)
and Figure 1 shows how the method described above organizes each block by distributing the elements in the z × z
pixels in the most grouped possible way. The initialization
was a random image in the constraint, the temperature was
T N = 10 × 0.999(N/#Ω) , and weighted 8-order neighborhoods were chosen along with periodic boundary conditions.
Application 2. Suppose a binary (0-1) image has been
zoomed out using a square block averaging procedure [2].
The obtained image is gray-valued and each gray level
equals the proportion of white in the corresponding block.
Our method is able to estimate the original image from its
4 abundance maps
result
F IGURE 1 – An interpretation of 4 abundance maps. The
black in the right image represents locations of element 0
(top-left abundance map) ; dark gray represents element 1
(top-right abundance map) ; and so on. The result is regular and contains some isolated grains. It represents a most
likely ground composition given the abundance maps.
data
initialization
result
F IGURE 2 – Reconstruction of a binary image from a
zoomed-out version (16 × 16 zoom).
zoomed-out version. Figure 2 illustrates this on a 16 × 16
zoom of a letter “a” with T N = 10 × 0.99(N/#Ω) .
Discussion. If the algorithm is intrinsically slow and
would deserve deeper insight, on the other hand the model appears to be interesting both from a practical and a
theoretical viewpoint. A penalized version would be useful
to be more robust to noise in abundance data.
Acknowledgement
The author wants to thank Laurent Delsol (Université d’Orléans) for their fruitful discussions.
Références
[1] J. M. Bioucas-Dias, A. Plaza, N. Dobigeon, M. Parente, Q. Du, P. Gader and J. Chanussot. Hyperspectral unmixing overview : Geometrical, statistical, and
sparse regression-based approaches. IEEE J. Selected
Topics in Applied Earth Observations and Remote Sensing, 5(2) : 354-379, 2012.
[2] F. Guichard and F. Malgouyres, Total variation based
interpolation. Proc. of Eusipco, 3 : 1741-1744, 1998.
[3] D. Mumford and J. Shah, Optimal approximations by
piecewise smooth functions and associated variational
problems, Commun. Pure Appl. Math., 42(4), 1989.