Snapshot spectral imaging using image replication
and polarising interferometry
A. S. Gorman
A. R. Harvey*
Engineering and Physical Sciences
Heriot Watt University
Edinburgh, UK
*a.r.harvey@hw.ac.uk
Abstract. The Image Replicating Imaging Spectrometer (IRIS) is a novel, two
dimensional snapshot spectral imaging technique suitable for use at visible
and infrared wavelengths. This paper outlines the design principles of IRIS
systems. The potential difficulties due to chromatic dispersion are discussed
as well as potential solutions. The technique used to register the spectral
images is also described briefly. Finally the results of preliminary experiments
for using IRIS to spectrally unmix fluorophore signatures are presented.
1. Introduction
Spectral imaging is a tool that can be used to discriminate and identify different
materials by detecting specific spectral signatures within an image. The majority of
spectral imaging techniques require some form of temporal scanning to reconstruct
the object spectra1. This can be a problem for scenes in which the spatial and/or
spectral information is changing.
IRIS has been developed to satisfy the need for 2-dimensional, snapshot spectral
imaging. IRIS is compact, has no moving parts and offers near 100% optical
efficiency in polarised light. The technique also has the advantage of measuring the
spectral information directly, whereas the only previously reported 2-dimensional
snapshot technique2 requires computationally intensive data inversion to reconstruct
the spectral data cube.
2. Concept
IRIS3,4 employs several beam-splitting spectral filters (BSSF). to simultaneously
replicate the input image and to spectrally filter. The principle of operation of this filter
is similar to a Lyot filter5 which consists of N+1 co-aligned polarisers separated by N
co-aligned retarders, with the optic axis of the retarders at 45° to the transmission
axis of the polarisers. Each polariser-retarder-polariser combination has
transmission,
T ( )  cos2 
(2.1)
where,
       no ( )  ne ( ) d 
(2.2)
and no ( ) and ne ( ) are the ordinary and extraordinary refractive indices of the
retarder material at wavelength,  , and d is the thickness of the retarder. The
transmission of a Lyot filter with N retarders is therefore
(2.3)
T ( )  cos2 1  cos2  2  cos2 3  cos2 N 
Like all interferometric techniques, the pass-band is a slow function of angle of
incidence.
The BSSF within IRIS is a straightforward extension of the Lyot filter; Wollaston
prisms are used in place of the linear polarisers following the first retarder (Figure 1).
Now the complimentary sin2  component is also transmitted so that a BSSF with N
retarders produces 2N beams with passbands given by the 2N distinct products of
the cos2  n  and sin2  n  terms. For example with N=2, the transmissions for 4
beams are.
T1( )  cos2 1  cos2  2 
T2 ( )  sin2 1  cos2  2 
T3 ( )  cos2 1  sin2  2 
(2.4)
T4 ( )  sin2 1  sin2  2 
For the Lyot filter, the thicknesses of consecutive retarders are related by a factor
of two, giving a single sharp spectral transmission peak. The choice of retarder
thicknesses for the BSSF is more complicated, they are selected by minimising some
suitable metric such as the orthogonality of the bands.
3. IRIS design
IRIS consists of the BSSF and several additional components which together act to
create a tiled array of sub-images at the detector plane. The 2 N sub images are

Ny
arranged 2Nx  2
 at the detector where N
x
( Ny ) is the number of times the beam
is split in the x-z (y-z) plane. In each plane the splitting angles double consecutively.
A schematic of the optical arrangement is shown in Figure 2.
An image of the scene is formed by lens L1 at the plane of the field stop of size S.
Lens L 2 collimates those rays passed by field stop which then pass through the
BSSF before being brought to focus at the focal plane array (FPA) by L 3 .
It is desirable that as much of the FPA area is used as possible; from Figure 4 and
Figure 4 the following approximate relations are evident.
x
y
(3.1)
f3 

4 tan   x  4 tan  y
 
f3
x
y
 N
 N
x
y
f2 2  Sx 2  Sy
(3.2)
The field-of-view of the system is then controlled by altering f1 . These equations,
together with the requirements that f-numbers be matched through the system and
that vignetting should not occur, facilitate the first-order design of IRIS systems. This
first-order design is then refined using birefringent ray-tracing software.
To control stray light, the field stop and system housing is blackened. To stop
unwanted light entering at L3 a blackened aperture stop is used at the exit of the
BSSF. In the thermal infrared these objectives become more difficult as blackened
surfaces are emissive and must be cooled. The need to cool the housing can be
removed by ensuring the system is designed so that all radiation reaching the
detector originates from plane of the field stop. Unwanted radiation is then controlled
by cooling the field stop.
4. Dispersion
One additional concern is spectral dispersion of the birefringence of the Wollaston
prism material, which causes smearing of the spectral images. This smear maps a
point in the object space to a line at the focal plane; the intensity of this line is
modulated by the appropriate passband and the object spectra. For small angles the
spatial extent of the smear due to a single Wollaston prism is approximately equal to,
(4.1)
d  f3 Bmax  Bmin tan
where Bmax  Bmin is the maximum difference in birefringence across the spectral
band, and  is the wedge angle of the prism. This highlights the need for careful
choice of prism material although the problem will be less severe for systems with a
narrow spectral bandwidth.
A more accurate estimate of the dispersion can be found by tracing rays through
the system. This has been carried out for an eight channel laboratory system with
splitting angles of 2.48° and 4.97° in the x-z plane and 3.87° in the y-z plane. The
Wollaston prisms are calcite and the retarders are quartz of thicknesses 121.5μm,
176μm and 351μm. The FPA is 1280x1024 with 6.7μm square pixels, from (3.1)
f3  50mm . The result of the ray-trace for one of the four cornermost sub-images for
an initially on-axis ray is shown in Figure 6, the degree of dispersion between 500nm
and 600nm is 14.8 pixels.
To validate this model the spectral smear of the laboratory system has been
measured using a liquid crystal tuneable filter. A corner has been chosen as the
reference point. The result is shown in Figure 6 and at 15.7 pixels is in close
agreement with the predicted value.
Although this smear can be avoided for narrow bandwidth systems, it is the
authors' belief that the spectral smear can be corrected or greatly reduced for
broadband systems by the use of inverse filtering techniques. Alternatively the effects
of dispersion can be alleviated using an array of narrow band filters at the image
plane or compensatory dispersive elements before the image plane.
5. Registration
Although IRIS is a direct snapshot technique the data-cube must still be constructed
by registering the sub-images. Two factors cause the sub-images to be warped
relative to one another: geometric distortion introduced by L3 and geometric
distortions introduced by the BSSF. Distortion introduced by the BSSF is minimal and
so the effects of L3 dominate. The lens distortions are radially symmetric about the
optical centre of the image and can be described by a polynomial in the scalar
distance from this point. Such distortion will cause straight lines to become curved in
a manner which depends on their position in the image.
The nature of the distortion means that once the correct transformations have
been determined they can be applied to any other image captured with that particular
IRIS. The registration can therefore be regarded as a one time calibration.
In practice the transformations are best found by imaging a panchromatic scene.
The sub-images for this scene are then cropped and a target image, T, is chosen
from among them. Each remaining sub-image, R, is in turn registered with this target
image by searching the polynomial transformation space to maximise the mutual
information, I, of the images. The search is performed using the simplex algorithm.
p T (i , j ), R(i , j )
I (T , R ) 
p T (i , j ), R(i , j )  log
(5.1)
p T (i , j ) p  R(i , j )
i
j

The registration accuracy obtained using this method is better than one tenth of a
pixel
6. Example application: IRIS for fluorophore separation
IRIS is currently being trialled for identifying and spectrally unmixing the emission of
fluorophores used to label biological samples. Shown here are the results of one
such trial in which IRIS is used for fluorescence resonance energy transfer (FRET)
measurements.
FRET is a mechanism by which the emission from one fluorophore (the donor)
excites another (the acceptor). For FRET to occur, the emission spectra of the donor
must overlap with the absorption spectra of the acceptor, the fluorophores also have
to be in close proximity. FRET measurements can thus be used to determine the
separation and relative orientation of molecules.
In this trial the sample is labelled with two dyes; Cerulean (donor) and Venus
(acceptor), Figure 7 shows the emission spectra of these dyes. The Cerulean is
excited using the appropriate source and the sample imaged using IRIS and a Nikon
Eclipse TE2000-E microscope. The integration of the two components is achieved by
locating the input field stop of IRIS in the image plane of the microscope. The bands
for the IRIS system are shown in Figure 8, these have not been optimised for this
task although a preliminary investigation suggested their suitability for this task.
Figure 9 shows the registered images for the FRET measurements. The goal is to
determine the mixing proportions of the dyes within each pixel. This can be achieved
using a least squares solution if the instrument response to the individual dyes is
known. In the context of unmixing, these responses are known as endmembers,
Figure 10 shows the theoretical and measured endmember for Cerulean.
Unfortunately the response for Venus could not be accurately measured due to
background light pollution. The unmixing is instead carried out using the measured
endmember for Cerulean and the theoretical endmember for Venus.
Figure 11 shows the results of an unconstrained least squares solution to the
unmixing problem, it appears that each pixel consists of an almost equal mixture of
the endmembers, although the Cerulean dominates. Also shown is the so called dark
endmember which accounts for shadowing, this is minimal. The next step in this
treatment is to gain accurate values for the endmembers and to compare the results
with a physical model of the emission to yield a value for the quantity of interest.
7. Conclusion
In this paper the performance and design of IRIS has been described. The
unorthodox transmission functions exhibited by IRIS mean that optimisation in design
of the BSSF is required. To use the BSSF for spectral imaging several additional
components are necessary. The relationships between the various system
parameters have been stated, allowing first-order design of IRIS systems. The
birefringent materials used must be carefully selected to maximise transmission of
the system while ensuring dispersion within the BSSF is minimal. Possible methods
for correcting this dispersion have been highlighted. Steps to take to minimise stray
light and maximise the signal-to-noise ratio have also been given.
The nature of the imaging distortions have been described as well as the
technique used to register the spectral images. Preliminary results of using IRIS to
spectrally unmix fluorophore signatures have been presented and show that IRIS is
capable of this task.
Future work includes an investigation into reducing or correcting the spectral
smear and further field trials for a number of applications including visible-light and
thermal-infrared surveillance, fluorescence lifetime imaging and retinal imaging for
medical purposes.
(a)
(b)
Figure 1: Two possible arrangements for a 4 channel beam splitting
spectral filter.
(a) With Nx=2 and Ny=0
(b) With Nx=1 and Ny=1
Figure 2: Schematic of the IRIS optical system
Figure 4: Relationship between
maximum splitting angle, FPA
size and f3
Figure 4: Ratio f3/f2 determined
by size of field stop and subimages.
Figure 6: Theoretical dispersion
Figure 6: Measured dispersion
Figure 7: Emission spectra of the dyes Cerulean and Venus
Band 1
Band 2
Band 3
Band 4
1
1
1
1
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0
0.45 0.5 0.55
Wavelength (nm)
0
0.45 0.5 0.55
Wavelength (nm)
Band 5
0
0.45 0.5 0.55
Wavelength (nm)
Band 6
0
0.45 0.5 0.55
Wavelength (nm)
Band 7
Band 8
1
1
1
1
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0
0.45 0.5 0.55
Wavelength (nm)
0
0.45 0.5 0.55
Wavelength (nm)
0
0.45 0.5 0.55
Wavelength (nm)
0
0.45 0.5 0.55
Wavelength (nm)
Figure 8: Transmission of IRIS used for fluorophore separation
ceve88: band1
ceve88: band2
ceve88: band3
200
80
150
60
100
40
50
40
40
30
20
20
50
20
0
0
ceve88: band5
ceve88: band4
60
ceve88: band6
40
10
0
0
ceve88: band7
ceve88: band8
30
60
20
40
10
20
0
0
100
30
20
50
10
0
0
Figure 9: FRET image of labelled with Cerulean (donor) and Venus
(acceptor)
Normalised response of IRIS to dyes
0.7
EMcerul - theory
mc18 - measured
0.6
Normalised emission
0.5
0.4
0.3
0.2
0.1
0
1
2
3
4
5
6
7
8
Band
Figure 10: Normalised theoretical and measured response of IRIS to the
Cerulean fluorophore.
Cerulean
Venus
0.5
0.6
0.4
0.5
0.3
0.4
0.3
0.2
0.2
0.1
0.1
0
Shadow
0.08
0.06
0.04
0.02
0
Figure 11: Results of unmixing the normalised spectra in Figure 9.
[1] A. R. Harvey, J. E. Beale, A. H. Greenaway, T. J. Hanlon, John W. Williams,
Technology Options for Hyperspectral Imaging Proc SPIE 4132, p13-24, Imaging
Spectrometry VI; Michael R. Descour, Sylvia S. Shen; Eds (2000)
[2] M. R. Descour, C. E. Volin, E. L. Dereniak, K. J. Thome, A. B. Schumacher, D. W.
Wilson, and P. D. Maker, "Demonstration of a high-speed nonscanning imaging
spectrometer ," Opt. Lett. 22, 1271-1273 (1997)
[3] A R Harvey, D W Fletcher-Holmes, Imaging Spectrometer, Patent WO 03/089890,
30 October 2003
[4] Andrew R. Harvey, David W. Fletcher-Holmes, Alistair Gorman, Kirsten
Altenbach, Jochen Arlt, Nick D. Read, Spectral imaging in a snapshot, Spectral
Imaging: Instrumentation, Applications, and Analysis III, Proc. SPIE Vol. 5694, p.
110-119, (2005)
[5] B. Lyot, Filter monochromatique polarisant et ses applications en physique
solaire, Ann. Astrophys. 7, 32 (1944)