Outline
• AO Imaging
• Constrained Blind Deconvolution
• Algorithm
• Application
- Quantitative measurements
• Future Directions
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
1
References
S.M. Jefferies & J.C. Christou, “Restoration of astronomical images by iterative blind deconvolution”, Astrophys. J.,
415, 862-874, 1993.
E. Thiébaut & J.-M. Conan, “Strict a priori constraints for maximum-likelihood blind deconvolution”, J. Opt. Soc.
Am., A, 12, 485-492, 1995.
J.-M. Conan, L.M. Mugnier, T. Fusco,, V. Micheau & G. Rousset, “Myopic deconvolution of adaptive optics images
by use of object and point-spread-function power spectra”, App. Optics, 37, 4614-4622, 1998.
B.D. Jeffs & J.C. Christou, “Blind Baysian Restoration of Adaptive Optics images using generalized Gaussian
Markov random field models”, Adaptive Optical System Technologies, D. Bonacinni & R.K. Tyson, Ed., Proc. SPIE,
3353, 1998.
E.K. Hege, J.C. Christou, S.M. Jefferies & M. Chesalka, “Technique for combining interferometric images”, J. Opt.
Soc. Am. A, 16, 1745-1750, 1999.
T. Fusco, J.-P. Véran, J.-M. Conan, & L.M. Mugnier, ”Myopic deconvolution method for adaptive optics images of
stellar fields”, Astron. Astrophys. Suppl. Ser., 134, 193-200, 1999.
J.C. Christou, D. Bonaccini, N. Ageorges, & F. Marchis, “Myopic Deconvolution of Adaptive Optics Images”, ESO
Messenger, 1999.
T. Fusco, J.-M. Conan, L.M. Mugnier, V. Micheau, & G. Rousset, “Characterization of adaptive optics point spread
function for anisoplanatic imaging. Application to stellar field deconvolution.”, Astron. Astrophys. Suppl. Ser., 142,
149-156, 2000.
E. Diolaiti, O. Bendinelli, D. Bonaccini, L. Close, D. Currie, & G. Parmeggiani, “Analysis of isoplanatic high
resolution stellar fields by the StarFinder code”, Astron. Astrophys. Suppl. Ser., 147, 335-346 , 2000.
S.M. Jefferies, M. Lloyd-Hart, E.K. Hege & J. Georges, “Sensing wave-front amplitude and phase with phase
diversity”, Appl. Optics, 41, 2095-2102, 2002.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
2
Adaptive Optics Imaging
Adaptive Optics systems do NOT produce
perfect images (poor compensation)
Seeing disc
Halo
Core
Artifacts
Binary Star
components
Without AO
IPAM 2004 January 28
With AO
Mathematical Challenges in Astronomical Imaging
3
Adaptive Optics Imaging
• Quality of compensation depends upon:
– Wavefront sensor
– Signal strength & signal stability
– Speckle noise
- d / r0
– Duty cycle
- t / t0
– Sensing & observing - λ
– Wavefront reconstructor & geometry
– Object extent
– Anisoplanatism (off-axis)
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
4
Adaptive Optics: PSF Variability
• Science Target and Reference Star typically observed at
different times and under different conditions.
• Differences in Target & Reference compensation due to:
- Temporal variability of atmosphere(changing r0 & t0).
- Object dependency (extent and brightness) affecting centroid
measurements on the wavefront sensor (SNR).
- Full & sub-aperture tilt measurements
- Spatial variability (anisoplanatism)
• In general: Adaptive Optics PSFs are poorly determined.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
5
Why Deconvolution and PSF Calibration?
• Better looking image
• Improved identification
Reduces overlap of image structure to more easily identify features in
the image (needs high SNR)
• PSF calibration
Removes artifacts in the image due to the point spread function (PSF)
of the system, i.e. extended halos, lumpy Airy rings etc.
• Improved Quantitative Analysis
e.g. PSF fitting in crowded fields.
• Higher resolution
In specific cases depending upon algorithms and SNR
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
6
The Imaging Equation
Shift invariant imaging equation
g(r) = f(r) * h(r) + n(r)
(Image Domain)
G(f) = F(f) • H(f) + N(f)
(Fourier Domain)
g(r) – Measurement
h(r) – Point Spread Function (PSF)
f(r) – Target
n(r) – Contamination - Noise
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
7
Deconvolution
• Invert the shift invariant imaging equation
i.e. solve for f(r)
INVERSE PROBLEM
given both g(r) and h(r).
- But h(r) is generally poorly determined.
- Need to solve for f(r) and improve the h(r) estimate simultaneously.
Unknown PSF information
Some PSF information
Blind (Myopic) Deconvolution
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
8
Blind Deconvolution
Solve for both object & PSF
g(r) = f(r) * h(r) + n(r)
contamination
Measurement
unknown
object irradiance
unknown or poorly
known PSF
Single measurement:
Under – determined - 1 measurement, 2 unknowns
Never really “blind”
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
9
Blind Deconvolution – Physical Constraints
• How to minimize the search space for a solution?
• Uses Physical Constraints.
– f(r) & h(r) are positive, real & have finite support.
– h(r) is band-limited – symmetry breaking
prevents the simple solution of h(r) = (r)
• a priori information - further symmetry breaking (a * b = b * a)
– Prior knowledge (Physical Constraints)
– PSF knowledge: band-limit, known pupil, statistical derived PSF
– Object & PSF parameterization: multiple star systems
– Noise statistics
– Multiple Frames: (MFBD)
• Same object, different PSFs.
• N measurements, N+1 unknowns.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
10
Multiple Frame Constraints
Multiple Observations of a common object
g1 (r )  f (r )  h1 (r )
g 2 (r )  f (r )  h2 (r )



g n (r )  f (r )  hn (r )
• Reduces the ratio of unknown to measurements from 2:1
to n+1:n
• The greater the diversity of h(r),the easier the separation
of the PSF and object.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
11
An MFBD Algorithm
• Uses a Conjugate Gradient Error Metric Minimization scheme
- Least squares fit.
• Error Metric – minimizing the residuals (convolution error):

~ ~
2
E   gik  g~ik   gik  f i  hik
ik
 r
2
ik
2
ik
ik
• Alternative error metric – minimizing the residual autocorrelation:
E   rik  rik
ik
2
Autocorrelation of residuals
Reduces correlation in the residuals
(minimizes “print through”)
So not sum over the 0 location.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
12
An MFBD Algorithm
• Object non-negativity
~
2
f


Reparameterize the object as the square of another variable i
i HARD
or penalize the object against negativity.
EObj 

~
u f i  0
~2
fi
SOFT
• PSF Constraints (when pupil is not known)
- Non-negativity
~
Reparameterize - hi , k   i2,k
- Band-limit
IPAM 2004 January 28
Ebl 

k ,u  u c
~
H k ,u
or penalize –
EPSF 

~
uhi ,k  0
~
hi ,k
2
2
Mathematical Challenges in Astronomical Imaging
13
PSF Constraints
Use as much prior knowledge of the PSF as possible.
Transfer function is band-limited
MTF
fc = D/
MTF
•
Normalized Spatial Frequency
•
PSF is positive and real
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
14
An MFBD Algorithm
• PSF Constraints (Using the Pupil)
- Parameterize the PSF as the power spectrum of the
complex wavefront at the pupil, i.e.
~

hik  a~ik a~ik
where
  2 iv 

a~ik  Wv exp  j 


vk 

v
  N 

PSF
Pupil
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
15
PSF Constraints
•
PSF Constraints (Using the Pupil)
- Modally - express the phases as either a set of Zernike modes of order M
M
 vk   qm Z vk
m 1
  v 2 
  which
- or zonally as  vk    vk where   exp  
2

 
 
enforces spatial correlation of the phases.
• Phases can also be constrained by statistical knowledge of the AO system
performance.
• Wavefront amplitudes can be set to unity or can be solved for as an
unknown especially in the presence of scintillation.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
16
Object Constraints
•
In an incoherent imaging system, the object is also real and positive.
•
The object is not band-limited and can be reconstructed on a pixel-by-pixel basis –
leads to super-resolution (recovery of power beyond spatial frequency cut-off).
•
Limit resolution (and pixel-by-pixel variation) by applying a smoothing operator in the
reconstruction.
•
Parametric information about the object structure can be used (Model Fitting):
- Multiple point source
f v  m   v
- Planetary type-object (elliptical uniform disk)
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
17
Object Constraints
Local Gradient across the object defines the object texture
(Generalized Gauss-Markov Random Field Model), i.e.
| fi – fj | p where p is the shape parameter.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
18
Object Constraints
GGMRF
example
IPAM 2004 January 28
truth
raw
over
under
Mathematical Challenges in Astronomical Imaging
19
Object Prior Information
• Planetary/hard-edged objects (avoids ringing)
Use of the finite-difference gradients f(r) to generate an extra error
term which preserves hard edges in f(r).
 &  are adjustable parameters.
EFD
IPAM 2004 January 28
 f r 
 f r  

 
 ln 1 
 
r  

Mathematical Challenges in Astronomical Imaging
20
An MFBD Algorithm
• Myopic Deconvolution (using known PSF information)
- Penalize PSFs for departure from a “typical” PSF or model
(good for multi-frame measurements)
ESAA   h
SAA
ik
~ SAA 2
 hik
ik
- Penalize PSF on power spectral density (PSD)
EPSD
~
 H H
i
i


 PSD H
i

2




where the PSD is based upon the atmospheric conditions and AO
correction.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
21
An MFBD Algorithm
• Further Constraints
– Truncated Iterations (Tikenhov)

~ ~
Econv   gik  f i  hik

ik
2


~ ~
  gˆ ik  f i  hik  ni ,k
2
  nik
ik
2
 k n2
ik
– Support Constraints
In many cases, a limited field is available and it is important to compute
the error metric only over a specific region M of the observation space, i.e.
2
E   gik  g~ik M ik
ik
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
22
idac – iterative deconvolution algorithm in c
• SNR Regularization (Fourier Domain)
Minimize in the Fourier domain rather than the image domain, i.e.
Econv  uk


2
ˆ
Guk  Guk u
where
IPAM 2004 January 28
u

G

u
2
 Nu
Gu
2

2
Mathematical Challenges in Astronomical Imaging
23
An MFBD Algorithm
• Forward Modeling of Imaging Process:
gˆ ik  gik Gik   sik  nik
Noise terms
Measurement
Signal
Background (sky + dark)
Gain (flatfield)
• Compute Error Metric based on Measurement
~
E   gˆ ik  gˆ ik
2
where data is not pre-processed
ik
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
24
idac – an MFBD Algorithm
• idac is a generic physically constrained blind-deconvolution algorithm written in C
and is platform independent on UNIX systems.
• Maximum-likelihood with Gaussian statistics – error metric minimization using a
conjugate gradient algorithm.
• It can handle single or multiple observations of the same source.
• It allows masking of the observation (convolution image) permitting the saturated
regions to make no contribution to the final results for both the target and the PSF.
• It has the option to fit a the strength of a bias term in the image (sky+dark) – asik
• The algorithm can be run as with either a fixed PSF or a fixed object or both
unknown.
• idac was written by Keith Hege & Matt Chesalka (as part of a collaborative effort
with Stuart Jefferies and Julian Christou) and is made available via Steward
Observatory and the CfAO.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
25
idac – iterative deconvolution algorithm in c
• Conjugate Gradient Error Metric Minimization
E  Econv  Ebl  ESAA
– Convolution Error
– Band-limit Error
– Non-negativity

Econv  ik mik g ik 
Ebl  k u u
2
fˆi  ai
c
and

Hˆ uk

fˆi  hˆik  asik

2
2
2
hˆik  bik
– PSF Constraint (for multiple images)
ESAA  i h
saa
i
IPAM 2004 January 28
saa
 hˆi
2
Mathematical Challenges in Astronomical Imaging
26
idac Software Page
http://cfao.ucolick.org/software/idac/
http://bach.as.arizona.edu/~hege/docs/docs/IDAC27/idac_package.tar.gz
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
27
Application of idac
•
Investigation of relative photometry and astrometry in
deconvolved image.
- Gemini/Hokupa’a Galactic Center data
- PSF reconstruction
•
Application to various astronomical AO images.
-
Resolved Galactic Center sources (bow-shocks)
-
Solar imaging
-
Solar system object (Io) – comparison with “Mistral”
•
Artificial satellite imaging
•
Non-astronomical AO imaging.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
28
Application of idac
•
How well does the deconvolved image retain the photometry
and astrometry of the data?
-
It has been suggested that it is better to measure the
photometry especially from the raw data.
-
Investigated using dense crowded field data from
Gemini/Hokupa’a commisioning data.
-
Comparison of Astrometry and Photometry from these data
to that measured directly via StarFinder.
-
Comparison of both techniques to simulated data.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
29
Hokupa’a Galactic Center Imaging
Crowded Stellar Field
with partial compensation
Difficult to do photometry
and astrometry because of
overlapping PSFs
- Field Confusion
Need to identify the
sources for standard datareduction programs.
See Poster
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
30
Observed GC Field
Gemini /Hokupa’a infrared (K
with texp = 30s) observations of a
sub-field near the Galactic Center.
4 separate exposures
Note the density of stars in the
field.
FOV = 4.6 arcseconds
Reduced with idac & StarFinder
StarFinder is a semi-analytic program in IDL
which reconstructs AO PSF and synthetic fields
of very crowded images based on relative
intensity and superposition of a few bright stars
arbitrarily selected. It extracts the PSF
numerically from the crowded field and then fits
this PSF to solve for the star’s position and
intensity.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
31
Gemini Imaging of the Galactic Center Deconvolution
Initial Estimates:
Object – 4 frames co-added
PSF – K' 20 sec reference
(FWHM = 0.2")
4.8 arcsecond subfield
256 x 256 pixels
(This is a typical start for this algorithm)
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
32
Gemini Imaging of the Galactic Center Deconvolution
4 frame average
for each of the
sub-fields.
idac reductions.
FWHM = 0.07"
Note residual PSF halo
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
33
Gemini Imaging of the Galactic Center –
PSF Recovery
gˆ PSF  fˆPSF  hˆ

fˆPSF

ĝ PSF
ĥ
Frame PSF recovered by isolating individual star
from f(r) and convolving with recovered PSFs, h(r).
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
34
Gemini Imaging of the Galactic Center
•
Data Reduction Outline
1. Blind Deconvolution to obtain target & PSF
2. Estimate PSF from isolated star and h(r)
3. Fixed deconvolution using estimated PSF
4. Blind Deconvolution to relax PSF estimates
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
35
Gemini Imaging of the Galactic Center
Object Recovery
Average observation
initial idac result
fixed PSF result
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
36
Gemini Imaging of the Galactic Center
Image Sharpening
FWHM
Compensated – 0.20 arcsec
Initial - 0.07 arcsec
Final - 0.05 arcsec
Diffraction-limit
α = 0.06 arcsec
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
37
Observed GC Field Reconstructions
The BD reconstruction solves
for the common object from
all four observed frames.
Reconstructed star field distributions
from StarFinder as applied to the four
separate observations. StarFinder is a
photometric fitting packages which
solves for a numerical PSF.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
38
Observed GC Field Reconstructions
• The fainter the point source, the
broader it is.
• Magnitude measurement depends upon
measuring area and not peak.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
39
Observed GC Field - Photometry
Common Stars
Comparison of Photometry and for the 55 common stars in the 4 frame StarFinder and IDAC
reductions. There is close agreement between the two up to 3.5 magnitudes. Then there is a
trend for the IDAC magnitudes to be fainter than the StarFinder ones. This can be explained
by the choice of the aperture size used for the photometry due to the increasing size of the
fainter sources. Even so, the rms difference between them is still  0.25 magnitudes. A more
sophisticated photometric fitting algorithm than imexamine is therefore suggested.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
40
Observed GC Field - Astrometry
Common Stars
Comparison of Astrometry and for the 55 common stars in the 4 frame StarFinder
and IDAC reductions. The x and y differences are shown by the appropriate
symbols. The dispersion of  10-14 mas is small, less than a pixel, and a factor of
four less than the size of the diffraction spot.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
41
Observed GC Field – PSF Reconstructions
Blind Deconvolution
StarFinder
Reconstructed PSFs for the four frames using IDAC (top) and StarFinder
(bottom). The PSF cores (green and white) are essentially identical with the
StarFinder PSFs generally having larger wings.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
42
Simulated GC Field Comparisons
Comparison of aperture
photometry from blind
deconvolution to true
magnitudes for the
simulated GC field.
Comparison of aperture
photometry from blind
deconvolution to StarFinder
analysis for the simulated GC
field.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
43
Observed GC Field – PSF Reconstructions
Reconstructed PSFs for the four frames using IDAC (top) and StarFinder
(bottom). The PSF cores (green and white) are essentially identical with the
StarFinder PSFs generally having larger wings.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
44
Extended Sources near the Galactic Center
IRS 10
IRS 1W
IRS 5
IRS 21
• Point sources show
strong uncompensated
halo contribution.
• Bow shock structure
is clearly seen in the
deconvolutions.
[Data from Angelle Tanner, UCLA]
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
45
Adaptive Optics Solar Imaging
Low-Order AO System
• Lack of PSF information.
AO
Deconvolved
• Sunspot and granulation
features show improved
contrast, enhancing detail
showing magnetic field
structure
[Data from Thomas Rimmele, NSO-SP]
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
46
ADONIS AO Imaging of Io
 = 3.8 m
Two distinct hemispheres
~ 11 frames/hemisphere
Co-added initial object
PSF reference as initial PSF
Surface structure visible showing
volcanoes.
(Marchis et. al., Icarus, 148, 384-396,
2000.)
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
47
Keck Imaging of Io
Why is deconvolution important? This is why …
(Data obtained by D. LeMignant & F. Marchis et al.)
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
48
Keck Imaging of Io
Why is deconvolution important? This is why …
(Data obtained by D. LeMignant & F. Marchis et al.)
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
49
Io in Eclipse
Two Different BD
Algorithms
Keck observations to
identify hot-spots.
K-Band
19 with IDAC
17 with MISTRAL
L-Band
23 with IDAC
12 with MISTRAL
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
50
Artificial Satellite Imaging
256 frames per apparition
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
51
Summary
• Blind/Myopic Deconvolution is well suited to AO imaging where the PSFs are
not well known.
• Incorporate as many physical constraints about the imaging process as possible.
• Building a specific algorithm to match the application is advantageous.
• This algorithm (idac) suffers from the same problem as others in that the PSF
get wider as the dynamic range increases ( a problem of half-wave rectification
of the noise with hard positivity constraint?)
• Aperture photometry yields good relative photometry (<0.1m for m < 4 and 
0.2m for 4.0 < m < 7.0.
• A general algorithm has limitations.Can one build a modular algorithm to
incorporate as much prior information as possible for the data?
• Deconvolution algorithms are not necessarily user-friendly. How can we do
this?
• Assumption of isoplanatism is assumed, how to incorporate anisoplanatism for
wide field imaging?
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
52
References
S.M. Jefferies & J.C. Christou, “Restoration of astronomical images by iterative blind deconvolution”, Astrophys. J.,
415, 862-874, 1993.
E. Thiébaut & J.-M. Conan, “Strict a priori constraints for maximum-likelihood blind deconvolution”, J. Opt. Soc.
Am., A, 12, 485-492, 1995.
J.-M. Conan, L.M. Mugnier, T. Fusco,, V. Micheau & G. Rousset, “Myopic deconvolution of adaptive optics images
by use of object and point-spread-function power spectra”, App. Optics, 37, 4614-4622, 1998.
B.D. Jeffs & J.C. Christou, “Blind Baysian Restoration of Adaptive Optics images using generalized Gaussian
Markov random field models”, Adaptive Optical System Technologies, D. Bonacinni & R.K. Tyson, Ed., Proc. SPIE,
3353, 1998.
E.K. Hege, J.C. Christou, S.M. Jefferies & M. Chesalka, “Technique for combining interferometric images”, J. Opt.
Soc. Am. A, 16, 1745-1750, 1999.
T. Fusco, J.-P. Véran, J.-M. Conan, & L.M. Mugnier, ”Myopic deconvolution method for adaptive optics images of
stellar fields”, Astron. Astrophys. Suppl. Ser., 134, 193-200, 1999.
J.C. Christou, D. Bonaccini, N. Ageorges, & F. Marchis, “Myopic Deconvolution of Adaptive Optics Images”, ESO
Messenger, 1999.
T. Fusco, J.-M. Conan, L.M. Mugnier, V. Micheau, & G. Rousset, “Characterization of adaptive optics point spread
function for anisoplanatic imaging. Application to stellar field deconvolution.”, Astron. Astrophys. Suppl. Ser., 142,
149-156, 2000.
E. Diolaiti, O. Bendinelli, D. Bonaccini, L. Close, D. Currie, & G. Parmeggiani, “Analysis of isoplanatic high
resolution stellar fields by the StarFinder code”, Astron. Astrophys. Suppl. Ser., 147, 335-346 , 2000.
S.M. Jefferies, M. Lloyd-Hart, E.K. Hege & J. Georges, “Sensing wave-front amplitude and phase with phase
diversity”, Appl. Optics, 41, 2095-2102, 2002.
IPAM 2004 January 28
Mathematical Challenges in Astronomical Imaging
53