Beating Noise
Observational Techniques
ASTR 3010
Lecture 11
Textbook
From previous classes
• We learned that in the astronomical data pre-processing,
bias subtraction,
dark subtraction,
flat-fielding,
bad pixel correction,
cosmic ray hits removal
are important.
• There are several observation techniques that can take care of the above
correction nearly automatically.
Subtraction of two images taken with a small offset
• image1 – image2
-
• bad pixels, bias, and dark are removed!
• But, cosmic rays remain.
• Need to apply flat-fielding
=
?Dither Observation?
• Dither Observation
Series of images taken with telescope offsets.
box5
box4
Dithering Observation
Detector Array
Observation No 1
dead pixels
hot pixels
cosmic rays
Dithering Observation
Observation No 2
dead pixels
hot pixels
cosmic rays
Dithering Observation
Observation No 3
dead pixels
hot pixels
cosmic rays
Dithering Observation
Observation No 4
dead pixels
hot pixels
cosmic rays
Dithering Observation
Observation No 4
dead pixels
hot pixels
cosmic rays
Dithering Observation
• Five observations
obs1 …
obs5
flux3
True Flux
flux1
flux5
flux2
flux4
Dithering Observation
• Five observations
obs1 …
obs5
Ordered list
flux4 < flux5 < flux2 < flux1 < flux3
median
value
This will take care of bias, dark, cosmic ray hits, bad pixels!
Dozens of Dithered images with small offsets
• SuperResolution
With the P&A telescope
Not Yet!
P&A Dithering by Hand
Real example : Keck NIRC2 AO observation of HIP88945
Shift and Add
look at those hot pixels!!
# demonstrating the Shift and Add concept.
#
# import libraries
from ds9 import *
from numpy import *
import pyfits
im1=pyfits.getdata('n0002.fits',ignore_missing_end=True)
im2=pyfits.getdata('n0003.fits',ignore_missing_end=True)
im3=pyfits.getdata('n0004.fits',ignore_missing_end=True)
im4=pyfits.getdata('n0005.fits',ignore_missing_end=True)
im5=pyfits.getdata('n0006.fits',ignore_missing_end=True)
# create a big enough array to cover any shifted images...
BIG = zeros((1201,1201))
# Then, we will find the positions of the star in each image.
# Shift the image in the BIG array such that the star appears
# at the center of the array (601,601)
posx_im1=276
posy_im1=279
BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im1
posx_im1=397
posy_im1=162
BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im2
posx_im1=155
posy_im1=403
BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im3
posx_im1=397
posy_im1=401
BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im4
posx_im1=154
posy_im1=161
BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im5
d=ds9()
d.set_np2arr( transpose(BIG) )
Shift and Median is better!!
Python HW2:
Create a Python script doing this!
Median sky subtraction
• When your object is
extended… or you need to
make a pretty picture…
• in near-IR: useful for 10-20
min depends on the quality of
the night.
In Astronomical observations…
• Three major noise sources…
o Poisson noise (aka “shot noise”)
o Sky noise
o Electronic noise
• Sky noise limited observation
• Dark current limited observation
• Readout noise limited observation(?)
Noise versus signal at various wavelengths
109
Signal Strength
Extremely bright objects
Sky noise
105
Detector noise
101
Optical
near IR
mid IR
Coadds
• Because of the increased sky background, frames will get saturated in short
exposures.
variable sky is the limiting factor  sky limited observations.
• Take as many unsaturated frames as necessary and combined them later.
too many files for a single object (several hundreds)
on-chip combine of multiple exposures = “coadd”
Near-IR observation Tips
• Need to sample the sky
o through dithering
o or from a dedicated sky observation in the nearby field  needs to be dithered
also. why?
• Choose a right number of coadds.
o Choose the maximum frame exposure time which will stay in the linear regime.
o Then, from the required total exposure time, calculate the necessary number of
coadds.
• Sky level is varying faster at longer wavelengths!
o at K-band (2.2 μm) : up to 20 minutes
o at L-band (3.6 μm) : max is about a minute
o at mid-IR (10 μm) : fraction of a second  different strategy
• Dither
• Repeat the dither sequence as many times as necessary.
Observation Planning : near-IR
• You will be observing at the K-band (2.2 micron), and you expect that your
source would have a count rate of about 1,000 counts per second where
this signal will be spread over ~10 pixels due to the PSF structure. Some
other relevant data are
o Detector linear regime: count < 10,000 counts
o Sky brightness at K = 100 counts per second per pixel
o Readout noise = 10 counts per readout
o Dark current = 25 counts per second per pixel
What would be the best observing strategy for this object if you want your final
S/N~100?
Observation Planning : near-IR
• Source ~ 2,000 count per second over 4 pixels. Goal S/N >= 100
o
o
o
o
Detector linear regime: count < 8,000 counts
Sky brightness at K = 1,000 counts per second per pixel
Readout noise = 10 counts per readout
Dark current = 100 counts per second per pixel
 Per pixel per second signal = 500 source count + 1,000 sky count + 100 dark
current = 1,600 counts/pxl/sec  In 5 sec exposure, the source will get into a
non-linear regime  max exposure time should be less than 5sec or we should
“coadd”
 In a single 5 second exposure : S/N = 500 / sqrt(1600)~12  if we coadd 16
frames, then a single exposure will be 40 second long and S/N=12*sqrt(16)=48
 For a box5 dither observation pattern, we will get five images  total readout
noise = 10 * sqrt(5) = 22.4 counts. This is negligible to shot noises from
source+sky+detector.
 If we take one image (40 sec long with 8 coadds) using a box5 pattern  final
S/N=48 * sqrt(5) = 107
 The total duration of the observation is 5 x 40sec + total overhead (~1 min) = 4
min. At K-band, the sky will be invariable for up to ~20 min. So, this will be OK.
Nod & Shuffle
• "Nod and Shuffle" is a technique used to obtain very faint spectra
• the natural glow of the nighttime sky would overwhelm the extremely faint
objects (e.g., far, far away galaxies)
• N&S allows astronomers to effectively subtract away the bright spectral
emission lines and fainter continuum of our atmosphere's nighttime glow while
retaining the faint spectra of dim, red galaxies. This also significantly reduces
readout noise.
1. obtaining a spectrum of an object
2. moving the electrical charge built up by the image of the spectrum on the
CCD to another location "buffer storage" on the CCD (Shuffling)
3. shifting the position of the telescope slightly (Nodding) so that the spectrum
of the target object shifts to a different part of the CCD
4. obtaining another spectrum in the nodded position
5. shuffling the charges moved in step 1 back to the original position (while
moving the charges from step 4 to the buffer storage)
6. nodding the telescope back to the position in step 1
7. repeating until enough light is collected.
Sky cancellation: ‘nod and shuffle’
Storage of ‘sky’ image next to object image via ‘charge shuffling’
Zero extra noise introduced, rapid switching (60s)
A
B
A-B
Typically A=60s/15 cy: 1800s exposure10-3 subtraction
Another example
Weird image artifacts
Latent images : badly saturated pixels do not come back to normal right away!
Weird image artifacts
ghost images : multiple reflections within an optical element (e.g., filter, beam
splitter, entrance pupil) seen near a very bright star
Handling various problems at IR
Non-uniform QE
Flat-field
High background level
• Point sources
• Extended sources
Dithering and coadds
• Small dither offset
• Off-chip dither
Non-linearity of detectors
Stay within linear range
Memory effects (latent)
Do not saturate
Bad pixels
Mask or median out from dithers
Cosmic rays
Shift and add
Dark current
Subtracts out with sky
Hot rows, hot pixels, amplifier
glow
Subtracts out well
In summary…
Important Concepts
Important Terms
• Various noise sources at different
wavelengths and their relation to
the observing techniques.
• Median combining of images
•
•
•
•
•
Dither observation
Shift and add
Shift and median
Coadd
Nod-and-shuffle
Chapter/sections covered in this lecture : 9