The Complete
Tutorial of
CCDStack
Version 2.0
Review:
A First Look
A First Look
•
The four main desktop dialogs are the Image Manager, Adjust Display,
Image Select/Blink and the Magnification tools.
– CCDStack will remember their last positions and settings if you should close
them. Re-activate them by going under the Window menu.
•
The Auto Scale feature of the Adjust Display window adjusts the screen
stretch depending on what brightness levels are being displayed in the
current window.
– If you zoom in or out and press Auto Scale, you will get different answers. These
adjustments only affect the white and black points (grayscale range) and do not
touch the underlying data.
– Auto Scale is especially useful when used together with DDP.
– Manipulate the Include status of images in a stack in order to operate on a subset of
them. This can save time in opening and closing images unnecessarily. However, first
time users of the program should probably have open only the images they want to
process.
– Always monitor the bottom right (and left) status bar of CCDStack. Important image
information is found there.
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Philosophy
Philosophy
•
•
•
•
•
An image is a two dimensional array of pixels.
A stack of images creates an array of stacks of pixels.
o When thinking about processing the data, consider a single stack of
pixels.
Interpolation (previously “impute”) calculates new values by examining the
values of neighboring pixels.
Missing Values are cool. This graphical representation will help us later
when we investigate this clever concept.
An implicit assumption is that images in a stack are the same dimension. If
they are not initially, CCDStack will attempt to resize (resample) them
accordingly.
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Mean, Median, Mode
Mean, Median, Mode
•
•
•
•
•
The words (arithmetic) “Mean” and “Average” are synonymous in my tutorial.
Semantically “Average” can have a more general meaning that may refer to
other ways to analyze a set of values- but not here.
The Mean is the number equal to the sum of all the values in the set divided
by the total number of members in the set. For typical set (sample) sizes, the
mean will often result in the smallest uncertainty (best S/N) once statistical
outliers are identified and discarded.
- Unfortunately for typical sample sizes that are small, the mean is
strongly affected by large outliers.
The Median is the middle value of the ordered set of all values. It is therefore
insensitive to outliers and can be a good approximation for the mean. This is
especially true for large sample sizes.
For this tutorial I arbitrarily choose sets of less than 7 values to be small. 750 values would be “typical.” And a set with more than 50 values is large.
The Mode is the most common value appearing in a set. In CCDStack it is
often used to calculate sky values.
RESOURCE:
meanmedianmode.xlsx
(Documents Directory)
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
A Bit of Information
A Bit of Information
•
•
•
•
•
The most important thing to take away from this section is to remember to
always save your data as a 32-bit floating point file. Be certain to look at the
Format type *before* pressing the save button.
The “Background” value in Adjust Display is the black point. This
corresponds to a Bitmap value and grayscale brightness of 0 on the screen.
The “Maximum” value in Adjust Display is the white point. This corresponds
to a Bitmap value and grayscale brightness of 255 on the screen.
I recorded this section just a before the Histogram option became live in
Version 2. You can now find it under the Window menu.
When creating a scaled image you are permanently discarding information
by fixing the white and black points. Thus, this is a very important processing
decision.
- When doing this to create a TIF image, for example, remember to
carefully set the black point by making it a little “gray.” This will help
prevent black clipping your images.
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Biases, Darks and Flats
Processing NGC 5216
(The Beginning)
•
•
•
I had to begin with this brief section to set up the processing workflow for the
NGC 5216 data. However this time each stop along the way will highlight the
details of processing. I will use this data set throughout.
I would encourage you to open your own data set and follow along either
with the first example (NGC 6140) or this one. My naming convention is
generally:
- Clearff (or ffclear) for Clear filtered flats and rff, gff, bff for RGB flats
respectively.
- LUM are the luminance images and files that end (suffix) in R,G,B are
the colored filtered data.
You probably noticed how I listed files as I opened them. In Windows (and
many other operating systems) you can isolate a list of specific files from a
larger collection by using wildcards characters like the * .
- *r* will list any file that has an “r” in the filename. This could be useful if
all I want to see are my red data. However if I also have a file in the
directory called “MeanN5216Green.fit” it will show up in the list (because
of the “r” in Green). Can you think of a way to avoid this with my data?
- Answer: *r.* (or *r.fit) Since my color filter names are a suffix at the
end of the filename, they are always next to the “.”
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
First Processing ExampleNGC 6140
First Processing Example
NGC 6140
•
The most important thing to take away from this session is the workflow of
Fundamental Steps:
1. Make Master Biases, Darks, and Flats (last section)
2. Calibrate Data
•
•
3.
4.
5.
6.
7.
8.
9.
•
•
Scale Darks (use Bias) if necessary
Subtract Bias from Flat if necessary
Examine Data (blink images)
Apply Hot/Cold Pixel Filter if necessary
Debloom data if necessary
Register images
Normalize data
Reject outlier values
Combine
Some of the conditions for when to perform certain processing steps (and
when not to) are demonstrated in future sections.
This session was truly “on-the-fly.” I encountered the data just as you heard
me. It isn’t perfect, but it is representative of a typical processing session. I
would recommend following along with your own data.
RESOURCE:
fundamentalsteps.doc
(in the Documents directory)
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Blur Filters (Gaussian)
Blur Filters (Gaussian)
•
•
•
•
If you are reading this, congratulations! I did not want to let a Gaussian
function defeat me. You might argue my attempt here was a Pyrrhic victory
at best.
A median filter is often a good tool to get rid of hot pixels. Unfortunately it
applies to every pixel thereby smoothing the image. You can use a median
filter as a “poor man’s dark” if you want to examine (temporarily, do not
save!) your data and you do not have a dark frame handy.
The important idea here is that a Gaussian function finds its way into many
aspects of astronomy. Although it is arbitrary since you cannot *prove* that
discrete data and measurements are described by a symmetric and infinite
function; but it is helpful in many areas.
This is the only section I demonstrate file math and pixel math utilities.
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Hot/Cold Pixel and Deblooming Filters
Hot/Cold Pixel and De-bloomng
Filters
•
•
•
•
•
•
•
•
The Hot/Cold Pixel Filter is a rejection algorithm that identifies pixels that are very
different than their neighbors. These pixels are selected and colored red.
- The strength parameter is the conditional statement that controls how
aggressively it makes this determination. Usually 1 or 2 is sufficient.
- I usually keep the Upper Limit threshold high which identifies hot pixels (and
maybe pixel-like cosmic rays) at any brightness.
- Interpolate pixels using a small sigma (width). 1 or 2 is usually quite good.
The Hot/Cold Pixel Filter is always used after calibration but before any image is
resampled. Thus you must do this before registering images. This is also true for the
De-blooming filter.
Try to avoid using the Hot/Cold Pixel Filter on undersampled data where stars are the
sizes of pixels. Instead dither data during image acquisition and later use rejection
techniques to identify hot pixels.
In general, If you have dithered data, more than 7 frames, and you can use Nearest
Neighbor for the resample method during image registration then skip using the
Hot/Cold Pixel Filter and rely on statistical rejection
The saturation level of the De-blooming filter determines how much “into” a star a
bloom is selected. Usually 5,000-10,000 counts below the chip’s saturation level is a
good choice.
Subsequent star shaping will always be necessary during post processing after bloom
removal.
When De-blooming, use the smallest width possible to interpolate pixels. This will
minimize the undesirable smooth areas where blooms are removed.
The Missing Value pixel substitution is an excellent alternative to De-blooming filters.
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Gain and Read Noise
Gain and Read Noise
•
•
The values of the gain and read noise of your camera are useful when computing
certain things in CCDStack. The Poisson description as used in the rejection utility is a
tool that needs to know these values.
- Exposure Calculators that help determine the best subexposure time will also
require these values.
Notice that when I created a selection in an image, all I needed to do was select other
images from the Image Manager (or blink) and I will measure the same region in other
images. A very handy feature.
Resource: gain_Readnoise.xlsx
(in the Documents directory)
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Gain and Read Noise
Gain and Read Noise
•
•
The values of the gain and read noise of your camera are useful when computing
certain things in CCDStack. The Poisson description as used in the rejection utility is a
tool that needs to know these values.
- Exposure Calculators that help determine the best subexposure time will also
require these values.
Notice that when I created a selection in an image, all I needed to do was select other
images from the Image Manager (or blink) and I will measure the same region in other
images. A very handy feature.
Resource: gain_Readnoise.xlsx
(in the Documents directory)
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Processing NGC 5216
(The Beginning)
Processing NGC 5216
(The Beginning)
•I had to begin with this brief section in order to set up the processing
workflow for the NGC 5216 data set. However this time each step and stop
along the way will highlight the details of processing. I will use this data set
throughout.
•I would encourage you to open your own data and follow along either with
the previous example (NGC 6140) or this one.
•For files, my naming convention is generally:
-Clearff (or ffclear) for Clear filtered flats and rff, gff, bff for RGB flats
respectively.
-LUM are the luminance images and the files that end (suffix) in R, G, and B are
the colored filtered data.
•You probably noticed how I listed files as I opened them. In Windows (and
many other operating systems) you can isolate a list of specific files from a
larger collection by using wildcard characters like the * .
-*r* will list any file that has an “r” in the filename. This could be useful if all I
want to see and open are red data. However if I also have a file in the directory
called MeanN5216Green.fit it will show up in the list as well (because of the “r”
in Green). Can you think of a way to avoid this with my data?
-Answer: *r.* (or *r.fit) Since my color filter names a suffix at the end of the file
name they are always next to the “.”
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Processing NGC 5216
(Calibration)
Processing NGC 5216
(Calibration)
•This section begins with the first fundamental processing step- Calibration.
•The added twist is that the data sets were taken on different nights. I used the
include status to manage calibrating the data while keeping the entire stack open.
•Calibrated data are stored an a new nested directory so that I do not over-write
(clobber) the original data at any point.
•Closely examine calibrated images.
-Look for dark pixels scattered throughout the image. These will be an indication
of a poorly subtracted dark frame. Also make certain the scaling factors (if any)
are correct.
-Look for unexpected gradients. If you still see dust donuts in your calibrated
images a flat error is most likely the cause. Data taken through clouds or during
a moonlit sky may not flat field perfectly.
-Identify the “best” frames. This will come in handy later. Registration requires a
reference frame. Normalization does not require a good frame, but picking one
will help you evaluate the quality of other frames. Some data rejection (Poisson)
require a good image.
•If you cannot remember whether an image has been calibrated, look into the FITS
header window. The History field will indicate operations that have been performed
on an image.
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Processing NGC 5216
(Registration)
Processing NGC 5216
(Registration)
•Before you enter the registration utility select an image (image manager) to be your
reference frame by displaying it. Then navigate to Stack->Register.
•The AutoStar (select) tool is a new feature to CCDStack.
-Lower the radius to try to select more stars at a given threshold level. Raise the
radius value if you are working with a starry field and select too many.
-Choose a “safe” upper limit so that you are not selecting saturated stars.
-CCDStack chooses the lower limit well. To be more conservative, raise this value a
bit to make certain you are using “solid” (good S/N) stars.
-You will see this tool again in the Deconvolution Utility.
•The website I presented that explains the interpolation methods is found here:
http://www.cambridgeincolour.com/tutorials/image-interpolation.htm
•When it is possible to use, I am big fan of using Nearest Neighbor (NN) for the
interpolation method. The conditions you generally want to meet include:
1. More than 7 frames
2. Dithered data
3. No rotation or scaling differences between frames
•Now, I need to get something off my chest. Did you notice that between the two nights the
images are slightly ROTATED? This is in conflict with the above. However… the potential
artifacts would be evident for only half of the data and they would be small (mostly near the
edges of the field where the rotation results in the greatest linear deviation). So I chose to
use NN anyway to compare with the Bicubic B-Spline.
•I generally use Bicubic B-Spline on my color data since I have less than seven (binned)
images. This means I will generally use the Hot/Cold Pixel filter earlier in the processing.
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Processing NGC 5216
(Normalization)
Processing NGC 5216
(Normalization)
•Normalization is a prerequisite for data rejection (the next step).
•It doesn’t matter which image you choose to be the reference for purposes of
normalization. However, choosing an average image will help you quickly
identify images that are very different when you look at the resulting scaling
factors/weights.
•You can normalize a stack of data multiple times without harm.
•The calculated weights are the inverse of the scaling factors.
-I generally throw away images of equal integration times with weights less
than 0.5.
•If your images have areas where there is no overlap, be certain to select
regions that are common to *all* images. (I say this from experience, this will
really mess with your mind.)
• The Auto normalization feature has two modes:
1. You press OK and CCDStack tries to identify the sky and signal values
on its own.
2. You first select a region that contains both sky values and some bright
signal. Then you press OK.
• I prefer to manually select both. Fields that are filled with nebulosity will
often require manual selection as well.
• Using just offset Normalization on color images will neutralize the sky color
(just like the color utility does as you will see later).
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Processing NGC 5216
(Rejection)
Processing NGC 5216
(Rejection)
•The rejection step merely highlights (selects) pixels with values that exceed the
rejection conditions of “good” or “likely.” These pixels are painted red.
•Make certain the “Clear before Apply” box is checked if you want to redo or adjust
the resulting number of rejected values.
•When using StD rejection it does not matter which image happens to be displayed.
•I usually use StD rejection on stacks that contain 7 or more images. The more the
better. For large stacks consider using several iterations (rejecting potentially several
values).
-For *very* large stacks simple min/max clipping will probably suffice.
•Poisson Rejection is a great tool for stacks that contain fewer than 7 images.
-The choice of image matters somewhat as the calculation of the sigma value is
based on the ADU of the top image. Use a good or typical image- not one with
many obvious issues.
• In general a good result is 2% (or less) rejection of values. Sigma values of 2-2.5
tend to give these results. (Easy to remember, just think “2”)
• If you have results with one image showing large numbers of rejected pixels and
the other images having few or none- check the normalization of the data.
Resources:
rejectionV2.xlsx
RejectionUsingExcel.pdf
(In the Documents directory)
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Deconvolution
Deconvolution
•Use AutoStar Selection to find a handful of stars in your image in order to model or
measure the PSF.
-Be certain not to include saturated stars or stars that are too dim.
•Positive Constraint is usually better behaved always tending towards a sharper
solution. Maximum Entropy can sometimes produce incredible results with bright
objects.
•You can “abort” Positive Constraint at any point. Only abort Maximum Entropy when
it settles on a solution.
•Based on my results to-date:
1. Use a matrix radius of between 1.5 to 2 times your measured stellar FWHM.
2. The sharpen count controls the sharpness of the PSF- but the resulting
behavior affects the speed of “condensation” and the sharpness of star
edges. Larger values will show less sharpening in your final image. Smaller
values (especially 0) are more aggressive.
3. The bias subdivisions control the ringing effect of dark circles. If dark circles
appear lower the value. Use the largest value you can that does not show
the artifact.
•I expect and accept all kinds of artifacts associated with saturated pixels- especially
saturated stars. I choose to blend images with the original (unsharpened version) in
Photoshop later.
•Maximum Entropy scales the brightness values of images after it has completed.
You will need to apply the same screen stretch (again) that the original image has in
order to compare them.
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
Deconvolution
Deconvolution
•Use AutoStar Selection to find a handful of stars in your image in order to model or
measure the PSF.
-Be certain not to include saturated stars or stars that are too dim.
•Positive Constraint is usually better behaved always tending towards a sharper
solution. Maximum Entropy can sometimes produce incredible results with bright
objects.
•You can “abort” Positive Constraint at any point. Only abort Maximum Entropy when
it settles on a solution.
•Based on my results to-date:
1. Use a matrix radius of between 1.5 to 2 times your measured stellar FWHM.
2. The sharpen count controls the sharpness of the PSF- but the resulting
behavior affects the speed of “condensation” and the sharpness of star
edges. Larger values will show less sharpening in your final image. Smaller
values (especially 0) are more aggressive.
3. The bias subdivisions control the ringing effect of dark circles. If dark circles
appear lower the value. Use the largest value you can that does not show
the artifact.
•I expect and accept all kinds of artifacts associated with saturated pixels- especially
saturated stars. I choose to blend images with the original (unsharpened version) in
Photoshop later.
•Maximum Entropy scales the brightness values of images after it has completed.
You will need to apply the same screen stretch (again) that the original image has in
order to compare them.
The Complete
Tutorial of
CCDStack
Version 2.0
Review:
DDP and USM
DDP and USM
•Use the Autoscale with DDP checked to get “close” first.
•DDP and slider behavior:
-Moving the DDP slider to the left will brighten the image in a non-linear way.
-Adjusting the DDP slider in combination with slight adjustments to the gamma
slider will permit you to keep some of the brightest things from becoming
clipped.
-Moving the Maximum slider to the right (making large values) will subtly
improve very bright regions. However this effect is very small and in general do
not expect much change.
-Adjust the Background slider to control the black point. NOTE: It is possible to
create a small enough range between the value that the DDP slider is at and the
background value that raising the background value (black point) DOES NOT
make the sky darker! You must move the DDP slide a bit back to the right in
order to accomplish this.
•Remember to examine the image and set the black point carefully looking for bitmap
values above 10-15 (for example) when rolling your cursor across the sky.
The Complete
Tutorial of
CCDStack
Credits