Instrument Science Report NICMOS 2008-003
Verifying the NICMOS Count
Dependent Non-Linearity
Correction
B. Shaw, R. S. de Jong
June 30, 2008
ABSTRACT
We re-investigate the NICMOS count dependent non-linearity, using the flat field lamps to
reach high count regimes. The NICMOS count dependent non-linearity has been previously quantified and is currently corrected in NICMOS pipeline. In this document we show
that the count dependent correction is still accurate to within 1.2% of true linearity for
counts above 100 ADU independent of filter used. This is a response to the count-rate
dependent non-linearity investigations, to eliminate questions about the current count
dependent corrections.
Introduction
The NICMOS cameras detect counts non-linearly. This effect is typical for all IR detectors, and was quantified in Cycle 7 by fitting a second degree polynomial to the count
detection for each pixel in long exposure flat field images. The count non-linearity is currently corrected through the standard calnica data pipeline. We revisit the count nonlinearity to verify that it is still corrected to within 1.2% of linear count collection for
counts greater than 100 ADU.
Data
Data from count rate non-linearity proposal 11062 and flat monitor programs 10728 and
11059 were used for this investigation. The non-linearity data consists of a sequence of
images of a stellar cluster, NGC 3603, taken out of focus, followed with the same
Copyright© 1999 The Association of Universities for Research in Astronomy, Inc. All Rights Reserved.
Instrument Science Report NICMOS 2008-003
sequence with the flat field lamp turned on, followed by darks. The lamp-on images provide artificially increased counts to verify the count correction in the high count
regime.The dark frames taken immediately after the stellar cluster images were used in
substitution of the calnica pipeline reference files to subtract the dark current. Similarly,
the flat monitor data includes a matching set of long exposure, lamp off-on images for an
empty field. The lamp-off frames taken before the flat field lamp was turned on, were used
in substitution of the calnica pipeline reference files to subtract the dark current.
Table 1. Multiaccum data sets used for this investigation. Each set includes a combination
of images with both the flat field lamp on and off for the same field and readout sequence.
Target
Rootname
Obs-Date
Camera
Filter
Pointed Flat
n6ki03
2002-07-16
1
F160W
Pointed Flat
n8i303
2002-11-08
1
F160W
Pointed Flat
n8um01
2003-10-15
1
F160W
Pointed Flat
n8um04
2004-04-25
1
F160W
Pointed Flat
n94n02
2005-03-31
1
F160W
Pointed Flat
n9hy01
2005-10-28
1
F160W
Pointed Flat
n9vf01
2006-11-07
1
F160W
Pointed Flat
n9vf02
2007-01-21
1
F160W
Pointed Flat
n9vf03
2007-02-07
1
F110W
Pointed Flat
n9hy01
2006-02-13
2
F160M
Pointed Flat
n9vf02
2007-01-21
3
F160W
Pointed Flat
n9hy02
2006-02-13
3
F160W
Pointed Flat
n9vf04
2007-08-26
2
F222M
NGC 3603
n9voa1
2007-01-20
1
F090M
NGC 3603
n9vo01
2007-01-20
1
F110W
NGC 3603
n9vo01
2007-01-20
1
F160W
NGC 3603
n9voa4
2006-11-28
2
F110W
NGC 3603
n9vo01
2007-01-20
2
F160W
NGC 3603
n9vo05
2006-11-28
3
F110W
NGC 3603
n9vo04
2006-11-28
3
F150W
All cluster observations were reduced using the standard calnica pipeline, except the
dark frames which were only reduced through the ZSIGCORR and the ZOFFCORR calibration steps. Both the _ima and _cal data products were used for this analysis. All _ima
2
Instrument Science Report NICMOS 2008-003
files for each camera and filter combination, taken before the flat field lamp was used,
were averaged together with MSCOMBINE. Similarly, all corresponding _cal files were
averaged together. All the lamp-on images for a unique camera, filter combination were
averaged as well. Lamp-off images taken after the flat field lamp-on images, were not used
since they contain count persistence from the lamp-on observations. The small number of
lamp-off images due to this restriction, limits the possible S/N that was attainable with
these data sets. Despite averaging, both noise and changing bias levels limits our investigation to a count range greater than 100 ADU. In this range, the noise has been sufficiently
constrained and the non-linearity tests produce useful results. To test the non-linearity for
counts below 100 ADU, more lamp off data sets are needed, as well as longer exposures.
However, for the purposes of this investigation, using only counts that are greater than 100
ADU will suffice to show that at moderate to high count levels the NICMOS count nonlinearity has been corrected to within a few percentage of true linearity
The final averaged _ima files were converted to counts by multiplying each extension
by the samptime keyword. Dependence on previous readouts were removed by using
SAMPDIFF to subtract the previous readout from each extension. The resultant images
were then converted back into count rates by dividing by the deltatime keyword for each
extension. Data quality masks were applied to remove all flagged pixels except those containing signal in the 0th read since all the pixels in the lamp-on images would be
unnecessarily flagged. All pixels in the _cal files were logarithmically binned by count
rate, and these pixel bins were subsequently used to investigate the average count rate in
the different samples of the _ima files. Figure 1 shows the individual readouts for the final
products from SAMPDIFF in units of count rate for a NIC1 lamp on set. The following
color coded data quality flags are overlaid on the image:
3
Instrument Science Report NICMOS 2008-003
Table 2. Data quality flags and corresponding colors used in Figures one and two.
Data Quality
Flag
Color
Reed Solomon
Cyan
Bad Linearity
Correction
Magenta
Bad Flat Field
Correction
Maroon
Bad Background Subtraction
Orange
Defective Pixel
Blue
Saturated Pixel
Green
Missing Data
Khaki
Bad Pixel
Detected by
Calibration
Red
Cosmic Ray
Yellow
Figure 1: Multiaccum readouts output from sampdiff in count rates of NGC 3603 with flat
field lamp for NIC 1, F160W, samp_seq =STEP16, exposure time~ 80 seconds. Images
are ordered from first readout to last going left to right with overlaid data quality flags.
4
Instrument Science Report NICMOS 2008-003
The pointed flat observations were also reduced using the standard calnica pipeline.
However, the lamp-off images were used as darks and prepared by using only the ZSIGCORR and ZOFFCORR steps in calnica. They were then median combined and used as
the DARKFILE for processing the lamp on images through the full calnica pipeline. The
final _ima and _cal lamp on images were each median combined and used for further processing. As with the cluster data the _ima file were converted to counts using the
samptime keyword. The subsequent readouts subtracted from one another with SAMPDIFF and converted back into count/s with the deltatim keyword. Data quality masks were
applied to remove all flagged pixels except those containing signal in the 0th read. The
individual readouts of the final products for one of the datasets is shown in Figure 2.
Figure 2: Pointed flat field readouts from sampdiff output in count rates for camera 1, filter F160W, samp_seq =STEP8, exposure time~152 seconds. Images are ordered from first
readout to last going left to right with overlaid data quality flags.
Results
To test the linearity correction, we look at the countrate of a pixel as function of the
counts in the pixel. Figures 3-6 show the average of a count rate bin in the _ima files normalized to the average of the same bin in the _cal files as a function of average counts in
5
Instrument Science Report NICMOS 2008-003
the _ima bin. For all lamp-off images the average value for the smallest count rate bins
were subtracted from all other bin averages to minimize fluctuations in the counts due to
bias jumps. Error in the average count rate for an individual bin is represented as the
inverse square root of the number of contributing pixels. Error bars are not included in the
lamp-off plots to keep them legible, especially near the 100 count range.
Figure 3: Count non-linearity test for lamp off cluster images.
Count Linearity Test NIC1 F090M
Count Linearity Test NIC1 F110M
1.05
1.05
Count Rate Bins (DN/s)
0-0.33
0.33-1.0
1.0-3.3
3.3-10
10-33
33-100
100-330
330-1000
1000-3300
1.04
1.03
1.02
1.01
Count Rate Bins (DN/s)
0-0.33
0.33-1.0
1.0-3.3
3.3-10
10-33
33-100
100-330
330-1000
1000-3300
3300-10000
1.04
1.03
1.02
1.01
1
1
0.99
0.99
0.98
0.98
0.97
0.97
0.96
0.96
0.95
0.95
1
10
100
1000
1
10
Counts (DN)
100
1000
Counts (DN)
Count Linearity Test NIC2 F110M
Count Linearity Test NIC1 F160M
1.05
1.05
Count Rate Bins (DN/s)
0-0.33
0.33-1.0
1.0-3.3
3.3-10
10-33
33-100
100-330
330-1000
1000-3300
3300-10000
1.04
1.03
1.02
1.01
Count Rate Bins (DN/sec)
0-0.33
0.33-1.0
1.0-3.3
3.3-10
10-33
33-100
1.04
1.03
1.02
1.01
1
1
0.99
0.99
0.98
0.98
0.97
0.97
0.96
0.96
0.95
0.95
1
10
100
1
1000
10
100
Counts (DN)
Counts (DN)
6
1000
Instrument Science Report NICMOS 2008-003
Figure 4: Count non-linearity test for lamp off cluster images.
Count Linearity Test NIC2 F160M
1.05
Count Linearity Test NIC3 F110M
1.05
Count Rate Bins (DN/s)
0-0.33
0.33-1.0
1.0-3.3
3.3-10
10-33
33-100
100-330
330-1000
1000-3300
3300-10000
1.04
1.03
1.02
1.01
1.03
1.02
1.01
1
1
0.99
0.99
0.98
0.98
0.97
0.97
0.96
0.96
0.95
Count Rate Bins (DN/s)
0-0.33
0.33-1.0
1.0-3.3
3.3-10
10-33
33-100
100-330
330-1000
1000-3300
3300-10000
1.04
0.95
1
10
100
1000
1
Counts (DN)
Count Linearity Test NIC3 F150M
Count Rate Bins (DN/s)
0-0.33
0.33-1.0
1.0-3.3
3.3-10
10-33
33-100
100-330
330-1000
1000-3300
3300-10000
1.03
1.02
1.01
1
0.99
0.98
0.97
0.96
0.95
1
10
100
100
Counts (DN)
1.05
1.04
10
1000
Counts (DN)
7
1000
Instrument Science Report NICMOS 2008-003
Figure 5: Count non-linearity test for lamp on cluster images.
Count Linearity Test NIC1 F090M
1.05
Count Linearity Test NIC1 F110W
1.05
Count Rate Bins (DN/s)
90-90.33
90.33-91
91-93.3
93.3-100
100-123
123-190
190-420
420-1090
1090-3390
1.04
1.03
1.02
1.01
1.03
1.02
1.01
1
1
0.99
0.99
0.98
0.98
0.97
0.97
0.96
0.96
0.95
100
Count Rate Bins (DN/s)
370-370.33
370.33-371
371-373.3
373.3-380
380-403
403-470
470-700
700-1370
1370-3670
3670-10370
1.04
0.95
100
1000
1000
Counts (DN)
Counts (DN)
Count Linearity Test NIC1 F160W
1.05
Count Linearity Test NIC2 F110W
1.05
Count Rate Bins (DN/s)
156-156.33
156.33-157
157-159.3
159.3-166
166-189
189-256
256-486
486-1156
1156-3456
3456-10156
1.04
1.03
1.02
1.01
1.03
1.02
1.01
1
1
0.99
0.99
0.98
0.98
0.97
0.97
0.96
0.96
0.95
Count Rate Bins (DN/s)
370-370.33
370.33-371
371-373.3
373.3-380
380-403
403-470
470-700
700-1370
1370-3670
3670-10370
1.04
0.95
10
100
1000
100
Counts (DN)
1000
Counts (DN)
8
Instrument Science Report NICMOS 2008-003
Figure 6: Count non-linearity test for lamp on cluster images.
Count Linearity Test NIC2 F160W
1.05
Count Linearity Test NIC3 F110W
1.05
Count Rate Bins (DN/s)
570-570.33
570.33-571
571-573.3
573.3-580
580-603
603-670
670-900
900-1570
1570-3870
1.04
1.03
1.02
1.01
1.03
1.02
1.01
1
1
0.99
0.99
0.98
0.98
0.97
0.97
0.96
0.96
0.95
100
Count Rate Bins (DN/s)
1600-1600.33
1600.33-1601
1601-1603.3
1603.3-1610
1610-1633
1633-1700
1700-1930
1930-2600
2600-4930
4930-11600
1.04
0.95
1000
1000
Counts (DN)
Counts (DN)
Count Linearity Test NIC3 F150M
1.05
Count Rate Bins (DN/s)
1310-1310.33
1310.33-1311
1311-1313.3
1313.3-1320
1320-1343
1343-1410
1410-1640
1640-2310
2310-4610
1.04
1.03
1.02
1.01
1
0.99
0.98
0.97
0.96
0.95
1000
Counts (DN)
A similar test was performed on the pointed flats and one of the cluster images. Instead
of binning, thirty random pixels for each quadrant of the images were chosen. The count
rate of these pixels in the sampdiff output was normalized to the count rate in the _cal file
and plotted versus the counts in the sampdiff output for each readout. Below each plot is
the average of all the pixels in the quadrant for each readout with errors bars indicating
one standard deviation of the count rate, unless otherwise indicated. Quadrants are labeled
as a-d clockwise around the image starting in the upper left corner. To verify that there are
9
Instrument Science Report NICMOS 2008-003
no undetected saturation in the pixels, the counts of the same random pixels are plotted
versus time in Figures 8,10, and 12.
Figure 7: Count Linearity Test for 30 random pixels in each quadrant of a NGC 3606
image with the flat field lamp for NIC 1, filter F160W. Below each linearity test is the
average behavior for the entire quadrant with one standard deviation of the count rate.
10
Instrument Science Report NICMOS 2008-003
Figure 8: Counts versus time for same images and pixels used to create Figure 7. The
plots in red show the actual pixels used for the linearity test, while the plots in blue show
the same pixels without using data quality information to throw out bad pixels.
11
Instrument Science Report NICMOS 2008-003
Figure 9: Count Linearity Test for 30 random pixels in a pointed flat field (previously
shown in figures 2) for NIC 1, filter F160W. Below each linearity test is the average
behavior for the entire quadrant with one standard deviation of the count rate .
12
Instrument Science Report NICMOS 2008-003
Figure 10: Counts versus time for same images and pixels used to create Figure 10. The
plots in red show the actual pixels used for the linearity test, while the plots in blue show
the same pixels without using data quality information to throw out bad pixels. Notice
there are no grossly saturated pixels used in the linearity test, even though there is clear
saturation in some of the pixels for longer readouts.
13
Instrument Science Report NICMOS 2008-003
Figure 11: Count Linearity Test for 30 random pixels in a pointed flat field for NIC 1, filter F110W. Below each linearity test is the average behavior for the entire quadrant with
one standard deviation of the count rate .
14
Instrument Science Report NICMOS 2008-003
Figure 12: Counts versus time for same images and pixels used to create Figure 11. The
plots in red show the actual pixels used for the linearity test, while the plots in blue show
the same pixels without using data quality information to throw out bad pixels.
15
Instrument Science Report NICMOS 2008-003
Figure 13: Count Linearity Test for 30 random pixels in each quadrant of a pointed flat for
NIC 2, filter F160W. Below each linearity test is the average behavior for the entire quadrant with two standard deviations about the mean.
16
Instrument Science Report NICMOS 2008-003
Figure 14: Count Linearity Test for 30 random pixels in each quadrant of a pointed flat for
NIC 2, filter F222M. Below each linearity test is the average behavior for the entire quadrant with two standard deviations about the mean
17
Instrument Science Report NICMOS 2008-003
Figure 15: Count Linearity Test for 30 random pixels in each quadrant of a pointed flat for
NIC 3, filter F160W taken in 2006. Below each linearity test is the average behavior for
the entire quadrant with two standard deviations about the mean.
18
Instrument Science Report NICMOS 2008-003
Figure 16: Count Linearity Test for 30 random pixels in each quadrant of a pointed flat for
NIC 3, filter F160W taken in 2007. Below each linearity test is the average behavior for
the entire quadrant with two standard deviations about the mean
Analysis
On first analysis, it appears the count correction improves with higher counts. This is
likely just the improvement in signal-to-noise for longer exposure times. The NIC 1, cluster plots show a slight non-linearity from +1% to ~ -0.5% between 100 and 5000 ADU.
Above 5000 ADU the non-linearity increases again to ~+1%. The pointed flat field data
verifies that NIC 1 does indeed appear to have a similar non-linearity as seen in the cluster
19
Instrument Science Report NICMOS 2008-003
data. The normalized count rate in the pointed flats decreases to -0.3% at 5000 counts then
increases to +~1% for greater counts. Additionally, all quadrants in this camera exhibit
this behavior consistently over the course of a few years for both Pre-NCS and Post-NCS
data regardless of the filter used. Appendix A includes linearity plots from pointed flats
taken with NIC 1 and filter F160W from 1997 to 2006. We recommend re-deriving the
count non-linearity parameters for NIC 1.
The NIC 2 results are more difficult to characterize. There does not seem to be any
systematic offset in the correction, but rather a random distribution about unity. The NIC 3
plots shows a systematic decrease in count rate, but it seems to be more related to time
than to total counts. The cause of this effect is currently not clear.
Conclusions
The overall trend for all filters and cameras is a maximum median offset of no more than
1.24% for counts above 100 ADU. The count dependent non-linearity correction appears
to still be accurate for both NIC 2 and NIC 3 cameras. The slight non-linearity in the NIC
1 results should be further investigated and corrected for counts above 100 ADU.
References
Barker, E., Pirzkal, N., et al. 2006, "NICMOS Instrument Handbook", Version 9.0, (Baltimore: STScI)
20
Instrument Science Report NICMOS 2008-003
Appendix A
This appendix includes plots of linearity tests for NIC1 1 with filter F160W data taken
from 1997 to 2006.
Figure 17: Count Linearity Test for 30 random pixels in each quadrant of a pointed flat
field image taken in 1997 for NIC 1, filter F160W. Below each linearity test is the average
behavior for the entire quadrant with 2 sigma about the mean. Note this data was taken
before the installation of NCS.
21
Instrument Science Report NICMOS 2008-003
Figure 18: Count Linearity Test for 30 random pixels in each quadrant of a pointed flat
field image taken in July 2002 for NIC 1, filter F160W. Below each linearity test is the
average behavior for the entire quadrant.
22
Instrument Science Report NICMOS 2008-003
Figure 19: Count Linearity Test for 30 random pixels in each quadrant of a pointed flat
field image taken in November 2002 for NIC 1, filter F160W. Below each linearity test is
the average behavior for the entire quadrant with 2 sigma about the mean.
23
Instrument Science Report NICMOS 2008-003
Figure 20: Count Linearity Test for 30 random pixels in each quadrant of a pointed flat
field image taken in October 2003 for NIC 1, filter F160W. Below each linearity test is the
average behavior for the entire quadrant with 2 sigma about the mean.
24
Instrument Science Report NICMOS 2008-003
Figure 21: Count Linearity Test for 30 random pixels in each quadrant of a pointed flat
field image taken in April 2004 for NIC 1, filter F160W. Below each linearity test is the
average behavior for the entire quadrant with 2 sigma about the mean.
25
Instrument Science Report NICMOS 2008-003
Figure 22: Count Linearity Test for 30 random pixels in each quadrant of a pointed flat
field image taken in March 2005 for NIC 1, filter F160W. Below each linearity test is the
average behavior for the entire quadrant with 2 sigma about the mean.
26
Instrument Science Report NICMOS 2008-003
Figure 23: Count Linearity Test for 30 random pixels in each quadrant of a pointed flat
field image taken in November 2006 for NIC 1, filter F160W. Below each linearity test is
the average behavior for the entire quadrant with 2 sigma about the meaan.
27