Instrument Science Report NICMOS 2002-001
M. Sosey
January 11, 2002
A BSTRACT
A study of the frequency and location of the NICMOS bad pixels in each of the three cameras had not been undertaken since the instrument was commissioned on-board HST in
1997. The original MASKFILEs located in the calibration database (CDBS) were compiled with data taken during Systems Level Thermal Vacuum (SLTV) ground testing in
August 1996. This analysis examined dark frames taken during the initial lifetime of NIC-
MOS, March 1997 - January 1999, and details the creation of new bad pixel masks for each camera. Updated flat field reference files were also created as a result of this analysis.
The NICMOS detectors are solid slabs of HgCdTe in which individual pixels are bump-bonded to single pixels of a silicon multiplexor array which is used for readout. The detectors are read out non - destructively, making it possible to see the charge accumulation in each pixel over the course of an observation. Dark current is signal accumulated in a detector that is not exposed to any external illumination. Such signal does not depend on the detectors flat-field response or non-linearity. Dark frames are taken to measure the linear accumulation of signal, making them an excellent candidate for identifying aberrant pixels. As with any detector, the sensitivity of individual pixels varies across the array. Hot pixels are defined as those with excessive charge when compared to surrounding pixels.
Conversely, cold pixels have little or no sensitivity - exhibiting extremely low dark current as well as low quantum efficiency as measured in flat field images.
Copyright© 2002 The Association of Universities for Research in Astronomy, Inc. All Rights Reserved.

Instrument Science Report NICMOS 2002-001
The current bad pixel masks that are available in CDBS were created in August 1996 following SLTV ground testing of NICMOS. After the cryocooler is installed during
SM3B, NICMOS will most likely be operating at a higher temperature. Dark frames taken during end of life (EOL) span the temperature range 62K ~ 78K and may be useful for predicting the number of bad pixels which might be expected around the Cycle 11 operating temperature of the instrument.
Dark frames were taken from March 1997 to January 1999 and span the temperature range 60.8K - 78.15 (many images exist after the detectors reached 78.15 K but this temperature is the limit of the mounting cup sensors). All of the images used in this analysis were processed in the following manner:
1. Images were run through CALNICA with the following header keywords set to
OMIT: NLINCORR, DARKCORR, FLATCORR, UNITCORR.
The resulting
IMA files were used to create a median image for all of the darks taken on a specific day which also have similar exposure time, SAMP_SEQ , and temperature. The majority of the analysis used darks taken with the STEP64 SAMP_SEQ , however, several other sequences were also used. These include STEP32 for 1998 camera 3 datasets, and STEP128 for all cameras in 1997.
IMA files are ideal for this purpose because they contain the fully processed frames from the exposure, as well as cosmic ray (cr) detection masks, but the individual pixels have not been reset according to the bad pixel lists in the DQ image (the CAL files do contain altered pixels as a product of the CRIDCALC
1
step). Before the final median combination (which also allows for better cosmic ray masking), amp glow was removed from each of the individual images and a mask containing only cosmic ray hits was created.
2. Shading, a time dependant bias imparted by the readout amplifiers, can also be a problem. The amplitude of the shading depends on the camera and the time since the array was last read out. To remove this effect the median value of each column or row was fit and subtracted from itself - sigma clipping was used to reject outlying pixels from the fit.
3. In order to remove any remaining large image artifacts, a ring median filter was passed over the median image and the resulting image subtracted from the original median - this accounted for a very small change in the actual image statistics. A final image, in units of sigma (from the mean) was then created for the purpose of bad pixel clipping.
The bad pixel thresholds were set to 5
σ
in all cameras. This level was chosen in each camera after examining the image histograms for general pixel dispersion information.
1.
During the CRIDCALC step, flagged pixel values don’t get used in the computation of the final pixel value. For hot/cold pixels, which have all their samples flagged, the output CAL file will have a 0 for that pixel in all of the extensions.
2

Instrument Science Report NICMOS 2002-001
Separate masks were generated for the hot and cold pixels, but the final bad pixel masks for each camera contain both.
Figure 1: Example sigma image from camera 1 with 2 evident bad pixels
Hot Pixels
Line (pixels)
The resulting masks were compared against the previous static bad pixel masks, as well as grot masks (which were created by examining flat-field images) in order to ensure only pixels inherently bad on the detector were included in the final masks. By comparing the cold pixels to the flat fields we can expose the difference between pixels with low dark current and pixels which have little or no response, showing small quantum efficiency.
Since grot are believed to be paint flecks which are statically bonded to the detector faceplates, they only exhibit themselves in flat field images where an external source illuminates the detector. However, there is no way to distinguish cold pixels detected in the darks from ‘grotty’ pixels in the flats if the same pixel is designated ‘bad’ in both masks. In some cases hot pixels were flagged as matching pixels in the grot mask. In previous analysis some of the grotty pixels appeared to contain bright edges, this can now be linked to the fact that some of the grot are merely next to hot pixels. For more information on grot
3

Instrument Science Report NICMOS 2002-001
see Sosey, et al 1999. Figure 2 below has example images from several of the processing
steps for reference.
Figure 2: Example processed camera 1 dark images, displayed at similar stretches
Original IMA image
IMA with amp glow
removed median of IMA images example bad pixel mask
Non EOL darks
Darks taken before the official beginning of the NICMOS end of life span the dates
March 1997 through October 1998 and the temperature range 57 - 62.5 K. These darks show a fairly stable number of bad pixels, however, there is a small fluctuation in the num-
ber and location of bad pixels throughout these images. Figure 3 shows the variation in the
detected number of bad pixels for camera 1 for January - October 1998 for a clipping value at 5
σ
.
4

Instrument Science Report NICMOS 2002-001
Figure 3: Bad pixels in camera 1 in 1998
A first glance at the above graph suggests that it is possible for bad pixels to change on short time scales. However, this is most likely a by-product of the image processing and changes in the well depth in the individual pixels. (The bias reset levels in a particular
quadrant can change between observations - a culprit of the pedestal effect) Table 1 shows
the processing results from 4 masks composed of images taken from May 16-19, 1998 in camera 1. The images were taken using the same observation parameters, and the masks show differences of up to 21 bad pixels - this large case is taken very lightly because the mask was created using only 2 images for cosmic ray rejection. By using all of the images
taken during this period to create a bad pixel mask (the last entry in Table 1), you get much
more reasonable results, which more closely resemble the statistics of the May 16, 1998 mask. Both masks are identical except for 2 extra pixels which were detected in the cumulative image mask - these bad pixels also exist in the sigma images from the daily medians, but at a slightly lower level. These results suggest that the wider range in detected hot pixels is not due to an actual increase or decrease in bad pixels, but false detections because of a changing bias and possible non-detections due to undetected cr hits (either low level cosmic rays which were classified as noise or higher level cosmic rays which escaped detection because of the low number of images used or early impact on the multiaccum sequence).
5

Instrument Science Report NICMOS 2002-001
Table 1. Example mask statistics for May 1998
Date Temp(K) Exptime (s) Hot Pixels Cold Pixels
5 - 16 - 1998
5 - 17- 1998
5 - 18 - 1998
5 - 19- 1998
5/16 - 5/19
62.19
62.18
62.19
62.17
~ 62.18
895.93
895.93
895.93
895.93
895.93
68
71
89
75
69
22
22
20
21
23
No. Images in median
2
2
8
4
16
For the purpose of creating static bad pixel masks, only pixels which were consistently bad were retained in the final image. These ‘survivor’ masks are the ones which will be delivered to CDBS for pipeline processing. The above example is probably the best reason for using the ‘survivor’ masks as the core bad pixel masks in the archive. Users are encouraged to make their own bad pixel masks to ensure the appropriate detections for their datasets and exposure times. The masks delivered to the archive contain only those pixels which were consistently bad on time scales of 100 ~ 2000s.
EOL darks
The dark frames taken during end of life represent a special chance to examine the rise and fall of bad pixels in each of the detectors according to temperature. While this may not exactly represent the expected cosmetics of NICMOS under NCS, it should give us some idea of what we may expect when the cameras reach the final operating temperature. However, since the detector moved from 63 K - > 78K in less than a week, they also represent the extreme end of pixel response, in a detector which was not in thermal equilibrium.
Images taken during this time, following January 2, 1999, were examined with caution.
The dark current bump was not corrected for in the reduction process, making anomalous pixel detections on these images questionable since the well depth varied significantly in some pixels as the temperature increased. See NICMOS-ISR-00-002 for more information on the changes in dark current during EOL.
The EOL darks in camera 1, taken before January 2, 1999, compare easily with earlier darks and their resulting bad pixel masks. After January 2, the detector temperature began increasing at a more rapid rate. This confuses essential cosmic ray masking and shading anomalies because images must be medianed together to make an acceptable baseline image for bad pixel detection. In order to avoid as much of this as possible, these darks were separated into image groups that differed by no more than 0.2 K for processing. This was not very effective for images taken after January 5, 1999 because the sampling of available dark images does not cover a short enough time scale to account for less than 0.2
K change in temperature or provide enough images for reliable cosmic ray rejection. For these reasons, darks taken after January 5 (~69 K), in camera 1, were not included in the
6

Instrument Science Report NICMOS 2002-001 analysis. Camera 2 did not begin its steep climb to the limits of the mounting cup sensors until January 6, 1999; data following this date was not included in the analysis.
In order to get a better idea of the actual number of static bad pixels in this regime, survivor masks were also created for the EOL datasets. The mask for camera 1, constructed from the 1999 EOL data, can be seen compared to the bad pixel mask from the
1998 EOL data in Figure 4. The first thing that is evident from Figure 4 is the decrease in
consistently bad pixels as the detector temperature increased. This same trend can be seen in Camera 2 as well. This is not saying that in general, the number of bad pixels detected in each camera decreased. On the contrary, the number of detected hot pixels increased linearly with temperature. This is most likely due to the high rate of temperature change and not a change in the permanent characteristics of the pixels; at higher temperatures some pixels experience greatly increased dark current.
Figure 4: Comparison of EOL survivor masks from 1998 and 1999 datasets for camera 1
EOL 1998 data
62.51 K - 62.96 K
Bad Pixel Survivor Mask
EOL 1999 data
63.17 K - 75.22 K
Bad Pixel Survivor Mask
Figure 5 compares a median image from November 2, 1998 (the top image on the left) and
January 6, 1999 (the bottom image on the left). The graph on the right shows a fairly consistent level of cold pixels throughout EOL. It should be noted here that darks taken after the detector reached ~70K were taken at a sampling rate of about 1 dark for every 0.5K; on
7

Instrument Science Report NICMOS 2002-001 the day in question, the temperature of camera 1 spanned the range 69.26K ~ 74.87 K.
Figure 5: Comparison of camera 1 EOL median images, displayed at similar stretches with a graph of the detected hot and cold pixels for data in camera 1 after November 1998.
EOL (December 1998)
EOL (January 1999)
Figure 5 also shows a nearly linear increase in the number of detected hot pixels.
These are believed to be part of the “salt and pepper” effect since their identifications are not consistent. The “salt and pepper” appearance of the darks during the last few days of
EOL was discussed in NICMOS-ISR-99-001 which concluded that the most likely explanation for the sharp increase in anomalous pixels - pixels which containing a much higher dark current than the mean - was that the detectors were not in thermal equilibrium. When a comparison is made between the consistently bad pixels during EOL with those previously identified only a small change can be seen, as the total number of consistently bad pixels decreased.
Figure 6 is a visual comparison of the original bad pixel masks which are in CDBS with
new bad pixel masks created with data from 1997 and 1998. Many of the bad pixels are the same but there are also quite a few which have changed. The separation in date for the camera 1 masks is not completely arbitrary. May 1997 is close in time to the expansion of the dewar, and increased thermal change in the instrument. Data taken in March 1997
8

O
L
D
C
D
B
S
Instrument Science Report NICMOS 2002-001 begins at a temperature around 57.5K and increases to ~61.1K by May 1997. After this date, the rate of temperature change remained fairly stable, at around 0.05 - 0.1 K per month. Masks from cameras 2 and 3 were also scrutinized around this date, but only camera 1 showed significant changes.
Figure 6: Old CDBS bad pixel reference masks for each camera compared with new reference masks for 1997 and 1998
Camera 1 Camera 2 Camera 3
N
E
W
Pre May 1997 1997
N
E
W
1997 & 1998
Post May 1997 1998
An independent analysis of bad pixels was conducted by the NICMOS IDT team using images from cameras 1 and 3 (This data is unpublished, but the resulting images
9

Instrument Science Report NICMOS 2002-001 were kindly supplied by the IDT members). The resulting data cubes have bad pixel information separated by exposure time and date (in days from January 1, 1998). Processing for these images was done on the RAW image files. Two values were read for each pixel and the smaller of the two was chosen to avoid cosmic ray hits. The masks were then created by selecting pixels above and below a 3
σ detection threshold. Comparing these masks with the masks generated in this analysis for cameras 1 and 3 show very good agreement.
All detected bad pixels at higher sigma in camera 1 and camera 3 are present in the detections from the IDT masks. When a 3
σ detection mask is made from data in this analysis, the masks tend to differ more, however this is probably due to the different processing methods and the increased number of images used in this analysis. No distinction between hot and cold pixels is made in the IDT images.
Table 2. Static bad pixel information for all cameras 1997 - 1998
Camera
Previous
Hot Pixels
New
Hot Pixels
Previous
Cold Pixels
New
Cold Pixels
Dates Valid
IDT Pixel
Counts (3
σ
)
NIC 1
NIC1
NIC 1
NIC 2
NIC 2
68
68 a
68 a
94
94 a
17
47
73
83
186
242
10
10 a
10 a
11
11 a
3
21
22
19
10
25 pre May 1997 post May 1997
1998
1997
1998
NA
NA
129
NA
NA
NIC 3 65 0 1997-1998 158 (1998) a. Previously only 1 mask was used for all data, regardless of date. Also, these numbers are reversed from what is reported in the instrument handbook for hot and cold pixels, possibly because they were accidentally switched.
In camera 1, all of the cold pixels detected after May 1997 in this analysis are contained in the old CDBS bad pixel masks. The mask for data before May 1997 also contains the same detections, with the addition of 1 previously unlisted pixel. This appears to dis-
agree with the numbers in Table 2 by a factor of 2! Unfortunately, separate hot and cold
bad pixel masks no longer exist from the previous analysis for comparison, perhaps it was a mere translation error between hot vs. cold pixels when the figures were added to the original version of the instrument handbook. For the hot pixels, 25 of the pixels detected in the pre-May 1997 mask matched those in the CDBS file, while 24 matched in the post-
May 1997 and 1998 masks.
In camera 2, none of the 10 cold pixels from the new masks matched the CDBS masks for the 1997 mask, while 1 pixel matched in the 1998 mask. The two new masks agree with each other though, with the addition of 2 new detected pixels in the 1998 mask, as well as pixels flagged as consistently bad in the lower left corner of quadrant I. For the hot pixels, 87 matched the CDBS masks for both 1997 and 1998. The increase in hot pixels
10

Instrument Science Report NICMOS 2002-001 from 1997 - 1998 in camera 2 seems rather large and the sampling of the available darks was not good enough to detect exactly where the change occurred. By visually comparing the masks it can be seen that the majority of newly detected hot pixels are associated with previous bad pixels.
In camera 3, 14 of the hot pixels in the new mask matched the previous one. There were no cold pixels which were found to be consistently bad in any of the camera 3 masks.
No temperature dependence (at the 5
σ level) was found for camera 3 in the 1997 and 1998 datasets.
The main products of this analysis include new bad pixel masks and updated flat-field reference frames. The flat-field images originally delivered CDBS were processed with the old bad pixel masks as well as older versions of other calibration reference files. The fully updated flatfield images (improved bad pixel masks and including the latest linearity files and temperature dependant dark frames) will be re-delivered to CDBS for archiving along with the new bad pixel masks. These flat fields will also be more sensitive to the changing temperature of the instrument during the 1997 - 1999 time frame. All new calibration images will also be available on the NICMOS instrument web site.
During SMOV3B new analysis of the chips will be done to assess any changes in detector cosmetics. Since the new masks contain only the pixels which were consistently bad from 1997 to 1998, users are encouraged to retrieve darks from the archive to create their own bad pixel masks, specific to their observation parameters. Dithering observations provides an even better check and correction for bad pixels and is the recommended way of accounting for bad pixels.
Boeker, T., et al., “NICMOS Dark Current Anomaly: Test Results”, NICMOS-ISR-00-
002, March 2000
Boeker, T., et al., “Analysis, Results and Assessment of the NICMOS Warm-up Monitoring Program”, NICMOS-IST-99-001, February 1999
Sosey, M., Bergeron, L. E., “Grot in NICMOS cameras”, NICMOS-ISR-99-008,
September 1999
11