Instrument Science Report NICMOS-98-001
Cosmic Ray Persistence in NICMOS Data
Joan Najita, Mark Dickinson, and Sherie Holfeltz
February 25, 1998
ABSTRACT
Examination of deep NICMOS exposures of faint galaxy fields reveals that a large fraction
are affected by image persistence due to cosmic rays. This results in a significant reduction in data quality and sensitivity. The presence of large numbers of persistent cosmic ray
images is found to be strongly correlated with the timing of the observation relative to
HST passages through the SAA. Observations taken very soon after emerging from the
SAA, and following long SAA passages, are the most severely affected. This correlation
was confirmed by the inspection of a large number of darks taken for the NICMOS calibration program. In this report, we discuss the implications for the overall quality of NICMOS data, quantify the tradeoff between data quality and timing of observations relative
to SAA passages, and discuss possible strategies to minimize the problem.
1. Introduction.
The illumination of NICMOS arrays by a bright source of infrared photons often
results in image persistence, i.e., the continued accumulation of charge even after illumination has ceased (see ISR NICMOS-97-023, NICMOS-97-024). A similar effect can
arise from exposure to cosmic rays.
2. Degraded Faint Galaxy Imaging from Cosmic Ray Persistence.
In order to investigate the impact of cosmic ray persistence on NICMOS data, we
examined 160 data frames representing 19 visits and19 different targets from GO programs 7454 and 7714. The data were taken with NIC2 using MULTIACCUM SAMPSEQs over a period of several months. Of the entire set of data for these programs, only
the longer exposures (the MIF512 and MIF1024 exposures) were included in the analysis.
Several frames from each program were excluded from the analysis due to other problems
with the data, e.g., large quadrant-dependent DC offsets (“pedestal”). In a given orbit,
there were typically three exposures of a single target field (1 MIF512 512s + 2 MIF1024
1
1024s). After processing the data through CALNICA, a quadrant-by-quadrant constant
offset was applied to produce a uniform background level across the entire array.
In order to quantify the quality of the data, the noise level in the final calibrated image
was measured as an iteratively clipped RMS. We chose to quantify the effect of cosmic ray
persistence by measuring the noise in the image rather than the excess count rate that persistence contributes for several reasons. Since an excess count rate can also arise from
different effects such as “pedestal”, elevated dark current, etc., the spatial noise in the
image is a more direct, more robust measure of cosmic ray persistence. In addition,
although the subtraction of a quadrant-dependent offset may partially compensate for
some of these other effects (e.g., “pedestal”), the correction is potentially large and subjective, and may introduce sources of error that directly affect measures of an excess count
rate such as the image median or mean.
Since the measured RMS has both components that depend on the integration time
(e.g., thermal background and cosmic ray persistence) and components that are independent of integration time (e.g., read noise), the measured values for the MIF512 and
MIF1024 sequences are on different relative scales. To enable a relative comparison of the
data quality for all the data, the RMS values for the MIF512 and MIF1024 data were normalized separately by the typical RMS value for “good” data obtained with each
sequence, i.e., the normalization constant for each sequence is the median RMS value for
data obtained in orbits unaffected by cosmic ray persistence. As a result, the RMS values
for “good” data taken with each sequence have an RMS of ~1.
We find a significant variation of the data quality between different orbits and between
exposures within a given orbit (see Figures 1-3). Visual inspection of the calibrated data
reveals that a large fraction of the exposures show residual cosmic ray tracks that have survived cosmic ray rejection in CALNICA. That these are persistent cosmic ray images is
verified by the examination of consecutive samples within a MULTIACCUM sequence as
well as consecutive MULTIACCUM exposures. These result from cosmic ray hits that
occured prior to the start of the integration. A visual comparison of these figures shows the
impact of cosmic ray persistence on NICMOS observations of faint objects. A comparison
of the image histograms for the data shown in Figures 1 and 2 (see Figures 4 and 5), shows
the magnitude and character of the effect. Cosmic ray persistence is a source of non-gaussian noise, potentially affecting a large number of pixels, that produces a skewed
background distribution with a long tail toward positive values.
The histogram of RMS values for the entire set of final calibrated images (Figure 6)
shows the range in data quality produced by cosmic ray persistence. Different shading patterns are used to separately indicate the results for orbits that are “dirty” or “clean” i.e.,
those that show (or do not show) persistent cosmic rays. For the "dirty" orbits, the frames
were given sequence numbers indicating whether they are the 1st, 2nd or 3rd exposure of
the "dirty" orbit. The decrease in the mean RMS with sequence number indicates the grad-
2
ual decay of cosmic ray persistence in subsequent exposures. Figures 7-9 show a specific
example of this effect.
The degree to which individual images are affected by cosmic ray persistence is
strongly correlated with the timing of the observations relative to passages through the
South Atlantic Anomaly (SAA). Figure 10 shows the normalized RMS in each image as a
function of the time spent in SAA Model Contour 2 (top left), time spent in Model Contour 5 (top right), time since emergence from Contour 2 (bottom left), and time since
emergence from Contour 5 (bottom right). The RMS is markedly larger in observations
following longer or deeper (i.e., Contour 2-crossing) passages through the SAA, and in
those taken more recently after emergence from the SAA. In particular, exposures taken
after spending at least 10 minutes in Contour 2 and within 25 minutes following emergence from Contour 5 are severely affected. The information in the following table may be
used to calibrate these results, as well as the RMS values in Figure 6, against the visual
impression of Figures 1-3 and 7-9. The table gives for each figure, the integration time,
image RMS, the time spent in Contours 2 and 5, and the times since emergence from SAA
Contours 5 and 2.
Table 1:
Figure
Tint
RMS
Tin 2
Tin 5
Tsince 2
Tsince 5
1
1024s
1.004
---
---
> 1 hr
~ 1 hr
2
1024s
2.452
13 m
28 m
34 m
23 m
3
1024s
3.108
18 m
25 m
16 m
12 m
7
512s
1.536
12 m
28 m
21 m
10 m
8
1024s
1.381
12 m
28 m
31 m
19 m
9
1024s
1.133
12 m
28 m
49 m
37 m
Among the 160 faint galaxy data frames, 90 (56% of all data) were obtained in orbits
affected by cosmic ray persistence. To obtain a rough indication of the impact of cosmic
ray persistence on the data quality for the entire data set, we can use the largest RMS measured during orbits that are unaffected by cosmic ray persistence as a conservative relative
standard; the RMS for 1/3 of all the data is worse than this. Although HST crosses SAA
Contour 5 ~9 times per day (~56% of all orbits) and Contour 2 ~7 times per day (~44% of
all orbits), in the present data set, we find that >50% of the data frames were taken in Contour 2-crossing orbits. Thus, the results for the faint galaxy data indicate that NICMOS
science observations experience more frequent degradation of data quality compared to
the expectation for unbiased scheduling of NICMOS observations. This result is possibly
due to scheduling restrictions for other HST instruments such as STIS.
3
3. Corroboration from Dark Calibration Data.
The strong correlation of cosmic ray persistence with SAA passage was confirmed by
visual inspection of the paper products for the NICMOS calibration darks (110 data
frames for programs 7793, 7794, 7796, 7797, and 7798). The dark data includes data taken
in 7 different MULTIACCUM sequences (STEP256, STEP128, MIF2048, SPARS64,
SPARS256, MIF512, MIF1024). For each program, the three NICMOS cameras were typically operated in parallel. For each frame that appeared to be affected by cosmic ray
persistence, the data for all three cameras were processed through CALNICA, including
subtraction of the appropriate synthetic dark, and a quadrant-by-quandrant offset applied
to produce a uniform background level across the array. An iteratively clipped RMS was
then measured from the final calibrated image.
Figure 11 shows the resulting RMS values (in counts). Note that since the RMS values
were not normalized to their respective mean values for “good” data obtained with the
same MULTIACCUM sequence, they are on different relative scales. Nevertheless, the
trend seen in the faint galaxy data is apparent here as well. Cosmic ray persistence is more
significant in exposures that occured after long or deep (i.e., Contour 2-crossing) SAA
passages and after a similar interval of time following emergence from the SAA.
Approximately half of the MULTIACCUM dark exposures were taken immediately
following ACCUM exposures with NREAD=10; the remainder were taken without a preceding ACCUM. (Almost all of the MULTIACCUMs were intended to be preceded by an
ACCUM, but in practice, some fraction of the preceding ACCUMs were scheduled before
the SAA passage rather than immediately preceding the MULTIACCUM. This illustrates
the importance of using the SEQ NON-INT special timing requirement in this situation.)
In Figure 11, different symbols are used to show the RMS values for exposures with preceding ACCUMs and those without. Although the present data are highly inhomogeneous
(representing different MULTIACCUM sequences, different trajectories through the SAA,
and different times since emergence from the SAA), this preliminary assessment of the
data indicates that inserting ACCUMs with NREAD=10 does not produce a significant
reduction in the image RMS. This general impression is consistent with the results of
comparing the few available MULTIACCUMs that were taken under similar conditions
(using the same sample sequence, similar time spent in the SAA, and similar time since
emergence from the SAA) but with and without preceding ACCUMs.
4. Implications and Actions.
LOWER DATA QUALITY. Cosmic ray persistence significantly reduces NICMOS data
quality, with a negative impact on science, especially for programs that involve faint object
imaging and/or depend upon an expected level of photometric accuracy. The flux level at
which data quality is affected is not particularly low. For example, the target galaxy in Figures 1-3 has an integrated K magnitude of 17 in a ~4” diameter aperture. Persistence
4
contributes non-gaussian, spatially correlated noise (multi-pixel events), that is positionally correlated in subsequent exposures. These features make it difficult to remove in postprocessing. Observational programs that do not utilize a large number of independent
dither positions per field are especially vulnerable.
DIFFICULT TO CALIBRATE. At the present, it is unclear how to calibrate out the effect of
cosmic ray persistence. It is also unclear whether persistence decays as a stronger function
of time or of the number of detector reads subsequent to the cosmic ray hit. This issue is
difficult to quantify with the present data. Since the persistent count rate is low (comparable to the count rate from faint galaxies), long integration times (> ~ 200s) are needed to
measure the persistent signal above the read noise. When the total integration time spent
on the galaxies is divided into 200s intervals, the uncertainty in each RMS measurement is
large enough that the number of data samples (200s time intervals) is insufficient to distinguish whether the decay is primarily a function of reads or of time. If it were possible to
quantify the persistence count rate as a function of time and/or detector reads, it might be
possible to remove the excess counts (but not the excess noise) from each pixel, for
improved data quality. To implement this approach, information on the read history of the
detector and cosmic ray energies and arrival times would probably be required. It is
unlikely that the latter information would be available.
USE OF ACCUMS. If persistence due to cosmic ray hits is physically similar to persistence due to bright photon illumination (i.e., if charge is deposited in the detector in a
similar manner) it may be possible to hasten the reduction of cosmic ray persistence by
flushing the array using ACCUMs with large numbers of reads (e.g., several NREAD=25
exposures) prior to the first science exposure. This idea is based on ground-based tests
conducted by the NICMOS IDT of the response of NICMOS flight spare detectors to illumination by a bright source of infrared photons. However, persistence due to cosmic rays
may be different enough in nature from photon persistence that multiple reads may not be
as effective as the ground-based tests would indicate.
Several factors imply the need for large data sets in order to quantify the effectiveness
of multiple reads in reducing persistence from the examination of archival data alone. In
addition to the large time intervals required to measure the persistence signal, the non-uniform way in which archival data are acquired (different trajectories through the SAA,
different intervals of time and/or number of reads between emergence from the SAA and
the beginning of the exposure, etc.) introduces additional complications. The preliminary
indication from the analysis of the dark data presented here is that the number of reads
required for significant persistence reduction is probably quite large (NREAD>10, i.e.,
>20 detector reads in MULTIACCUM mode).
Several of these complications could be reduced or eliminated by conducting an onorbit test of the effectiveness of multiple reads in reducing cosmic ray persistence. The test
could be structured along the same lines used in the ground-based test of photon persis-
5
tence conducted by the NICMOS IDT, and using the SAA itself as the source of cosmic
rays. The test would involve multiple passes through the SAA along the same trajectory in
order to obtain as uniform a flux of cosmic rays as possible. Following emergence from
the SAA, a pattern of MULTIACCUMs interspersed with ACCUMs would be taken. The
MULTIACCUMs would be used to measure the decay in the persistence rate, and the
ACCUMs would be used to introduce a variable (and potentially large, >50) number of
reads while maintaining a low data rate. In each pass through the SAA, a different number
of reads (e.g., 0, 25, 2x25, etc.) for the ACCUMs would be used in order to determine
whether reads are effective and how many are needed to produce a significant reduction in
cosmic ray persistence. The timing of the MULTIACCUMs relative to the SAA would be
the same in each passage. Depending on the stability of the cosmic ray flux in the SAA as
a function of time, multiple passes through the SAA at a given number of ACCUM reads
may be required.
SCHEDULING. While it is unclear how cosmic ray persistence can be removed from
NICMOS data, the circumstances under which the effect arises are well-understood, and
therefore it is known how the effect can be minimized. Since SAA-crossing orbits have a
significant impact on data quality, it is desirable from a data quality perspective that these
orbits are avoided in scheduling observations that require deep imaging, low spatial noise,
and/or a specific level of photometric accuracy (e.g., the scheduling strategy adopted for
the NICMOS GTO observations of the HDF). Although shallow SAA-crossings (e.g.,
Contour 5 only) have a much more limited impact than Contour 2-crossing orbits, in practice, since a large fraction (~7/9) of SAA-impacted orbits cross Contour 2, the distinction
between SAA-crossing orbits that cross Contour 2 and those that do not may not be very
meaningful in terms of scheduling. Since observations requiring deep imaging, low spatial
noise, and/or a specific level of photometric accuracy represent a large fraction of ongoing
NICMOS programs, at the present the general use of non-SAA orbits for NICMOS observations appears to be unfeasible given the desire to complete as many approved NICMOS
programs as possible within the cryogen lifetime. The more general use of non-SAA
orbits for NICMOS observations may be an option for the future (e.g., Cycle 9).
OBSERVING STRATEGY. Given that the circumstances under which cosmic ray persistence occurs are well understood, observers may wish to consider its potential impact on
their observing programs in devising an observing strategy. For example, frequent dithering between a fairly large number of dither positions (> 4) is advantageous, although this
would come at the cost of reducing the integration time at a given dither position. Because
it is clear that data quality will be preferentially reduced following emergence from the
SAA compared to observations taken further downstream, programs that, for example,
involve imaging in multiple filters may wish to alternate the filter that is observed first in a
sequence.
6
5. Conclusion.
Given the significant impact of cosmic ray persistence on data quality and the difficulty of calibrating out this effect, the most effective solution to this problem appears to be
to avoid it altogether: from a data quality perspective, it is desirable that programs requiring deep imaging, low spatial noise, and/or a specific level of photometric accuracy be
scheduled in non-SAA impacted orbits. While this will increase the demand for non-SAA
orbits, at the present there appears to be no other viable approach that can deliver the
expected level of data quality in the allocated integration time. While the general use of
non-SAA orbits for NICMOS observations appears to be unfeasible at the present time,
this may be an option for the future (e.g., Cycle 9).
7
target galaxy
Figure 1. An example of a “clean” NIC2 image of a faint galaxy field.
Figures 1-3 show images of the same field taken during different orbits and
at different dither positions. These figures show the wide range in data
quality produced by cosmic ray persistence and illustrate the impact of cosmic ray persistence on faint galaxy imaging. For reference, the radio galaxy
in the field has an integrated K magnitude of 17 in a ~4” diameter aperture.
In this figure, the normalized RMS in the 1024s MIF1024 exposure is
1.004 (see text).
8
target galaxy
Figure 2. A lower quality NIC2 image of the same faint galaxy field
shown in Figures 1 and 3, but taken during a different orbit and at a different dither position. In this figure, the normalized RMS in the 1024s
MIF1024 exposure is 2.452 (see text).
9
target galaxy
Figure 3. A very low quality NIC2 image of the same faint galaxy field
shown in Figures 1 and 2, but taken during a different orbit and at a different dither position. In this figure, the normalized RMS in the 1024s
MIF1024 exposure is 3.108 (see text).
10
Figure 4. Image histogram for the image shown in Figure 1.
11
Figure 5. Image histogram for the image shown in Figure 2.
12
"Clean" orbit
80
"Dirty" orbit, 1st exposure
"Dirty" orbit, 2nd exposure
60
"Dirty" orbit, 3rd exposure
40
20
0
0
1
2
3
Figure 6. Histogram of the normalized RMS values for the entire set of faint
galaxy images showing the large range in data quality produced by cosmic ray
persistence.
13
Figure 7. The first of 3 consecutive NIC2 images (Figures 7-9) of another
faint galaxy field taken during the same orbit and at different dither positions.
This sequence of images illustrates the gradual decay of persistent cosmic ray
features. Persistent images from cosmic rays (e.g., “streaky” features) and
bright stars seen in this figure appear at the same pixel position in Figures 8
and 9 but at reduced amplitude. In this figure, the normalized RMS in the 512s
MIF512 exposure is 1.536.
14
Figure 8. The second of 3 consecutive NIC2 images (Figures 7-9) of
another faint galaxy field taken during the same orbit and at different dither
positions, illustrating the gradual decay of persistent cosmic ray features.. In
this figure, the normalized RMS in the 1024s MIF1024 exposure is 1.381.
15
Figure 9. The third of 3 consecutive NIC2 images (Figures 7-9) of another
faint galaxy field taken during the same orbit and at different dither positions,
illustrating the gradual decay of persistent cosmic ray features. In this figure,
the normalized RMS in the 1024s MIF1024 exposure is 1.133.
16
Figure 10. The normalized RMS in each image as a function of the time spent in SAA
Contour 2 (top left), time spent in Contour 5 (top right), time since emergence from Contour 2 (bottom left), and time since emergence from Contour 5 (bottom right). The RMS is
markedly larger in observations following longer passages through the SAA and those
taken more recently after emergence from the SAA.
17
Figure 11. The RMS in counts in each image as a function of the time spent in SAA
Contour 2 (top) and time since emergence from Contour 2 (bottom). The RMS is not normalized for the 7 different MULTIACCUM sequences that make up this plot.
Nevertheless, the same trend is clear: the RMS is markedly larger in observations following longer passages through the SAA and those taken more recently after emergence
from the SAA.
18