Instrument Science Report STIS 97-005
The STScI STIS Pipeline IV: Combining
REPEATOBS Data (Rev. A)
Paul Goudfrooij, Phil Hodge, Steve Hulbert, and Stefi Baum
March 1997
ABSTRACT
STIS data can be taken as a series of identical exposures, the so-called REPEATOBS
mode. We describe the procedure for combining these data in the STIS Pipeline for different instrument configurations. Purposes and shortcomings of the followed procedure are
identified.
1. Introduction
For both MAMA and CCD observations with STIS, the observer has the option that a
given exposure be repeated two or more times, all at the same pointing, aperture, and grating or filter, and with an identical exposure time. Use of this observation mode (designated
REPEATOBS) can be envisaged in case of, e.g., observers interested in obtaining timeresolved observations. These repeats may be specified during Phase II, using the NEXP=n
(with integer n > 1) option. The n exposures are to be associated with one another to produce a summed, calibrated image. The individual exposures will be fully calibrated in the
case of MAMA exposures, and through flatfielding in the case of CCD exposures. This
ISR describes the procedure on how to combine the repeated observations in the STIS
pipeline for the different instrument configurations. This ISR updates and replaces ISR 96017.
2. Purpose of combining REPEATOBS data
The primary purpose of combining a series of REPEATOBS observations in the context of the pipeline is to obtain a quick-look evaluation analysis of the data (e.g., for
inclusion in the paper products which are to be used by the General Observers and the
Contact Scientists). The method of combining data described here does not necessarily
produce optimal results for all scientific points of view (cf. Section 4).
1
3. Combining exposures to produce STIS datasets
The STIS pipeline will join together, to keep in one physical file, all the sequential
REPEATOBS data (images/spectra). In addition to calibrating each of the science exposures individually, the STIS pipeline will produce a single integrated image/spectrum from
the whole series of exposures. The separate exposures in the REPEATOBS set are written
as sequential triplets of image extensions in the science file by generic conversion (the
production of the flat-fielded individual exposures is performed if header keyword
EXPSCORR=PERFORM).
The format of the STIS FITS files is described in detail in Baum et al. (STIS ISR 95006). Briefly, data are stored in FITS IMAGE extensions in sets of three, each set consisting of the science data, an error array, and a data quality array. Such a set of three
extensions will be referred to in this ISR as a “imset”. The imsets are numbered using
keyword EXTVER. The number of these imsets in the uncalibrated FITS file, i.e., the
number of REPEATOBS subexposures, is given by the keyword NRPTEXP. The combined
subexposures are written to a FITS file with one imset.
The association of exposures to produce the final integrated dataset will be dependent
on the instrument configuration used:
1. CCD REPEATOBS observations will merely be treated as CRSPLITs and hence
combined by calstis-2 after calstis-1a (A-to-D conversion and bias overscan subtraction) has been performed (after having been combined, calstis-1 will then perform dark subtraction and flat field correction). In the case of spectroscopy, 2-D
extractions will be performed on the combined image.
2. MAMA imaging and spectroscopic observations will be processed as individual
exposures all the way through flat-fielding; MAMA spectroscopic observations
will be further processed through 2-D extraction. In the case of MAMA spectroscopy of point sources, the processing will continue through the 1-D extraction
scheme. After this, the individual 2-D rectified images will be simply added; the 1D spectra will be resampled (per order) to a common, linear wavelength scale and
co-added.
Summation of the rectified MAMA 2-D images (taken in imaging or spectroscopic
mode) will be performed in calstis-8. Summation of the individual extracted MAMA 1-D
spectra will be performed in calstis-13. This new task co-adds the n individual 1-D spectra
after rebinning them onto a common linear wavelength scale. Technical information on
calstis-8 can be found in Section 6. Calstis-13 will be described in a forthcoming Instrument Science Report.
The flow through the pipeline for REPEATOBS data taken with different instrument
configurations is tabulated in in Tables 1-2. Note that filetypes _sfl, _sx1, and _sx2
are restricted to combined output data (REPEATOBS for CCD or MAMA, or CRSPLIT
for CCD).
2
4. Shortcomings of the combination procedure
We emphasize again that the method of combining data described here does not necessarily produce results in an optimal form for all scientific points of view, and users of this
observation mode may well prefer to treat the individual (flat-fielded) exposures in a different way in order to realize their scientific goals.
The simple addition of the 2-D spectra after calstis-1 and calstis-2 (the latter only in
case of CCD exposures) has the potential problem of not always yielding good enough
results as the individual spectra might not be sufficiently well aligned with one another
(along as well as perpendicular to the dispersion), especially in the case of long, multiple
exposures. The degree of alignment of the spectral images will be dependent on the thermal stability of the MSM in orbit, and additional alignment correction can only be done if
sufficient wavecals will be taken in the process. For the start of Cycle 7, we assume that
the shifts in either direction will be small enough over the course of multiple orbits; if it
turns out that the thermal (in-)stability causes significant shifts during repeated observations, the procedure of combining REPEATOBS data described here (and the
corresponding calstis tasks) will have to be updated. An alternative combination procedure
for CCD REPEATOBS data is described in Appendix A.
5. Error handling for combined REPEATOBS data
For the reason of simplicity, all errors associated with the combined REPEATOBS
dataset will be calculated by summing (in quadrature) the errors from the individual exposures. The algorithm for computing the errors (and its obvious shortcomings, expanded
upon below) will be advertised to the community.
The error estimates for the combined data are not computed properly when merely
summing (in quadrature) the error estimates for each sub-group of the calibrated data (cf.
Section 6 below). This is because the errors in the calibrated data include contributions
from both Poisson noise and uncertainties in the reference data, so that combining them
would improperly reduce the errors contributed by the reference images by the square root
of NRPTEXP. Also, rebinning n individual 1-D spectra onto a common linear wavelength
scale (calstis-13) involves a slight smoothing of the input data and will thus result in correlated errors for the resulting combined 1-D spectra.
3
Table 1. Processing steps applied to individual and summed REPEATOBS data.
Detector
OBSTYPE
Pipeline steps
(individual exp.)
Pipeline steps
(combined exp.)
CCD
IMAGING
cs-1a, cs-1
cs-1a, cs-2, cs-1, cs-7
CCD
SPECTROSCOPY
cs-1a, cs-1
cs-1a, cs-2, cs-1, cs-7,
MAMA
IMAGING
cs-1, cs-7
cs-1, cs-7, cs-8
MAMA
SPECTROSCOPY
cs-1, cs-7, cs-6
cs-1, cs-7, cs-8 (no 1-D extr.
cs-1, cs-7, cs-6, cs-13 (1-D extr.)
Table 2. Data types contained in a given filetype for pipeline products.
File Suffix
Explanation
OBSTYPE
Individual Exp.
Summed Exp.
_raw
Uncalibrated
science data
IMAGING
SPECTROSCOPIC
CCD/MAMA
CCD/MAMA
_flt
Flat fielded
science data
IMAGING
SPECTROSCOPIC
CCD/MAMA
CCD/MAMA
_crj
Cosmic ray
rejected science
data
IMAGING
SPECTROSCOPIC
CCD
CCD
_sfla
Summed, flatfielded
science data
IMAGING
SPECTROSCOPIC
MAMA
MAMA
_x1d
1-D extracted
spectra
SPECTROSCOPIC
MAMA
_x2d
2-D extracted
spectra or rectified image
IMAGING
SPECTROSCOPIC
MAMA
MAMA
_sx1
1-D extracted
spectra from
summed REPEATOBS
SPECTROSCOPIC
MAMA
_sx2
2-D extracted
spectra or rectified image from
summed REPEATOBS
IMAGING
SPECTROSCOPIC
MAMA
MAMA
CCD
CCD
a. Files with _sfl suffix will only be a final pipeline product in case geometric correction will
not be applied to exposures taken in imaging (or spectroscopy) mode.
6. CALSTIS-8: Summation of rectified MAMA 2-D images
This calstis module sums MAMA 2-D images and spectra after they have been rectified using calstis-7. The module is performed if the image header switch RPTCORR is set
4
to PERFORM. The number of imsets to be added together is obtained by dividing the
value of keyword EXTEND by three. The resulting number is checked to be equal to the
value of keyword NRPTEXP; if not, a warning message is printed.
The first imset is read into memory from the input file; then, each subsequent
imset is read and added to the first. If the data have been converted from ADU to absolute flux (by calstis-7), then the data will be weighted by the exposure time for the addition
(the exposure time is typically identical for the images to be combined). This is the case if
the header keyword FLUXCORR is equal to COMPLETE. As each imset is read into
memory, and if FLUXCORR = COMPLETE, the SCI and ERR extension data are multiplied by the exposure time (keyword EXPTIME). The ERR extension data values are
squared to obtain the variance. The data are then combined (pixel-by-pixel) with the first
imset; the SCI data are added, the variance values in the ERR array are added, and the
DQ values are bitwise ORed. After all groups have been added together, the square root of
the ERR values is taken to convert back from variance to standard deviation. If FLUXCORR = COMPLETE, the SCI and ERR array values are divided by the sum of the
exposure times.
Statistics are computed on the summed data if keyword STATFLAG = T, and the
results are written to the SCI and ERR extension headers, like in other calstis modules
(e.g., calstis-1, calstis-7).
Header keywords are updated after summing the imsets. In the primary header of the
output file, keyword RPTCORR is set to “COMPLETE”, NEXTEND is set to three (i.e., one
imset), and history records are written. NRPTEXP retains its original value, in order to
correspond to the proposal instructions. In the SCI extension header, EXPTIME is set to
the sum of the EXPTIME values of all the input imsets. and EXPEND is set to the
EXPEND of the last input imset. Finally, the three arrays and headers in memory are
written to the output file.
Appendix A. An alternative combination procedure for CCD REPEATOBS data
In case the thermal conditions within STIS cause significant image shifts during
repeated observations, the _crj files (cf. Baum et al., STIS ISR 95-006; see also Table 2)
will contain a summed image of improperly aligned images or spectra. A possible alternative combination procedure for CCD data would be the following.
After calstis-1a (A-to-D conversion and bias overscan subtraction) has been performed
on the individual images, they could be combined pair-by-pair only (or, in case of a odd
number of exposures, by pairs and one set of three exposures) by calstis-2. This would
take care of removing the bulk of cosmic rays in each pair, while the objects in the images
would probably not have experienced a significant shift yet, so that the cosmic-ray rejec-
5
tion procedure in calstis-2 would not have erroneously identified actual objects as cosmic
rays.
After dark subtraction and flatfielding (calstis-1), the individual pairs could then run
through calstis-7 (2-D Image Rectification), which would (in the case of spectroscopy)
also align them with one another, both along and perpendicular to the dispersion1, so that a
procedure similar to calstis-8 (cf. Section 6 above) could sum the individual pairs to provide the final summed exposure (with suffix _sx2).
It is clear that this alternative procedure would require significant changes to the pipeline, e.g.:
1. calstis-2 would have to be changed to be able to deal with pair-wise cosmic ray
rejection, e.g. commanded by the value of keywords NRPTEXP or CRSPLIT;
2. The pairwise combined images would have to be accomodated into a FITS file
with multiple image extensions (three for each imset; an imset would now represent a pairwise summed image);
3. calstis-8 would need to be able to handle both MAMA and CCD data.
On-flight STIS commissioning data will teach us about the MSM stability, and thus
about the necessity of resorting to alternative combination procedures such as the one
described here.
1.
provided enough wavecals are available
6