STIS Instrument Science Report 97-02 The STScI STIS Pipeline VII: Extraction of 1-D Spectra Steve Hulbert, Phil Hodge, and Ivo Busko February 1997 ABSTRACT This report discusses the extraction of 1-D spectra performed by CALSTIS-6. CALSTIS-6 processes flat-fielded 2-D ACCUM images to produce a collection of one-dimension spectra. In the first iteration of CALSTIS-6 an unweighted extraction algorithm will be used. The resultant flux- and wavelength-calibrated spectra are stored in 3-D binary tables. 1. Introduction This reports presents a subset of the spectroscopic reductions planned for the STIS calibration pipeline. Following the original design described in “Plans for the STScI STIS Pipeline I: Overview” (Baum & Hodge, STIS ISR 95-006) this report deals with CALSTIS-6. This part of CALSTIS takes the flat-fielded science image and extracts onedimensional spectra. This report is organized as follows: • Section 2 summarizes the basic spectroscopic calibration processing and data flow as they apply to 1-D spectral extraction. • Section 3 describes details of the possible extraction schemes. • Section 4 describes the data quality and error propagation performed by CALSTIS-6. • Section 5 summarizes the output data format for spectra. • Section 6 lists specific programming requirements. 2. Extraction of One-Dimensional Spectra in CALSTIS-6 The extraction of one-dimension spectra from the flat-fielded (or CR-rejected) twodimensional image is a multistage process. The processing of spectroscopic ACCUM observations in CALSTIS-6 is summarized in Figure 1 on page 2. Figure 2 on page 3 depicts a representative spectrum (first-order or single echelle order) in a hypothetical 1 point-source input image. Figure 2 also provides the definition of the coordinate system used throughout this report. Figure 1: CALSTIS-6 Processing Flow. (_flt or _crj) Extract 1-D Spectra Subtract Background Assign Wavelengths Assign Fluxes Apply Heliocentric Wavelength Correction (_x1d) 2 Figure 2: The STIS coordinate system for spectroscopic observations AXIS 2 / Slit Position Spectrum AXIS 1 / Dispersion Note: For the case of observations using the cross-disperser, AXIS1 becomes Slit Position and AXIS2 becomes Dispersion. Read the input data The input data consist of a flat-fielded (_flt) or cosmic ray rejected (_crj) 2-D science image. In addition to reading the input image, CALSTIS-6 reads header keywords from the input image that describe the instrument configuration: DETECTOR, OPT_ELEM, APERTURE, CENWAVE. These keywords are, in turn, used to select the appropriate calibration data from the calibration reference tables. Table 1 lists the set of calibration switch keywords that controls calibration processing and the keywords containing the names of the supporting reference table. Table 1. CALSTIS-6 Calibration Switch and Reference File Keywords Calibration Step Calibration Switch Calibration Reference File(s) Needed Read small scale geometric corrections SGEOCORR SDSTFILE Extract 1-D spectra X1DCORR SPTRCTAB XTRACTAB Subtract background BACKCORR XTRACTAB Apply dispersion solution DISPCORR DISPTAB INANGTAB APDESTAB MOFFTAB ESPTAB Convert to absolute flux FLUXCORR APERTAB PHOTTAB Convert to heliocentric wavelengths HELCORR 3 SGEOCORR: Read Small-Scale Geometric Distortions If SGEOCORR is set to “PERFORM”, CALSTIS-6 corrects for small scale geometric distortions in the MAMA detectors. These distortions are not adequately removed by the dispersion or spectrum tracings. The corresponding reference file, SDSTFILE, contains the distortion offsets for each pixel in the MAMA image. For CALSTIS-6 processing, all AXIS2 positions in the input image must be modified by the AXIS2 small scale distortion deltas in the small scale distortion file. Since we do not interpolate pixels in the dispersion direction, no corrections are made to the AXIS1 positions prior to reading or extracting pixel values. Instead, the AXIS1 deltas are used to correct the assigned wavelengths. X1DCORR: Extract 1-D Spectra CALSTIS-6 assumes that a 1-D spectrum is present and that it can be extracted independently of the X1DCORR calibration switch. In the pipeline, CALSTIS-0 determines if CALSTIS-6 should be called based on the value of the X1DCORR calibration switch. Initially, X1DCORR will be set to “PERFORM” for observations using one of the echelle gratings and to “OMIT” for all other spectroscopic observations. Locate the spectrum The nominal location of the spectrum is specified in the spectrum trace table, SPTRCTAB and is given by (A1CENTER, A2CENTER) from this table. These coordinates are not constrained to be integers. The nominal position along the slit must be modified to include the previously updated position information found in the header. The possible position deltas are listed in Table 2. Table 2. Possible Offsets to Nominal Spectrum Center Keyword Description SHIFTA1 Offset determined from the associated WAVECAL caused by the limits of the MSM repeatability and by thermal drifts. SHIFTA2 MOFFSET1 MOFFSET2 Commanded offsets for MAMA observations to reduce degradation of MAMA detectors The nominal A2CENTER position of the spectrum (i.e., the position of the target in the AXIS2 or A2 or slit direction) is calculated as follows (where the variables are as described in Table 2. A2CENTER = A2CENTER + SHIFTA2 + MOFFSET 2 (Equation 1) 4 SPTRCTAB also contains the description of the distorted shape of the spectrum. The shape is stored as a vector consisting of pixel offsets (in the AXIS2 direction) relative to the nominal center of the spectrum (A2CENTER). This spectrum trace is used to find (and eventually to extract) the 1-D spectrum. See Table 7 on page 16 for a description of the columns in the spectrum trace table, SPTRCTAB. The location of the spectrum is improved by “searching” in the vicinity of the nominal location of the spectrum by performing a cross-correlation between the distortion vector and the input spectrum image. Figure 3 on page 6 schematically shows the cross correlation process. The search extends for ± n pixels around the nominal center, where n is read from the MAXSEARCH column in the XTRACTAB table. At each AXIS2 position in the search range (which differs from the nominal center by an integer number of pixels) a sum of the counts along the spectrum shape is formed. This sum is created by adding the value of one pixel’s worth of data at each of the AXIS1 pixel positions. The pixel extracted in the AXIS2 direction is centered on the spectrum position (A2CENTER + pixel offset) and may include fractional contributions from two pixels. Quadratic refinement (as shown in Figure 4 on page 6) is used to locate the spectrum to a fraction of a pixel. The final A2CENTER becomes: A2CENTER = A2CENTER + CRSCROFF (Equation 2) where CRSCROFF is the offset found during the cross correlation. If the cross correlation fails, the value of CRSCROFF is set to zero, a warning message is written to STDOUT, and the A2CENTER calculated prior to the cross correlation attempt is used as the location of the spectrum. CRSCROFF is written to the output science header. An alternate method for performing the cross correlation may be employed. In this case a 2-D template is created from the spectrum trace table. The cross correlation is carried out between the 2-D template and input image. Quadratic refinement is used as above to refine the position of the center of the spectrum to a fraction of a pixel. Extract the 1-D spectra The extraction of the spectrum is defined by a triplet of extraction “boxes” found in the reference table, XTRACTAB. Figure 5 on page 7 shows a schematic representation of the extraction boxes. For each pixel in the dispersion direction, CALSTIS-6 sums the values in the spectrum extraction box. The extraction box is nominally one pixel wide and x pixels tall, centered on the spectrum. (Remember that we determined the center of the spectrum in the previous step.) The height of the extraction box may include a fractional part of one or two pixels. In the case of a fractional pixel, CALSTIS-6 will scale the counts in the given pixel by the fraction of the pixel extracted. Thus, each pixel in the output spectrum consists of the sum of some number (or fraction) of pixels in the input image. 5 The extraction of the 1-D spectrum is shown in Figure 6 on page 8. See Table 9 on page 17 for a description of the columns in the extraction parameter table. Cross Correlation Range AXIS2 Pixel Figure 3: Repeated Extraction Strategy for “Finding” Spectrum Center Spectrum Trace Spectrum AXIS1 Pixel Figure 4: Quadratic Refinement Used to “Find” Actual AXIS2 Center of Spectrum Actual AXIS2 Position Sum Nominal AXIS2 Position AXIS2 Pixel The extraction of the spectrum allows for unweighted or optimal extraction. The extraction algorithm is selected based on the value of the reference table parameter, XTRACALG. This flag has possible values of UNWEIGHTED and OPTIMAL. See Sec- 6 tion 3 for a description of the extraction algorithms. The value of XTRACALG should be written to the header of the output spectrum data file. As a first cut, CALSTIS-6 will perform unweighted extraction of the 1-D spectra. Implementation of the optimal extraction algorithm in CALSTIS-6 will be done at a later time. At the end of the 1-D extraction step, a spectrum of gross counts/second is produced. The conversion to count rate is as follows: R G i = -----i t (Equation 3) where i is the pixel number, G i is the gross counts/sec in the spectrum at pixel i, R i is the extracted raw counts, and t is the exposure time. Figure 5: Extraction Box Geometry Background tilt angle Background Extraction Box Spectrum Extraction Box Spectrum Background Extraction Box 1 pixel BACKCORR: Subtract the Background If the calibration switch BACKCORR has the value of “PERFORM”, the background is calculated and subtracted from the extracted spectrum. The background is extracted above and below the spectrum and a function is fit to the background. The fitting function is restricted to a zeroth or first order polynomial fit and is a function of the AXIS2 position. The polynomial order, BACKORD, is read from the XTRACTAB table and written to a header keyword of the same name in the output spectrum table. Average background (c/s/pixel) values are calculated from each background bin. A complete accounting of the 7 fractional pixel contributions to the background is required. In the case of BACKORD=0, a simple average of the two background bins is computed. For BACKORD=1, a linear fit to the background values as a function of AXIS2 position is computed. A background value is then interpolated at the center of each pixel that contributes to the extracted spectrum. The background in the spectrum extraction box is totaled and subtracted from the sum of the spectrum box. The total background at each pixel in the output spectrum is saved and eventually written out to the output data file.ry AXIS2 Pixel Figure 6: Extracting the 1-D Spectrum Spectrum Extraction Box Length AXIS1 Pixel In general, it is not assumed that the background or sky is aligned with the detector pixels. To accommodate this, the definition of the background extraction apertures includes not only a length and offset (center-to-center) but also a linear tilt to assist in properly subtracting the background. This tilt, as shown in Figure 5 on page 7, is taken into account when calculating the average background in the background extraction boxes. DISPCORR: Assign Wavelengths Wavelengths are assigned using dispersion coefficients from the reference table, DISPTAB when the calibration switch, DISPCORR is set to “PERFORM”. IF DISPCORR is equal to “OMIT”, no wavelengths are assigned. Offsets introduced by using apertures other than a reference aperture (used to derive the coefficients) will be removed using coefficients in INANGTAB. Offsets introduced by the monthly MAMA dither offsets will be removed using coefficients in the MOFFTAB table. In the case of echelle 8 observations, small shifts introduced by the tilt of the spectral features will be removed using coefficients in the ESPTAB table. The DISPTAB table of dispersion contains coefficients for fits to the following dispersion solution: 2 2 s = A 0 + A 1 mλ + A 2 ( mλ ) + A 3 m + A 4 λ + A 5 m λ + A 6 mλ 2 (Equation 4) where, λ is the wavelength in Angstroms, s is the detector AXIS1 position, m is the spectral order, and A i are the dispersion coefficients. For each pixel in the AXIS1 direction, a wavelength will be calculated. First, any modification to the dispersion coefficients due to spectrum offsets must be made. Table 3 on page 9 lists the possible offsets and the appropriate corrections. For each integer value of s in the AXIS1 direction, a wavelength is calculated. The wavelength value must be solved for iteratively using the Newton-Raphson method (see, for example, Press, W.H. et al. Numerical Recipes in C. 1992. p. 362). Table 3. Modifications to the Dispersion Coefficients Caused by Offsets Correction Ref Table Incidence Angle INANGTAB Algorithm Definitions Ai = Ai + c1 s A0 = A0 + c2 s 2 Ai dispersion coefficients ci incidence angle coefficients s aperture offsets in the axis 1 direction calculated as difference of relative aperture centers (arcsec) MAMA Offsets MOFFTAB A i dispersion coefficients Ai = Ai + o1 x A0 = A0 + o2 x 2 oi MAMA offset coefficients x MAMA offset (MOFFSET1) (pixels) Echelle Spectrum Tilt ESPTAB A 0 = A 0 + y tan θ A i dispersion coefficients y axis 2 offset from nominal A2CENTER during spectrum locate process (pixels) θ 9 spectrum tilt angel FLUXCORR: Convert to absolute flux If FLUXCORR is set to “PERFORM”, the raw counts are corrected to FLAM (erg cm2 s-1 Å-1) using reference files PHOTTAB and APERTAB. Execution of the flux conversion calibration step requires that wavelengths have been assigned. Corrections for vignetting and echelle blaze are handled within the PHOTTAB reference files–at this time, no attempt will be made to decouple all of the various sources of response variation seen with STIS. The conversion to absolute flux is calculated as: hc F λ = -------------------------------------- N λ A HST R λ T λ λ∆λ (Equation 5) F λ is the calibrated flux at a particular wavelength, h is Planck’s constant, c is the speed of light, A HST is the area of the unobstructed HST primary mirror (45238.93416 cm2), R λ is the throughput of the STIS instrument configuration at a particular wavelength when a clear full aperture is in place, λ is a particular wavelength, ∆λ is the dispersion (Å/pixel)at a particular wavelength, N λ is the net count rate at a particular wavelength, and T λ is the aperture throughput at a particular wavelength. HELCORR: Apply heliocentric corrections to the wavelengths The correction of wavelengths to a heliocentric reference frame is controlled by calibration switch HELCORR and DISPCORR—if both switches are set to “PERFORM” then the correction is made. The functional form of the correction (shown below) requires the calculation of the heliocentric velocity (v) of the earth in the line of sight to the target. v λ helio = λ 1 + -- c (Equation 6) λ helio is the heliocentric wavelength, λ is a particular wavelength, v is the component of the velocity of the earth in the direction of the target, and 10 c is the speed of light. The derivatives of low-precision formulae for the Sun’s coordinates described in the Astronomical Almanac are used to calculate the velocity vector of the earth in the equatorial coordinate system of the epoch J2000. The algorithm does not include Earth-Moon motion, Sun-barycenter motion, nor light time correction from the Earth to the Sun. This value for the earth’s velocity should be accurate to ~0.025 km/sec during the lifetime of STIS. (Note: the uncertainty of 0.025 km/s is much less than the ~2.6 km/s resolution obtained with the STIS high dispersion echelle gratings.) The algorithm used in CALSTIS-6 should be identical to that used in CALSTIS-7. The value of heliocentric velocity, v , is written to the header keyword, HELIOVEL in the output header. 3. Extraction Algorithm Extraction of one-dimensional spectra for point sources can be described by: ∑ W sλ ( C sλ – Bsλ ) s N λ = -------------------------------------------W ∑ sλ (Equation 7) s where, N λ is the net extracted spectrum, W sλ is the weighting applied to each pixel in the spectrum, C sλ is the observed count rate at each slit position and wavelength, and B sλ is the fitted background (and/or sky) count rate at each slit position and wavelength. Unweighted In the case of unweighted extraction, the factor W sλ has the value of 1 at every pixel in the spectrum and is 0 outside of the spectrum. As implemented in CALSTIS-6, a rectangular extraction box centered in the slit direction on the spectrum is used to identify that portion of the 2-D spectrum to be summed to produce the output 1-D spectrum. Optimal The case of optimal extraction is described by Horne (1986) with additional work specific to handling the problematic case of distorted spectra by Marsh (1989). Optimal extraction has as its goal identifying a set of weights that maximizes the signal-to-noise 11 ratio in the extracted spectrum. The optimally-extracted spectrum is found by using Equation 8 where: P sλ W sλ = -------V sλ (Equation 8) P sλ is the spatial profile of the spectrum and V sλ is the variance of the background/sky subtracted data points. The two items of importance in this method are good error estimates for the original data points and an accurate knowledge of the spatial profile. Assignment of error estimates to the data points is handled in CALSTIS-1. The method for determining the best spatial profile is under investigation. 4. Data Quality and Error Propagation Data Quality Propagation The data quality value for any pixel in an extracted 1-D spectrum is the “or-ed” value of all data quality values that were used to produce the spectrum. At this time CALSTIS6 does not set any additional data quality values. Error Propagation CALSTIS-6 will propagate errors during the conversion of counts to count rate, the extraction of the 1-D spectra, the subtraction of the background, and the conversion to the absolute flux. In those cases where fractional pixels are coadded to form a sum, the relative error from each fractional part is combined to form the error associated with the sum. The errors associated with the calibrated data are part of the output data products. These errors are consistent with the highest level of flux calibration applied to the data (i.e., if the data are only processed as far as background subtracted count rates, the error estimates are those associated with the count rate values). In equations 9-11 below, the deltas ( ∆ ) represent the associated errors for a given measurement. The gross count rate spectrum is calculated as: R ∆R C ± ∆C = --- ± ------t t (Equation 9) C ± ∆C is the count rate, R ± ∆R is the raw counts, and t is the exposure time. 12 The net spectrum is calculated as: 2 1⁄2 2 N ± ∆N = ( C – B ) ± ( ∆C + ∆B ) (Equation 10) N ± ∆N is the net spectrum and B ± ∆B is the background. The flux calibrated spectrum is calculated as: F ± ∆F = N × S ± ( ∆N × S ) (Equation 11) F ± ∆F is the flux calibrated spectrum and hc S = -------------------------------------- is the flux conversion factor from Equation 6. A HST R λ T λ λ∆λ 5. Data Formats A complete spectrum consists of seven arrays containing: wavelengths, gross count rates, background count rates, net count rates, absolute fluxes, absolute flux error estimates, and data quality flags. A given spectrum will be written to one row of a FITS binary table extension. Additionally, each row will contain columns describing the spectral order and number of data points in the spectrum. Table 4 lists the table column labels, units, and datatypes. To accommodate a single spectrum per row, the table cells will contain arrays of numbers corresponding the physical quantities of the spectrum. NELEM gives the number of elements in these vectors for a given spectrum. Table 4. Column Definitions for Output Binary Table Column Name Datatype Units SPORDER I*2 NELEM I*2 WAVELENGTH R*8[n] Angstroms FLUX R*4{n] erg cm-2 s-1 Å-1 ERROR R*4{n] erg cm-2 s-1 Å-1 DQ I*2 GROSS R*4[n] counts/sec BACKGROUND R*4{n] counts/sec NET R*4[n] counts/sec Table 5 summarizes the details of the FITS files created by CALSTIS-6. The mapping is effectively one output table extension to each input group of image extensions. 13 Table 5. Output spectrum FITS definition Output Source Primary header Primary header of input science data file Primary data empty 1st binary table extension header image extension header from 1st science extension [sci,1] PLUS definition of binary table 1st table extension data extracted spectra from 1st science extension nth table extension data extracted spectra from nth science extension 6. Programing Requirements I/O Routines that perform I/O should be isolated (in code) from routines that perform numeric processing. HSTIO should be used to access STIS images. Calling Sequence cs6 input output -t -c a2center -r maxsearch -x extrsize -b1 bk1size -b2 bk2size -o1 bk1offst -o2 bk2offst -k bktilt -n backord -a xtracalg where, cs6 is the name of the CALSTIS-6 executable input is the name of the input FITS file output is the name of the output FIT file, -t controls the printing of OPUS style time stamps. -c a2center is the nominal AXIS2 coordinate of the center of the spectrum to be extracted, -r maxsearch is the cross correlation search range, -x extrsize is the size of spectrum extraction box, -b1 bk1size is the size of background extraction box #1, -b2 bk2size is the size of background extraction box #2, -o1 bk1offst is the offset of background extraction box #1 from spectrum extraction box, -o2 bk2offst is the offset of background extraction box #2 from spectrum extraction box, -k bktilt is the angle of background extraction boxes w.r.t. axis2, 14 -n backord is the order of polynomial fit to background, and -a xtracalg is the extraction algorithm. Parameters read from the command line override the corresponding values read from any of the reference tables. 7. Acknowledgments Special thanks to Don Lindler for many discussions and whose “Space Telescope Imaging Spectrograph: Science Data Management and Analysis Requirements Document” provided a valuable road map. 8. References Baum, S. and Phil Hodge. July 1995. “Plans for the STScI STIS Pipeline I: Overview.” STIS ISR 95-006. Horne, K. 1986. Pub.A.S.P., 98, 609. Marsh, T. 1989. Pub.A.S.P., 101, 1032. Lindler, D. March 1995. “Space Telescope Imaging Spectrograph: Science Data Management and Analysis Requirements Document.” Advanced Computer Concepts, Inc. 15 9. Calibration Reference Files and Tables Below are listed the calibration reference tables used by CALSTIS-6. Only those columns used by CALSTIS-6 are included. For more details on the reference tables refer to ICD-47. Table 6. DISPTAB– Dispersion Coefficients Table Column Name Data Type OPT_ELEM C*8 CENWAVE I*2 SPORDER I*2 spectral order REF_APER C*12 reference aperture A2CENTER R*4 NCOEFF I*2 number of coefficients in dispersion solution COEFF R*8[10] dispersion solution coefficients Units Description optical element in use Angstrom central wavelength pixel nominal axis2 coordinate for center of spectrum Table 7. SPTRCTAB–1-D Spectrum Trace Table Column Name Data Type OPT_ELEM C*8 CENWAVE I*2 SPORDER I*2 spectral order NELEM I*2 number of data points in spectrum A2DISPL R*4[1024] pixel displacement along axis2 A1CENTER R*4 pixel nominal axis 1 coordinate of center of spectrum A2CENTER R*4 pixel nominal axis 2 coordinate of center of spectrum Units Description optical element in use Angstrom central wavelength Table 8. INANGTAB–Incidence Angle Correction Table Column Name Data Type OPT_ELEM C*8 CENWAVE I*2 SPORDER I*2 spectral order NCOEFF1 I*2 no. coefficients in IAC solution for first term Units Description optical element in use Angstrom central wavelength 16 Column Name Data Type COEFF1 R*8[8] incidence angle correction coefficients for first term NCOEFF2 I*2 no. coefficients in IAC solution for second term COEFF2 R*8[8] incidence angle correction coefficients for second term Units Description Table 9. XTRACTAB–1-D Extraction Parameter Table Column Name Data Type APERTURE C*16 aperture in use OPT_ELEM C*8 optical element in use CENWAVE I*2 SPORDER I*2 EXTRSIZE R*4 pixel size of spectrum extraction box BK1SIZE R*4 pixel size of background extraction box #1 BK2SIZE R*4 pixel size of background extraction box #2 BK1OFFST R*4 pixel offset of background extraction box #1 from spectrum extraction box BK2OFFST R*4 pixel offset of background extraction box #2 from spectrum extraction box BKTILT R*4 degrees angle of background extraction boxes w.r.t. axis2 BACKORD I*2 order of polynomial fit to background XTRACALG C*12 extraction algorithm MAXSEARCH I*2 Units Description Angstrom central wavelength spectral order pixel maximum search size for cross correlation Table 10. MOFFTAB–MAMA Offset Correction Table Column Name Data Type OPT_ELEM C*8 CENWAVE I*2 SPORDER I*2 spectral order NCOEFF1 I*2 no. coefficients in MAMA offset solution for first term COEFF1 R*8[8] MAMA offset correction coefficients for first term NCOEFF2 I*2 no. coefficients in MAMA offset solution for second term COEFF2 R*8[8] MAMA offset correction coefficients for second term Units Description optical element in use Angstrom central wavelength 17 Table 11. ESPTAB–Echelle Spectrum Tilt Table Column Name Data Type OPT_ELEM C*8 CENWAVE I*2 SPORDER I*2 SPTILT R*4 Units Description optical element in use Angstrom central wavelength spectral order degrees angle of spectrum w.r.t. axis2 Table 12. APERTAB–Aperture Throughput Table Column Name Data Type APERTURE C*16 aperture in use NELEM I*2 number of data points in throughput array WAVELENGTH R*8[65535] THROUGHPUT R*8[65535] Units Description Angstrom reference wavelength total system throughput at each wavelength Table 13. PHOTTAB–Photometric Conversion Table Column Name Data Type OPT_ELEM C*8 CENWAVE I*2 SPORDER I*2 spectral order NELEM I*2 number of data points in throughput array WAVELENGTH R*8[500] THROUGHPUT R*8[500] total system throughput at each wavelength ERROR R*8[500] error associated with THROUGHPUT Units Description optical element in use Angstrom central wavelength Angstrom reference wavelength Table 14. APDESTAB–Aperture Description Table Column Name Data Type APERTURE C*16 OFFSET1 R*4 arcsec offset from nominal position in axis1 OFFSET2 R*4 arcsec offset from nominal position in axis2 Units Description aperture in use 18