Instrument Science Report STIS 96-013
The STScI STIS Pipeline III:
TIMETAG Data
Stefi Baum, Nelson Zarate, and Phil Hodge
July 22, 1996
ABSTRACT
In this ISR we describe the plans for the processing of TIMETAG mode data taken with the
MAMA detectors, both by generic conversion and calstis. We describe the data formats employed for the TIMETAG event stream, and we describe a stand alone STSDAS
task inttag which users can use to integrate a specified time range of their event data
into an image. Subsequently, this image can be processed through the calstis reduction
software to produce a fully calibrated spectrum or image.
1. Introduction
The MAMA detectors on STIS have two operating or data taking modes, ACCUM and
TIMETAG. In ACCUM mode, the photon events are accumulated in memory and an
image of total counts received at each pixel integrated over the exposure is transmitted to
the ground. In TIMETAG mode each individual event is recorded as a detector x position,
detector y position, and time. For more information about the commanding and operation
of TIMETAG mode see also STIS ISR 95-11.
In this ISR we describe the processing of TIMETAG data in the OPUS calstis calibration pipeline (see also STIS ISR 95-06 for an overview of the calstis pipeline). The
plans for reduction of TIMETAG data within the Pipeline and in STSDAS are now as
follows:
•
Generic Conversion will produce a cleaned, Doppler corrected (as required for
medium and high resolution spectroscopic modes) event stream in the
rootname_tag.fits file. This file will consist of a primary header and two binary table
extensions, the first containing a list of the events themselves (EXTNAME=EVENTS)
and the second containing a list of good time intervals (EXTNAME=GTI.
•
Generic Conversion will integrate the TIMETAG event stream in time to produce a
lowres ACCUM mode image, correcting for the Doppler motion of the spacecraft, as
needed, and integrating over only the good time intervals.
1
•
A stand-alone STSDAS task, inttag will be provided for users, which takes as
input the name of the file containing the TIMETAG stream and a time interval, or
range of time intervals, over which to integrate. The output of inttag will be
ACCUM mode images which can be subsequently processed by the user through
calstis, to produce a calibrated image or spectrum over that time interval (or set of
time intervals).
The remainder of this ISR is organized as follows.
•
In Section 2, “TIMETAG Data As it is Received by Generic Conversion” on page 2,
we describe the data received by generic conversion.
•
In Section 3, “Science Outputs of Generic Conversion - Structure of The Event File”
on page 3, we describe the structure of science data outputs of Generic Conversion for
TIMETAG data.
•
In Section 4, “Generic Conversion Creation of the Event Stream and ACCUM image”
on page 6, we describe how those uncalibrated science files are created.
•
In Section 5, “INTTAG - A Stand-Alone STSDAS Task for TIMETAG data” on
page 7, we describe the STSDAS task users will have available to work with the
TIMETAG event stream.
2. TIMETAG Data As it is Received by Generic Conversion
In TIMETAG mode, each photon event is recorded individually and is transmitted to the
ground as a 32 bit word. The first bit in the word is used as an indicator to say whether the
information in the line contains event data, a coarse time, or is an integration interrupt signal. This bit is followed by a spare (unused) bit. The next 8 bits contain the ‘fine’ time in
units of 125 microseconds, the next 11 bits give the x detector location of the event and the
last 11 the y detector location of the event. No on-board Doppler compensation is provided in TIMETAG mode.
Every 32 msec within which an event is recorded, a coarse time word is inserted into the
data stream. The fine time words are referenced to this coarse time (i.e., are the time since
the last coarse time). TIMETAG mode allows for continuous counting of events at rates up
to ~25,000 count/sec over periods of 40 minutes. At lower rates, much longer exposure
times are possible, however at counting rates above ~25,000 count/sec only very short
(<2.5 minute) exposures are possible. Finally, at counting rates of upwards of 50,000
counts/sec in TIMETAG mode the MAMA microprocessor is unable to keep up with the
events and uncorrectable non-linearity sets in. Subarrays can and may frequently be used
in TIMETAG mode since the counting rate and corresponding exposure time limits apply
only to the counts coming from within the specified subarray. See STIS ISR 95-011 for the
details of the science uses, operation and limits of TIMETAG mode.
If the rate at which counts are received is greater than the expected rate (or specifically is
greater than the rate at which the MAMA memory buffer is emptied), the on board memory will fill up and the integration will be interrupted until the onboard memory has been
2
read out, at which point the integration will resume. When there is such a ‘pause’ in data
taking, a flag word (‘faaaaaaa’ hex ) will be inserted into the data stream immediately preceding the next value. If the rate at which counts are received is less than the expected rate,
the data stream will have fill data (5569) inserted in it. This is because the ground system
predetermines the rate at which data is transferred from STIS buffer to the HST Data
Recorder; if fewer counts are actually received, STIS adds fill.
As with raw ACCUM mode data, raw TIMETAG data (when received in OPUS) has an
internal header, identifying the exposure, other programmatic’, details about the volume of
data in the dump and other important information about the exposure. Unlike ACCUM
mode data, however, the data from a single TIMETAG exposure can be transferred from
buffer to HST Data Recorder in batches. In that case each batch will have its own internal
header, however the internal header for each batch from a single exposure will be identical
to the that in the first batch. The batch internal headers may also have an identification running number (e.g., dump1, dump 2) which can be used by generic conversion to join
together the data from the multiple batches for a single exposure into a single output
product.
To summarize, the data stream received by generic conversion will, in general, contain a
stream of x,y,t events interrupted at 32 msec intervals by a coarse time word. If there was a
pause in data taking due to buffer overflow, a flag word indicating that the exposure paused
will be inserted in the stream; if the data rate was lower than expected the stream will have
zeros inserted as fill. The data from a single exposure may be contained within several segments, each having its own internal header; generic conversion will piece those together to
output a single time tag data stream from the exposure.
3. Science Outputs of Generic Conversion - Structure of The Event File
The science outputs of generic conversion for the time tag science data stream will be one
fits file, rootname_tag.fits, containing the event stream, and a second ACCUMulated
image file (rootname_sci.fits), identical in structure to the science file which would have
been produced were the observation taken in ACCUM mode.1The event file will contain a
primary header with no data, followed by two BINTABLE extensions.The primary header
file contains the observation defining keywords, and will be identical in structure to the
primary header of an ACCUM mode image. The first binary table contains the science
data as an event stream and has EXTNAME=EVENTS, the second contains a listing of the
good time intervals for the TIMETAG exposure and has EXTNAME=GTI. This file has
1. TIMETAG science exposures will always be single science exposures, will always be accompanied by a support file (rootname_spt.fits) containing the standard header packet and engineering data
stream, and, for spectroscopic observations, will be accompanied by an ACCUM mode wavecal exposure in the rootname_wav.fits file (see STIS ISR 95-06). Here we refer only to the science file itself.
3
been designed to be compatible with the existing QPOE software for processing event
lists, and can be read into a QPOE file using the fits2qp task in IRAF.
Table 1. Structure of the rootname_tag.fits file produced by generic conversion
unit
contents
primary header
primary header (observation defining) keywords
bintable extension header
EXTNAME=EVENTS header keywords
bintable extension data
event stream
bintable extension header
EXTNAME=GTI header keywords
bintable extension data
start and stop time of good time intervals
Event Table
The event table is a FITS BINTABLE extension with three or four columns. The time of
each event is given in the TIME column. The data type of this column is 32-bit integer,
and the actual value in the table is in units of clock ticks (125 microseconds) since the
beginning of the exposure. The FITS scaling keywords are used to automatically convert
these times to units of seconds for users and software. This also changes the effective data
type to floating point. TIME is listed before other columns because the time would
become a double-precision value if the FITS table is converted to a QPOE file, and alignment is important in QPOE files. The location of an event in detector pixel coordinates is
given by columns AXIS1 and AXIS2. Generic coversion will use the same mapping from
fsw x and y coordinates to AXIS1 and AXIS2 coordinates for TIMETAG data as it does
for ACCUM mode data (i.e. fsw x = AXIS2, fsw y = AXIS1); for spectroscopy, AXIS1 is
the dispersion direction and AXIS2 is the spatial direction. If Doppler correction was
applied to the coordinates, a fourth column DETAXIS1 will be present which will contain
the uncorrected wavelength values (i.e., the AXIS1 column will contain the Doppler corrected values and the DETAXIS1 will contain the Doppler uncorrected coordinates at
which the event was received). Doppler correction does not affect the AXIS2 (spatial)
position of the event, only its wavelength addressing.
Table 2. Important Header Parameters of the Event Table
EXTNAME
EXTVER
TTYPE1
TFORM1
TUNIT1
TSCAL1
=
=
=
=
=
=
‘EVENTS
1
‘TIME
‘1J
‘s
1.e-4
‘
‘
‘
‘
/
/
/
/
/
/
Photon list data
Version number of table
Event clock time
32 bit integer
Seconds since exposure start time
Scale factor to convert ticks to seconds
4
TZERO1
TTYPE2
TFORM2
TUNIT2
TTYPE3
TFORM3
TUNIT3
TTYPE4
TFORM4
TUNIT4
NAXLEN
AXLEN1
AXLEN2
=
=
=
=
=
=
=
=
=
=
=
=
=
0
‘AXIS1 ‘
‘1I
‘
‘pixels ‘
‘AXIS2 ‘
‘1I
‘
‘pixels ‘
‘DETAXIS1 ‘
‘1I
‘
‘pixels ‘
2
1024
1024
/
/
/
/
/
/
/
/
/
/
/
/
/
Starting time of observation in seconds
Detector coordinate X
Data type for field
Doppler corrected AXIS2 coordinate
16 bit integer
Raw AXIS1 detector coordinate
16 bit integer
Dimension of ACCUM image
Length of first axis
Length of second axis
In addition the following keywords will be present in the event table header, to describe
the coordinates for the spatial columns. These keywords are not yet “standard”, but they
are in use by some existing event processing software, and fits2qp can use them to
copy the coordinate information to a QPOE file.
Table 3. Additional Orientation Keywords for EVENT tables. Example shown is for
imaging data - for spectroscopic data, axes will represent wavelength and space.
TCTYP2
TCTYP3
TCRVL2
TCRVL3
TCDLT2
TCDLT3
TCRPX2
TCRPX3
= ‘RA---TAN’
/
= ‘DEC--TAN’
/
=
0. /
=
0. /
=
1. /
=
1. /
=
1024.0. / axis1
=
1024.0 /
axis type for dim. 1 (e.g. RA---TAN)
axis type for dim. 2 (e.g. DEC--TAN)
sky coord of 1st axis (deg.)
sky coord of 2nd axis (deg.)
axis1 degrees per pixel
axis2 degrees per pixel
pixel of tangent plane direction
axis2 pixel of tangent plane direction
Good Time Intervals Table
The good time intervals table consists of two double-precision columns. Each row gives
the start and end times of a valid time interval during the exposure. It may be that there is
only one row. The relevant keywords in the header of this binary file are summarized in
Table 4 below
Table 4. Important Header Parameters of the Good Time Interval Table
EXTNAME
EXTVER
TTYPE1
TFORM1
= ‘GTI
= 1
= ‘START
= ‘1D
‘
‘
‘
/
/
/
/
Good Time Intervals
Version number of table
Start of time interval
Double precision datatype
5
TUNIT1
TTYPE2
TFORM2
TUNIT2
=
=
=
=
‘s
‘STOP
‘1D
‘s
‘
‘
‘
‘
/
/
/
/
Seconds since start of observation
End of time interval
Double precision datatype
Seconds since start of observation
4. Generic Conversion Creation of the Event Stream and ACCUM image
In general, the event stream input to generic conversion may be messy, containing fill data
or events received immediately following data dropouts and prior to the receipt of a coarse
time word (see “TIMETAG Data As it is Received by Generic Conversion” on page 2).
The rootname_tag.fits file produced by generic conversion will be clean, in the following
sense. Generic conversion will strip all fill data from the event stream, thus only non-fill
events will be written to the rows of the event binary table in the rootname_tag.fits file.
The fsw x and y will be mapped to AXIS2 and AXIS1, respectively, as for ACCUM mode
data, such that for spectroscopic data, AXIS1 is the dispersion direction. Additionally, for
data taken in spectroscopic configurations requiring correction for the Doppler motion of
the spectrograph (as defined by the same rules used by Commanding to command onboard Doppler compensation for ACCUM mode observations), generic conversion will
calculate the Doppler corrected wavelength (AXIS1) coordinate of each event (using the
same algorithm and methodology utilized by calstis-1 and the flight software) and
output that value (in addition to the raw detector location) to the rootname_tag.fits file (see
“Science Outputs of Generic Conversion - Structure of The Event File” on page 3).
Additionally, generic conversion will produce a list of good time intervals which it will
write to the GTI binary table extension in the rootname_tag.fits file. Good time intervals
are those times during which the take-data flag was up and there was no pause in the data
taking due to the STIS data buffer being full. To identify ‘good times’, generic conversion
will utilize the following hierarchical scheme (i.e., times must pass both these constraints
to be considered good)
1. good times are those between the START1 and STOP1, START2 and STOP2, etc.
keywords (these keywords mark the dropping and raising of the take data flag due
to loss of lock). START1 is populated by the FSW with the start time of the exposure, however STOPn is populated only when the take data flag goes down, and not
at the end of the exposure. Thus, if STARTn is filled, but STOPn is not filled,
STOPn is equal to EXPEND - i.e., the exposure continued from STARTn to the
scheduled end of the exposure, without further interruptions.
2. good times are those between the first coarse time since a stoptakingdata bit and
the last coarse time prior to the stoptakingdata bit, in the TIMETAG data stream.
6
The event stream x and y coordinates are in highres (2048 x 2048 mode) pixels. Generic
conversion will create a lowres (1024 x 1024 mode) MAMA ACCUM mode image from
the rootname_tag.fits stream by integrating in time over the good time intervals specified
in the EXTNAME=GTI binary table and binning the pixels by 2 in x and 2 in y, using the
same algorithm as employed by the FSW for OPMODE=ACCUM. The output will be a
rootname_sci.fits file, identical to the file which would be produced by generic conversion
data taken in ACCUM mode in the same observing configuration (see STIS ISR95-03),
except that the OPMODE keyword will still be set to TIME-TAG. The MAMA_BINX and
MAMA_BINY keywords will be set to 2 in the header of this file and the LORESCNV
calibration switch will be set to OMIT. The EXPTIME time and related keywords include
only the times corresponding to the good time intervals (e.g., EXPSTART will be earliest
starttime within a good time interval, EXPTIME will be the sum of the good time intervals, EXPSTOP will be latest stoptime of the good time intervals).
The rootname_sci.fits file so produced will be processed through the calstis reduction
in OPUS and processed as if it had been received as an ACCUM mode image directly
from the telescope. The rootname_tag.fits file will be part of the science dataset, but will
not be further processed by the calibration pipeline itself. As described in the next section,
a stand alone task is provided to users to work with this TIMETAG event stream.
5. INTTAG - A Stand-Alone STSDAS Task for TIMETAG data
The rootname_tag.fits file has been structured so that it can be easily transformed into a
QPOE file using the IRAF task fits2qp. That makes available to users the full suite of
event related software contained within the XRAY and EUV software packages within
IRAF.
In addition, we provide an STSDAS task, inttag, specifically designed for use with
STIS TIMETAG data. This task takes as input
•
name of file containing TIMETAG event stream (e.g., rootname_tag.fits)
•
output file name.
•
times to integrate over (as a start_time, duration, and number_of_intervals)
•
highres
flag.
The output of inttag is a time integrated (ACCUM mode) image with the same structure as the rootname_sci.fits file (i.e., a primary header followed by a single or series of
triplet extensions of type SCI, ERR, DQ). The number of triplets is determined by
the value of the number_of_intervals parameter. The time interval in the nth triplet covers
from (start_time + [N-1] ∗ duration) to (start_time + N ∗ duration). Note that the integration time
in each interval need not be identical, because events are included in the image only if they
occur during good time intervals (as determined by the GTI extension file).
7
The inttag task will properly determine and set the following keywords in the primary
and extension headers of its output images:
•
TEXPSTRT - primary header keyword
•
TEXPSTOP - primary header keyword
•
TEXPTIME - primary header keyword
•
NRPTOBS - primary header keyword
•
EXPSTART - extension header keyword
•
EXPSTOP - extension header keyword
•
EXPTIME - extension header keyword
•
MAMA_BIN1 - primary header keyword
•
MAMA_BIN2 - primary header keyword
This output file can then be processed through calstis to produce a calibrated image or
spectrum, for just that specified time interval, or set of intervals.
In default mode, inttag will produce MAMA images in the default mode (i.e., with
MAMA_BIN1 and MAMA_BIN2 = 2). There will be an optional parameter available in
the task to allow production of highres images.
8