Instrument Science Report ACS-98-01
CALACS Design: Lessons Learned from
CALSTIS
Warren J. Hack
26 May 1998
ABSTRACT
This report documents the basic design of those parts of CALSTIS relevant for CALACS. A
listing of the basic calibration steps required for ACS and a comparison of ACS with STIS
provide the basis for determining what functionality needs to be retained or used from
CALSTIS. Although CALSTIS provides a solid foundation for CALACS design, ACS data
requires changes in how memory is managed along with new functionality to interpret the
complex ACS associations. These changes are highlighted in the description of the overall
design of CALACS, with its 5 tasks: CALACS, ACSCCD, ACS2D, ACSREJ, and ACSSUM.
The HSTIO changes necessary to support this new pipeline and the implementation plan
with testing schedule complete the description of how CALACS will be created from CALSTIS.
1. Introduction
ACS uses detectors designed for STIS, therefore, it is natural to determine if software
designed for STIS data can be adapted to reduce ACS observations. At the very least, lessons learned from the STIS calibration pipeline software CALSTIS should be applied in
creating the ACS calibration pipeline software, CALACS. This paper describes differences in data format and processing requirements for ACS in the next section. The
following section outlines the current CALSTIS software, point out what functions will be
needed for CALACS, and provides details of the file I/O currently used in CALSTIS.
These elements are put together in an outline of the CALACS software in Section 4. Keywords used for CALACS are given in Section 6, while the required changes in HSTIO are
described in Section 7. The remainder of this paper provides an outline of the development
schedule for CALACS (in Section 8).
1
2. ACS Data and Software Requirements
There are a number of difference between ACS and STIS data that preclude a direct use of
most of the CALSTIS code. The type of rewriting that needs to be done relies, of course, on the
exact nature of the new data, so by understanding the new ACS data, a clear picture of the
required changes becomes evident. Once the changes are identified, then the CALSTIS code will
be copied and modified as needed to create a separate pipeline for the ACS data.
Differences between ACS and STIS data can be classified into two areas: data format and
reduction requirements. The ACS data format itself derives from the STIS FITS file format using
FITS extensions for science data, data quality arrays and error arrays, resulting in a three extension IMSET within the FITS file. However, there are these differences:
•
2 chip readout for the wide field camera (WFC)
•
maximum 2048x4096 science data arrays in each of 2 chips
•
strictly imaging data (aside from grism exposures)
The decision was made to store the WFC data using 2 IMSETs in one FITS file, with one
IMSET for each chip. Each IMSET will consist of a science (SCI) array, a data quality (DQ) array
and an error (ERR) array. The size of the image arrays, though, results in a maximum size of
160Mb for a single calibrated WFC ACS observation, under the assumption that the ERR array is
stored in floating point format.
The following are the design decisions made for the baseline ACS pipeline:
•
CR-SPLIT and REPEATOBS options are mutually exclusive
•
geometric correction will be handled ‘off-line’ as a stand-alone task
•
no spectroscopic reductions (even for GRISM data) in baseline version
•
use of NICMOS styled associations for CR-SPLIT and REPEATOBS data, but no other associations
• no combining of dithered observations
• no special reductions for polarized observations
In addition to the changes to the calibration steps which will be performed by CALACS, one
major change in design is required in order to allow it to run at all on most systems: line-by-line
image I/O. This will require the greatest amount of work. The effect of this new file I/O model
will be discussed in detail in the following section along with the summary of the functions that
will still be needed from CALSTIS. CALACS will retain as much of the same code from CALSTIS as possible in an effort to allow the two pipelines to eventually share libraries of functions in
later builds for tasks which are camera independent, such as cosmic ray rejection.
2
3. Description of Relevant CALSTIS Functions
Modularity defines the basic design used for CALSTIS, a modularity of functionality which
was used to build separate tasks run under a controlling task, CALSTIS0. In all, CALSTIS contains 12 separate tasks each of which can be run individually from the command line when they
are not called from CALSTIS0. Figure 1 illustrates the overall structure of CALSTIS0, with the
spectroscopic steps being combined into one ‘function’ call since they will not be used in
CALACS.
Figure 1: Flow Diagram of CALSTIS0.
CRCORR
Yes (CCD)
No (MAMA)
ATOD,DQI, or
BLEV
Yes
CALSTIS 1
(AtoD, DQI, BLEV)
CRCORR
Yes
CALSTIS 2
Cosmic Ray Reject
CALSTIS 1
Yes
FLAT
CALSTIS 1
FLTCORR
SPECTROSCOPY
Spec
OBSTYPE
GEO CORR
(CALSTIS1,11,7,4,12,7,6)
Yes
CALSTIS 7
Yes
RPT CORR
CALSTIS 8
3
CALSTIS relies heavily on modularity, something which will work to our advantage in adapting STIS code for CALACS. File management is one aspect of CALSTIS which is shown in
Figure 1. Since each step may or may not be used for the calibration of an image being processed,
the responsibility for accessing the image lies with the individual functions which perform the
calibration steps. This results in CALSTIS0 only passing the image name from one step to the
next. This makes it easy to customize the file I/O to match the needs of the data and calibration
step.
CALSTIS Handling of Headers and Image Data
Overall, CALSTIS attempts to perform as many operations in memory and in place on the
images as possible. This involves having the header information in memory during processing as
well as the image data. Several structures are defined within CALSTIS to package up the relevant
keywords and switch values and pass them along for processing. The structure definitions for
CALSTIS0 and CALSTIS1 can be found in Appendix A. Data arrays for the images are read into
and stored in an HSTIO structure SingleGroup. This structure packages the primary header information, the extension specific header information, and the data arrays for the SCI, DQ and ERR
arrays (for STIS). All three arrays are read in and written out using HSTIO routines that handle
the reading or writing with one function call.
The data and header I/O can be seen in the following summary of routines relevant to ACS
data:
[CALSTIS0]
• Read in primary image header and store key values in several structures, including StisInfo and specific structures for controlling CALSTIS1.
• Use values in StisInfo structure to control what processing steps are performed. None of these values are modified
by underlying functions.
• Pass information from CALSTIS1 specific structures to processing steps in CALSTIS1. This information is not
directly modified at any point.
[CALSTIS1]
•Open image. Read primary header into local header
structure.
•Update StisInfo1 structure with switch information
input from CALSTIS0 and from header structure.
•Close input image and free structure header structure.
•Pass StisInfo1 structure to function Do2D for processing of each IMSET in the input image.
[Do2D]
4
Read data arrays into memory as SingleGroup structure.
Extract IMSET specific header information into
StisInfo1 structure.
Pass SingleGroup structure to following functions
for operation on data arrays in memory:
doDQI StisInfo1[Input] SingleGroup[Input/Output]
Operates on DQ array in-place
doAtoD StisInfo1[Input] SingleGroup[Input/Output]
Applies AtoD correction in-place.
doBlev StisInfo1[Input] SingleGroup[Input] extver
SingleGroup(&z)[Output] &meanblev &done
&driftcorr
Subtract bias from SingleGroup and
output to SingleGroup(&z).
[CALSTIS0]
• Check values in StisInfo structure to determine if cosmicray rejection needs to be performed. If so, pass name of
files from StisInfo to CALSTIS2.
[CALSTIS2]
•Open input file template pointing to list of images to
be processed
[CRREJ_DO]
Open Input files and temp files, make sure images
have same characteristics (done in CRREJ_CHECK)
Extract parameters from image headers
[CRREJ_SKY]
Calculate sky for each image, read in
each image into memory for the
calculation.
Read first image in list into memory as SingleGroup
[CRREJ_INIT]
Use line-by-line I/O to read in
individual lines from all images in list
at once and compute comparison image
[CRREJ_LOOP]
Read in one image at a time to perform
final rejection against the comparison
image. Resultant cleaned image
SingleGroup computed in place.
5
[CRREJ_DO]
Cleaned image scaled by exposure time
and sky value with the results kept
in SingleGroup.
Header values updated in SingleGroup.
Combined SingleGroup written to output
file in final processed form.
SingleGroup finally closed and temp arrays freed.
[CALSTIS2]
•Input file list closed and return to calling routine
[CALSTIS0]
• Complete basic 2D calibration on all IMSETS (including
combined cosmic-ray rejected products) as required:
[CALSTIS1]
•Process each IMSET based on the calibration switches
set in the primary header.
[Do2D]
doNoise StisInfo1[Input] SingleGroup[Input/Output]
Evaluate ERR array in-place
doLoRes StisInfo1[Input] SingleGroup[Input]
SingleGroup(&y)[Output] &done
Bins SingleGroup to create
SingleGroup(&y)
doNonLin StisInfo1[Input]
SingleGroup[Input/Output] &gsat &lsat
Corrects or flags non-linearity
in-place.
doBias StisInfo1[Input] SingleGroup[Input/Output]
Subtract bias image from SingleGroup.
doDark StisInfo1[Input] SingleGroup[Input/Output]
&meandark
Subtract dark image from SingleGroup.
doFlat StisInfo1[Input] SingleGroup[Input/Output]
Divide SingleGroup by flat-fields.
doShad StisInfo1[Input] SingleGroup[Input/Output]
Applies shutter shading correction
to SingleGroup.
doPhot StisInfo1[Input] SingleGroup[Input/Output]
6
Updates SingleGroup header in-place
with calculated photometry information.
doStat SingleGroup[Input/Output] sts.sdqflags
Computes good min/max/means for data
and error and updates SingleGroup
header in-place.
[Do2D]
•Update SingleGroup structure header in memory after
each processing step is performed.
•Write out SingleGroup structure after all processing
steps are performed, complete with updated header
information.
•Close image and free memory used by SingleGroup.
[CALSTIS0]
• Check StisInfo structure to see if images need to be summed
together. If so, pass some information from StisInfo to
CALSTIS8 for processing.
[CALSTIS8]
•Initialize StisInfo8 structure, and header structure.
•Open input image and read primary header into header
structure.
•Copy pertinent keywords into StisInfo8 structure and
delete primary header structure.
[SUMGRPS]
initialize 2 SingleGroups x and y
Read in input image (SingleGroup x)
Get IMSET specific header info into local variables
For each successive image, read SingleGroup into
memory as y and add to SingleGroup x in memory.
For each image, update keyword values in SingleGroup x’s header in memory
Free each SingleGroup y once added to x
Update header info in x, then write SingleGroup to
output file.
Free SingleGroup x
[CALSTIS0]
• Free reference file structures and quit.
In the above summary, function and task names are given in bold and structure variables are
listed in italics.
7
The handling of the header information and data arrays all are completed in memory until the
processing is completed, which works well for datasets small enough to fit in memory but
becomes untenable for very large data arrays. In general, the data array I/O is confined to the lowest level routines possible, reserving that activity until absolutely necessary and when it needs to
be done the least. The header information, on the other hand, is read in early and controls the
operations of the program. This information is then passed down to the processing steps but not
modified until the data arrays are modified, and even then it is done in memory and only written
out upon completion of the reduction step. This safeguards the input file’s headers from being
modified out of step with the reductions and localizes any errors in writing the headers to the lowest level routines possible, making for a more secure program overall.
4. CALACS Design
In comparison to CALSTIS, the overall design for the CALACS package will be much less
complex, with fewer separate tasks to maintain. The same basic procedures used in CALSTIS will
be applied to the ACS data, including the separate calibrations for MAMA and CCD data, as
shown in Figure 2. This design will form the basis for CALACS, the task responsible for managing the operation of the remainder of the pipeline. The task CALACS will serve primarily as a
wrapper for the other pipeline tasks, retaining some aspects of CALSTIS0, revising the STIS-specific logic to be applicable to ACS data, and incorporating new code to interpret the NICMOSstyle ACS association file. As a result, the task CALACS will be new code which shares basic
design features with CALSTIS0.
The ACS pipeline, shown in Figure 2, will be derived from CALSTIS tasks as follows:
CALACS
CALSTIS0
Control operation of all tasks
ACSCCD
CALSTIS1
AtoD, DQI, BLEV functions only
ACS2D
CALSTIS1
DQI plus remaining functions,
including MAMA specific steps
ACSREJ
CALSTIS2
complete task rewritten for ACS
ACSSUM
CALSTIS8
complete task rewritten for ACS
These new tasks will be revised to accommodate the larger image sizes for ACS, the use of
NICMOS-style associations, and the replacement of STIS-specific logic and data parameters. The
management of the associations will take place in CALACS as it replaces looping over the
IMSETs in a file with looping through the files listed in the ACS association file. The routines
listed in Table 2 will be revised as they contain logic specific to STIS data which can not be
applied to ACS observations. Finally, a new convention for naming the CALACS tasks breaks
from the CALSTIS model where tasks are named CALSTIS1, CALSTIS2, and so on, however, it
provides a name which better describes how each task was designed to be applied.
8
In an extension of the design of CALSTIS1, the functions called prior to cosmic-ray rejection
and those called afterwards will be separated into individual tasks for ACS, as shown in Figure 2.
This will not only isolate calls specific to the CCDs from those used for all the images, but will
allow the creation of the intermediate overscan trimmed image on disk for use in further processing steps. Therefore, the task ACSCCD will perform the initial processing on the input images
resulting in the overscan trimmed image being written to disk, while the remaining CALSTIS1
tasks will be merged into the ACS2D task. The ACS2D task will also include the tasks which initialize the data quality and error arrays so that it can be used for both CCD and MAMA
observations.
The cosmic ray rejection task CALSTIS2 will require significant revision to use the memory
model required by CALACS, but will still remain a single task as ACSREJ. Finally, the task for
summing repeated observations will also need to be modified for line-by-line I/O and to conform
to the CALACS memory model. The following sections describe what changes in design will be
needed for the CALACS package to accommodate ACS data.
CALACS Association Table
A major change in the design of CALACS will be in the need to interpret association tables
and process the appropriate files. Association files are necessary due to the complex set of relationships between the exposures. The current baseline version of CALACS will recognize and
correctly process CR-SPLIT or REPEAT-OBS exposures resulting in either combined cosmic-ray
rejected images or a combined image of all the repeated exposures. As shown in Figure 2, the
CALACS task would process related CR-SPLIT exposures through ACSCCD then combine them
into a single CR-combined product using ACSREJ. This CR-combined product would then be
processed through ACS2D to produce the calibrated product designated in the association table.
This process would then be repeated for the remaining sets of CR-SPLIT exposures in the association table.
In future enhancements, each CR-combined product could represent a position in a dither pattern, which would need to be combined into a dither-combined exposure. To support the creation
of the CR-combined products and future dither combining, the NICMOS-style association table
was adopted containing MEMNAME, MEMTYPE and MEMPRSNT columns. Unlike the NICMOS tables, ACS tables will involve multiple output products. The MEMNAME column lists all
the exposure filenames which comprise the entire association, complete with output product
names. The MEMTYPE columns specifies the type of role the file has in the association. A different set of MEMTYPEs specific to ACS were adopted in order to provide the support for the
9
multiple products. CALACS will recognize the following MEMTYPE values in the baseline
version:
MEMTYPE
EXP_CRn
Description
Input CR-SPLIT exposure for CR-combined image n
PROD_CRn
CR-combined exposure n
EXP_RPTn
Input REPEAT-OBS exposure #n
PROD_RPT
REPEAT-OBS combined output product
(PROD_DTH)
(Dither-combined output product, not in baseline version)
An example association table for a 2 position dither set with CR-SPLIT=2 is given in Table 1.
The MEMTYPE for each component of the first CR-SPLIT exposure, j1xx52ecm and j1xx52egm,
are given as EXP_CR1. The output product is not only given a name based on the association
table’s name, j1xx5201, but it is designated in the table with a MEMTYPE of PROD_CR1. This
example not only shows how the MEMTYPE can be used to associate input and output products,
but also demonstrates how the MEMNAME will be used as well. The last digit of each product’s
filename corresponds to the output product number in the MEMTYPE, with the designation of 0
being reserved for the final dither-combined output product (which will not be produced in the
baseline version).
Table 1. Sample Association Table J1XX52010_ASN
MEMNAME
MEMTYPE
MEMPRSNT
j1xx52ecm
EXP_CR1
yes
j1xx52egm
EXP_CR1
yes
j1xx52011
PROD_CR1
yes
j1xx52emm
EXP_CR2
yes
j1xx52eom
EXP_CR2
yes
j1xx52012
PROD_CR2
yes
j1xx52010
PROD_DTH
yes
CALACS will then use the number in the MEMTYPE to associate the input exposures, designated as EXP_CR or EXP_RPT, and process all of them before combining them into the output
product. This output product then gets calibrated by the rest of the pipeline. Code from the NICMOS pipeline software CALNICB will be copied and revised to handle the interpretation of ACS
association tables.
10
CALACS Memory Model
Accommodating the larger ACS images will require a significant change in the memory management of the IMSETS. The working memory design model will assume a maximum size of
approximately 80Mb per observational IMSET and that one IMSET could be held in memory as
the output image, which would be built up during processing. A limitation on writing out an entire
IMSET at once is imposed by the FITS kernel which only allows one extension of an IMSET to
be open for line-by-line I/O at the same time. Without keeping the entire output in memory, a temporary file for each extension would have to be created as needed, then copied into the final output
file one extension at a time. This would dramatically increase the amount of disk I/O involved in
working with the image.
As a result, we decided on using a hybrid I/O model for CALACS. By keeping an output
IMSET in memory during processing, we can operate on the entire IMSET in place. Once processing is complete in the task, the output file will be written out using the currently available
HSTIO functions without the need for temporary files. Line-by-line I/O will then be used to read
in the potentially large input images, especially when more than one image is operated on at the
same time, such as in bias image subtraction or in summing images. In particular, the output
image would be created in memory as a SingleGroup from one of the input images, then the other
image(s) would be read in line-by-line and applied to the output image in-place. The functions
which need to be revised for line-by-line I/O are listed in Table 2, along with some comments on
the form those revisions will take.
Table 2. Functions Which Require ACS Motivated Revisions
Function
Revisions Needed
doBlev
Operate on X in-place, but specify data section within overscan sections for use
by PutSingleGroup to write out smaller image to file.
doBias
Remove the dependency on binning and convert to line-by-line I/O
doDark
Remove the dependency on binning and convert to line-by-line I/O
doFlat
Remove the dependency on binning and convert to line-by-line I/O
doShad
Remove the dependency on binning and convert to line-by-line I/O
doNonLin
Remove the dependency on binning.
doDQI
Remove dependency on binning and dispaxis.
11
Function
Revisions Needed
crrej_loop
Convert to operate on image sections
sumGrps
Convert to use line-by-line I/O
sub2d,div2d,mult2d
Convert to use line-by-line I/O
RptSum
Convert to use line-by-line I/O
FindBin
Convert to work with only one line at a time. Also, remove dependency on binning and dispaxis as they impose STIS specific logic.
CALACS Data Products
Output from this calibration pipeline will consist of files containing the calibrated observations stored as IMSETs in each file. Each IMSET consists of a SCI (science data) extension, a DQ
(data quality) extension, and an ERR (error) extension for a maximum size of 80Mb plus the
header information. The file extensions which will be used for the ACS observations are listed in
Table 3, which closely mimics the extensions used by STIS.
Table 3. Filename extensions used by CALACS:
IPPPSSOOT_RAW
Raw data
_ASN
Association file for observation set
_FLT
Calibrated individual exposure
(not produced if CR-split)
_BLV_TMP
Overscan-trimmed individual exposure
(not kept unless requested by the user)
_CRJ_TMP
Uncalibrated, CR-split-combined image
(renamed to _CRJ if ACS2D is not run, otherwise deleted)
_CRJ
Calibrated, CR-split-combined image.
_SFL
Calibrated, Repeat-Obs-combined image.
_DTH
Calibrated, Dither-combined image (not in baseline)
The initial inputs to the task CALACS will be the _RAW files from generic conversions and
the _ASN (association) table for the complete observation set. Output files for a REPEAT_OBS
association will be individually calibrated _FLT exposures and a calibrated summed _SFL image,
while CR-SPLIT observations will produce _CRJ images. The _BLV_TMP file will be created
for CCD images as output from ACSCCD, will serve as input to either ACSREJ or ACS2D and
will be deleted once it is no longer needed for processing. Similarly, for CR-SPLIT exposures,
_CRJ_TMP will be created as output from ACSREJ and will either be copied to an _CRJ file is
ACS2D is not run, or simply deleted upon completion of ACS2D. Finally, if later versions
CALACS ever perform dither-combining of _CRJ images, they will produce a _DTH image.
12
Optional ERR Array
Not everyone wants or needs the ERR extension. Since the ERR extension is identical in size
to the SCI extension (in uncompressed form), the ERR array comprises 32Mb of an output
IMSET, or up to 64Mb for the entire FITS file for a 2 chip WFC observation. Therefore, an option
will be built into CALACS to output a NULL ERR extension with a keyword WRTERR set to
‘NO’. This would cause CALACS to process the observation as usual, including the ERR arrays,
and then simply write out a NULL ERR extension upon completion with the keyword signifying
that the errors were not written out at all, rather than assuming that they are zero because of their
absence. It should be stressed that the presence of a NULL ERR array will not adversely affect
pipeline should this observation be run through again, and that HSTIO already knows how to
work with these NULL arrays.
Figure 2: Flow Diagram for CALACS.
Detector
CCD
ACSCCD
Another Image in
CR-SPLIT set?
(AtoD, DQI, BLEV, Bias)
MAMA
Another
Image in
association?
CRCORR
ACS2D
Yes
ACSREJ
DQI,Flat,Dark,...
Cosmic Ray Rejection
ACS2D
Flat, Dark,...
Another Set of CR-SPLIT Images?
RPT CORR
Yes
ACSSUM
5. CALACS Tasks
This section describes how the design changes described in the previous section will be
applied to the CALSTIS code to create the baseline tasks that will make up CALACS.
13
ACSCCD Summary
This routine contains the initial processing tasks that need to be performed on all ACS CCD
data and will be comprised of four main functions from CALSTIS1, performed in the following
order:
•
error array initialization in ‘doNoise’,
•
data quality initialization in ‘doDQI’,
•
AtoD conversion in ‘doAtoD’,
•
bias level subtraction and trimming in ‘doBlev’, and
•
bias image subtraction in ‘doBias’.
The upper level task, ACSCCD, will use header keyword values to determine what steps
should be performed, and only perform them if needed, rather than blindly applying the correction
more than once on the data. The bulk of this task will be based on the code in CALSTIS1, but then
be simplified to control only these four tasks using logic applicable to ACS rather than using the
current STIS-specific logic. Input to this task will be an image list or single image, which could
either be provided by the task CALACS or the user directly. It will process each image in the list
or the single image and create an output image with the _BLV_TMP file extension.
Two of the functions in this task do not need any revision at all to work with ACS data:
‘doNoise’ and ‘doAtoD’. These functions simply operate in-place on the output image stored in
memory.
The function ‘doDQI’, on the other hand, needs major revision to function properly on ACS
observations. First, it relies on keywords which are unnecessary for ACS, such as dispaxis for
designating the dispersion axis in the spectroscopic observation. It also defines variables specific
to STIS observations; namely, high_res which designates whether the observation was subsampled or not, specifically for MAMA high-resolution observations and associated reference
files. Finally, the routine determines how to process the data based on logic specific to STIS
observations which can not be generalized to ACS observations without ‘falsifying’ the values for
the STIS specific parameters to treat ACS data appropriately. For STIS, MAMA data are designated as high resolution with a binning of 1, when ACS data would be considered low resolution
for the same binning factor. In short, the entire function needs to be re-written specifically for
ACS.
Since binning will not be possible for the MAMA, all code related to binning of MAMA data
will be removed, along with all code related to the keyword dispaxis. Binning, however, will
still be a possibility for CCD observations, so that code will remain in place, but likely will be
untested until needed to support flight-software changes to allow CCD binning.
The function ‘doBlev’ requires conversion to line-by-line I/O when fitting the bias level from
the overscan region to the image data. In a change from the CALSTIS design, bias image subtraction will now be performed prior to cosmic-ray rejection using ‘doBias’. This move was made to
14
allow proper accounting of the bias contribution from each pixel in the CR-combined image.
When performed after ACSREJ, the bias image is scaled by the number of images which were
combined. However, for cosmic-ray rejected pixels, the flux in that pixel would be scaled by the
exposure time (not number of total images), scaling the bias contribution as well, and if the cosmic-ray pixel came from an image which had a different exposure time, the bias contribution
would be incorrectly scaled. Therefore, in order to correctly account for the bias contribution from
each pixel, this correction will be performed prior to combining the images. This function will
need to be re-written to apply the correction to an image which still contains the overscan regions,
and to read the bias reference image using line-by-line I/O, in order to operate with ACS images.
Finally, ACSCCD will require a new HSTIO function to write out to disk the section of the
observation that does not contain the overscan data. This trimmed output image would then be the
final product from this task, and would be used as input into the remaining of the processing steps
in CALACS.
ACS2D Summary
Every observation will eventually be processed by this task, whether it is MAMA data or CCD
data, as it contains the basic functions necessary for calibration. It will be a direct derivative of
CALSTIS1, with the exclusion of the AtoD conversion and bias level subtraction and the STIS
specific function ‘doLoRes’ for rebinning sub-sampled MAMA data. It will, however, contain the
data quality initialization function used in ACSCCD and the error array initialization function
‘doNoise’ for the benefit of those observations which would never be run through ACSCCD. A
check will be made to insure that the array initialization is not performed on CCD data twice. Calibration switches in the image header control the performance of the remaining calibration
functions as they do in CALSTIS1, with MAMA specific functions being initiated only when the
MAMA specific calibration switch was set, for example.
The primary tasks contained in ACS2D will be (in operational order):
doNoise
Applies a simple noise model to blank error array, if not done in ACSCCD
doDQI
Initialize data quality flags, if not done in ACSCCD
doNonLin
Apply and flag non-linear data
doDark
Perform dark image subtraction
doFlat
Combine and subtract flat field images
doShad
Perform shutter shading correction for CCD, if not CR-combined
doPhot
Compute photometric values for header
doStat
Computes min, mean and max of good SCI data
In addition to these primary functions, other functions will be used to ensure that the appropriate comments are saved to the history section of the header. Of these functions, ‘doFlat’,
15
‘doDark’, and ‘doShad’ rely on binning information specific to STIS to determine the resolution
of the data compared to the reference data. These functions can not be re-written for use with both
STIS data and ACS data as a result of this use of STIS specific information and logic. As with
ACSCCD, the code will be copied from CALSTIS and revised specifically for use with ACS data,
retaining CCD binning and discarding MAMA binning.
Revising the STIS specific tasks for ACS will actually result in much simpler code. The revisions will eliminate the need for the switch doppcorr and related keyword value for
dispaxis. Both of these keywords and related code will have to be removed from all ACS code
because of the conflict that would arise in trying to apply them to ACS data in ‘doDQI’ and
elsewhere.
Therefore, of all the routines derived from CALSTIS1, the following will need to be rewritten
specifically for ACS: doDQI, doDark, doFlat, doNonLin, and doShad, along with the function
‘GetGrpInfo1’ which sets parameters based on header keyword values and switch settings. In
essence, this results in a completely new, ACS specific task which relies solely on logic applicable
to ACS observations. Along with these revisions, changes to support line-by-line I/O for those
functions working with more than one image at a time will be performed. These revisions will be
based on code copied from CALSTIS but this code will evolve separately without trying to apply
it to STIS data.
ACSREJ Summary
Unlike CALSTIS2, this task will not be able to keep multiple images in memory due to the
potentially very large size of the images arrays. Instead, line-by-line I/O will be used while still
keeping the overall design and basic algorithm for finding and rejecting the cosmic rays applied in
CALSTIS2. Slight modifications, however, may be necessary to insure compatibility with the list
of images extracted from the association file. CALSTIS2 relies on more direct access of the science and data quality arrays through the use of lower-level CVOS routines, allowing this routine
to be revised without relying on changes to HSTIO like the rest of CALACS. A revised version of
the cosmic ray rejection task could then be developed and tested separately from the CALACS,
using STIS data if necessary.
This new task will be nearly identical to CALSTIS2 with the majority of the revisions taking
place in the function ‘crrej_loop’. The processing will start by computing the initial comparison
image from the list of images using line-by-line I/O, and will not need any modifications from
CALSTIS2 at this point. This comparison image will have to be written out and the filename
passed onto the function ‘crrej_loop’ for final rejection and creation of the summed output
image. This routine will need to be re-written to work with only 1 output image and read in sections of the input image and the comparison image. Once cosmic-rays are detected in the input
image section, they will be marked and added to output image in memory. This can be seen in
Figure 3, where those elements kept in memory are drawn in bold lines and file I/O is indicated by
arrows.
16
One additional change in the way ACSREJ will work involves the application of the shuttershading correction, previously done in ‘doShad’. This correction, while small for long exposure
images, can be significant for very short exposures. CALSTIS2 does not apply this correction,
since it is applied after CALSTIS2 completes processing the CR-SPLIT observations. The problem with that approach is that it improperly applies the shading correction to pixels that have had
the values from one or more input images rejected for cosmic rays. It assumes that there have
been N exposures and bases the correction on that number. To properly apply it after cosmic ray
rejection would require keeping track of the number of input exposures contributing to each pixel,
which is not available from the input data sets.
The way that the shading correction will be applied as part of cosmic ray rejection in ACSREJ
is described as follows:
1. the initial guess image is computed in the same manner as CALSTIS2.
2. each input image pixel is compared to the initial guess image by scaling the initial guess
image to match the effective exposure times of the single image and the initial guess
image. The effective exposure times include the shutter shading term correction, i.e., for a
single exposure, effective exposure time = nominal exposure time (as expressed by the
header keyword) + shutter shading correction for that pixel. The effective exposure time
for the initial guess image is the total of the individual nominal exposure times + N times
the shutter shading correction for that pixel where N is the number of exposures in the CRsplit set. Depending on whether or not the individual pixels value is within the acceptable
range of the value derived from the initial guess image, it is marked as accepted or
rejected.
3. If the pixel for that image is accepted, it is summed into the output array unscaled. The
output array consists of a simple sum of input array values with no adjustments other than
those applied before the CR rejection step. If the pixel is rejected, it is not summed into the
output array. The mask for the input image is updated to indicate a value of 1 for accepted
pixels and 0 for rejected pixels. (The format of the mask array will be described in more
detail below)
4. when all input images have been processed, every pixel in the summed output array is
scaled to reflect the proper number value corresponding to the total nominal exposure time
(the same for all pixels). This is done by scaling the pixel by the factor
( Σ i ( T i + ∆t i ( x ,y ) )m i ( x ,y ) ) ⁄ T
where T i is the nominal exposure time for image i, ∆t i ( x ,y ) is the shading correction for the
pixel x, y, and m i ( x ,y ) is the mask value for the i’th image at pixel x, y, and T is the total nominal exposure time for all input images (regardless of whether they were all used for that particular pixel). In short, the cosmic ray rejection step sums all the accepted values, computes
the true exposure time for that pixel and scales the sum to correspond to the total nominal
exposure time.
A mask array will be used instead of an accumulated exposure time array throughout this task.
This array will use 1 bit for every pixel requiring only 1MB per input image. In contrast, an accu-
17
mulated time array would require keeping another 32MB array in memory during processing, a
requirement which was deemed excessive in addition to the 80Mb of memory already being used
for the output image. The accumulated exposure time will be calculated from the mask arrays
using the above equation after all input arrays are processed and will require much less memory if
the shading correction can be represented as a highly subsampled image or in a functional form
(e.g., a 2nd order polynomial).
Figure 3: ACS Cosmic Ray Rejection
Input Images
Comparison:
Median/Min of
Input Images
Identify Cosmic
Rays
Written to
Disk
Input
Images
Summed
Output
Written to
Disk
In short, the calculations will be done on each image one section at a time, and the summed
cosmic-ray corrected image will be built up in memory one section at a time as well, along with a
bit-mask reflecting what images contributed to each pixel. This will minimize the amount of
memory necessary for the computations, but the line-by-line I/O should not have a dramatic
impact on the speed of the processing when done. Testing will be conducted to not only insure
that this section-by-section approach identifies the same cosmic rays identified by CALSTIS2, but
also to see what impact the line-by-line I/O has on the processing speed.
ACSSUM Summary
This routine stands as the probably the simplest task in CALACS, simply summing a list of
images together one line at a time and updating 4 keywords to reflect this operation. In general,
CALSTIS8 can be used with very little modification, primarily to allow the use of file list read in
from the association file and to use line-by-line I/O for the input images, while deleting all code
related to spectral orders.
The function ‘SumGrps’ will need to be revised to loop over image files rather than extensions of one image, since STIS associations were always contained in the same image file. The
first image on the list will be read into memory, just as it is done now in CALSTIS8. However,
only enough memory for 1 line of data will be allocated for use by the remainder of the input
images, then each image will be read in line-by-line using the new HSTIO functions. The function
will then loop over each line in the input image and add it to the output image in memory. Once all
the images are summed together, the output image will be written out as done in CALSTIS8 now
and the task will be complete. Unlike CALSTIS8, however, we do not have images of different
18
sizes which will be summed together, like the different spectral orders. Therefore, all code related
to spectral orders will be removed for ACSSUM.
6. Keyword Usage
Processing of the images in CALACS will be controlled by the population of keywords and
calibration switches in the input image headers, as it is in CALSTIS. Those keywords essential for
CALACS remain a subset of those used by CALSTIS, with keywords related to spectroscopy and
to binning of the images being absent. Table 4 provides a summary of those keywords which will
be derived from CALSTIS for use by ACS, along with any that are unique to ACS.
Table 4. ACS Keywords Used in CALACS
Keyword
STIS Function(s)
I/O
Header
Sample of Possible Values
APERTURE
calstis1,calstis8
Input
Primary
WFC1,WFC2,WFC1-FIX,
WFC2-FIX,WFC,HRC,SBC,...
ASN_MTYP
calstis1
Input
SCI
PRODUCT,REPEATOBS,CRSPLIT,NONE
COMPLETE,OMIT,PERFORM,SKIPPED
ATODCORR
atodHistory
Input/Output
Primary
BADINPDQ
calstis2
Output
Primary
BIASCORR
do2D
Input/Output
Primary
BIAS_REJ
calstis1
Input
Primary
BINAXIS1,BINAXIS2
calstis0,do2d
Input
Primary
BLEVCORR
do2D
Input/Output
Primary
COMPLETE,OMIT,PERFORM,SKIPPED
CCDAMP
calstis1
Input
Primary,SCI
ABCD,ABC,ABD,AC,AD,ACD, BC,BD,BCD
CCDGAIN
calstis1
Input
Primary,SCI
1,2,4,8
COMPLETE,OMIT,PERFORM,SKIPPED
CD1_1, CD2_2
do2D
Input
SCI,ERR,DQ
CENWAVE
photmode
Input
Primary
CRCORR
do2D,calstis2
Input/Output
Primary
CRPIX1,CRPIX2
calstis1
Input
SCI,ERR,DQ
CRRADIUS
calstis2
Output
Primary
CRSIGMAS
calstis2
Output
Primary
CRTHRESH
calstis2
Output
Primary
DARKCORR
do2D
Input/Output
Primary
COMPLETE,OMIT,PERFORM,SKIPPED
DETECTOR
calstis1,calstis8
Input
Primary
WF1,WF2,HRC,SBC
DQICORR
do2D
Input/Output
Primary
COMPLETE,OMIT,PERFORM,SKIPPED
EXPSCORR
calstis0
Input
Primary
COMPLETE,OMIT,PERFORM,SKIPPED
EXPEND
calstis8
Input/Output
SCI
EXPSTART
do2D
Input
SCI,ERR,DQ
EXPTIME
do2D,calstis8
Input/Output
SCI,ERR,DQ
COMPLETE,OMIT,PERFORM,SKIPPED
FILENAME
calstis8
Output
Primary
FLATCORR
calstis1
Input/Output
Primary
COMPLETE,OMIT,PERFORM,SKIPPED
FLUXCORR
calstis8
Input
Primary
COMPLETE,OMIT,PERFORM,SKIPPED
GLINCORR
calstis1
Input/Output
Primary
COMPLETE,OMIT,PERFORM,SKIPPED
GLOBLIM
doNonLin
Output
SCI
EXCEEDED, NOT-EXCEEDED,
NOTAPPLICABLE, UNDETERMINED
GLOBRATE
calstis1
Input
SCI
GOODMAX
do2D,calstis8
Output
SCI,ERR
GOODMIN
do2D,calstis8
Output
SCI,ERR
19
Keyword
STIS Function(s)
I/O
Header
GOODMEAN
do2D,calstis8
Output
SCI,ERR
INITGUES
calstis2
Output
Primary
LFLGCORR
calstis1
Input/Output
Primary
LTM1_1,LTM2_2
calstsi1
Input/Output
SCI,ERR,DQ
LTV1,LTV2
calstis1
Input/Output
SCI,ERR,DQ
Sample of Possible Values
COMPLETE,OMIT,PERFORM,SKIPPED
MEANBLEV
do2D
Output
SCI
MEANDARK
do2D
Output
SCI
MEANEXP
calstis2
Output
Primary
NCOMBINE
calstis1
Input
SCI
NEXTEND
calstis1,calstis8
Input
Primary
NGOODPIX
do2D,calstis8
Output
SCI, ERR
NRPTEXP
calstis8
Input
Primary
OBSMODE
calstis1,calstis8
Input
Primary
ACCUM,ACQ,ACQ/PEAK, ALIGN,ANNEAL
PHOTCORR
do2D
Input/Output
Primary
COMPLETE,OMIT,PERFORM,SKIPPED
PHOTFLAM
doPhot
Output
SCI
PHOTMODE
doPhot
Output
SCI
PHOTZPT
doPhot
Output
SCI
PHOTPLAM
doPhot
Output
SCI
PHOTBW
doPhot
Output
SCI
REJ_RATE
calstis2
Output
Primary
ROOTNAME
calstis1,calstis8
Input
Primary
RPTCORR
calstis8
Output
Primary
SCALENSE
calstis2
Output
Primary
COMPLETE,OMIT,PERFORM,SKIPPED
SDQFLAGS
do2D
Input
Primary,SCI
SHADCORR
calstis1
Input/Output
Primary
SKYSUB
calstis2
Output
Primary
SKYSUM
calstis2
Output
Primary
SNRMAX
do2D,calstis8
Output
SCI
SNRMEAN
do2D,calstis8
Output
SCI
SNRMIN
do2D,calstis8
Output
SCI
SUBARRAY
calstis1
Input
Primary
TRUE,FALSE
STATFLAG
calstis0
Input
Primary
T,F
TEXPTIME
calstis2
Output
Primary
COMPLETE,OMIT,PERFORM,SKIPPED
Unique to ACS
ATODGN1, ATODGN2
do2D
Output
SCI
CCDOFST1, CCDOFST2
calstis1
Input
SCI
CDRKFILE
doDark
Input
Primary
Input
SCI,DQ,ERR
1,2
calstis1,calstis8
Input
Primary
CLEAR1L,F555W,F775W,F625W,F550M,F850
LP,CLEAR1S,POL0UV,POL60UV,...
N/A
Primary
doDark
Input
Primary
Input
Primary
Output
SCI
Input
Primary,ERR
CHIP
FILTER1,
FILTER2
GEOFILE
MDRKFILE
OSCNTAB
READNSE1, READNSE2
do2D
WRTERR
Reference Tables
APERTAB
do2D
Input
Primary
ATODTAB
doAtoD
Input
Primary
20
Keyword
STIS Function(s)
I/O
Header
BIASFILE
doBias
Input
Primary
BPIXTAB
doDQI
Input
Primary
CCDTAB
do2D
Input
Primary
CRREJTAB
calstis2
Input
Primary
PFLTFILE
doFlat
Input
Primary
DFLTFILE
doFlat
Input
Primary
LFLTFILE
doFlat
Input
Primary
MLINTAB
do2D
Input
Primary
PHOTTAB
do2D
Input
Primary
SHADFILE
doShad
Input
Primary
Sample of Possible Values
7. HSTIO Development
The line-by-line I/O and sectional output required by CALACS will be based on additional
functions to HSTIO which have been implemented by Michele de la Peña. This section describes
the changes necessary for these new capabilities.
Since HSTIO is an existing entity, it was important in the design of these new functions to
maintain consistency with the current implementation. In particular, this refers to the nomenclature, and modularity of the functions. For line-by-line I/O, the acquisition of a line closely (but not
exactly!) parallels the acquisition of an entire image. Although in this specific implementation an
IMSET consists of three components (SCI, ERR, and DQ), the design can be extended in the
same manner as HSTIO currently accommodates both the STIS and NICMOS IMSETs.
The new functions for line-by-line I/O will have the following properties:
1. The primary header (globalhdr) and the individual extension headers will only be loaded
once.
2. As typical, the getSingleGroupLine structure should always be initialized before it is used.
3. One level of indirection has been removed as compared to getting an entire image.
4. A corresponding “put” line high-level routine will be written to output a single line to a
single image.
Until there seems to be a real need to output single lines to groups of images, a suite of “put”
routines will be considered low priority and will only be implemented as time permits.
Three new data structures are needed: SingleGroupLine, FloatHdrLine, ShortHdrLine. These
new structures will be nearly identical to the current SingleGroup, FloatHdr and ShortHdr structures within HSTIO, with the addition of a flag to tell whether the header has already been loaded
or not, and the line number.
These new structures will be used by the following new functions for implementing the actual
I/O:
High-Level (public)
void initSingleGroupLine (SingleGroupLine *x);
21
int openSingleGroupLine (char *filename, int extver, SingleGroupLine *x);
int getSingleGroupLine
(char *filename, int extver, int line,
SingleGroupLine *x);
void closeSingleGroupLine (SingleGroupLine *x);
void freeSingleGroupLine (SingleGroupLine *x);
In addition, many new low-level functions were developed to support these cuntions.All other
routines (open, close, acquiring header information, getFloatLine/getShortLine, etc.) are already
available in HSTIO.
The new routines could be used as shown in the following semi-pseudocode. This pseudocode example is to be taken as an illustration of how a line could be obtained from an IMSET to
fill an awaiting buffer. Once the routines are implemented, tested examples will be provided as
illustrations of how the routines should be used.
initSingleGroupLine() {
Initializes data structure SingleGroupLine members
initFloatHdrLine()
Initializes the data structure SciHdrLine members
initFloatHdrLine()
Initializes the data structure ErrHdrLine members
initShortHdrLine()
Initializes the data structure DQHdrLine members
}
openSingleGroupLine() {
This routine incorporates much functionality in setting up to
acquire image lines. After this function is called, the programmer
has access to the length of the data lines so a buffer can now be
allocated dynamically to accommodate the data lines. Internally, the
length has been determined by NAXIS1 (for images with true size) or by
NPIX1 (for constant images).
openInputImage()
Opens the primary and fills the IODesc structure
allocSingleGroupLine()
allocate space for the global (primary) header
allocFloatHdrLine (SCI)
allocFloatLine()
allocHdr()
allocFloatHdrLine() (ERR)
allocFloatLine()
allocHdr()
allocFloatHdrLine() (DQ)
allocShortLine()
allocHdr()
getHeader()
Obtain the primary header information
closeImage()
Closes the primary
getSciHdr() == getFloatHdr()
22
openInputImage()
Opens the SCI extension
getHeader()
Obtains the SCI header
getErrHdr() == getFloatHdr()
openInputImage()
Opens the ERR extension
getHeader()
Obtains the ERR header
getDQHdr() == getShortHdr()
openInputImage()
Opens the DQ extension
getHeader()
Obtains the DQ header
allocSciLine() == allocFloatLine()
Allocates the proper amount of memory for SCI line
allocErrLine() == allocFloatLine()
Allocates the proper amount of memory for ERR line
allocDQLine() == allocShortLine()
Allocates the proper amount of memory for DQ line
}
At this stage, the SCI, ERR, and DQ extensions have not been closed.
getSingleGroupLine() {
Obtain the requested data line from the IMSET
getSciLine() == getFloatLine()
Gets a line from the SCI extension, length based upon NAXIS1
getErrLine() == getFloatLine()
Gets a line from the ERR extension, length based upon NPIX1
getDQLine() == getShortLine()
Gets a line from the DQ extension, length based upon NPIX1
}
closeSingleGroupLine()
Closes the SCI, ERR, and DQ image extensions.
freeSingleGroupLine()
Frees all allocated memory for the data structure
A new public routine has been added to the HSTIO library which allows one to output to a
new file a subsection of an image in memory. This subsection from memory becomes the full size
of the output image. The new function for supporting this output is:
int putSectSingleGroup (char *, int, SingleGroup *, int, int, int, int, int);
In addition, there are several low-level routines which support this function. This one function
can be called by itself in order to handle the output of an IMSET section.
23
8. Development Schedule
Although based heavily on a fully tested software package, this new pipeline will require considerable effort to implement the design changes described in this report. A tentative time line for
this development, with time set aside for adequate testing, can be seen in Table 5. This schedule
breaks down the development of the software into components which can be finished independently and then tested before adding more complexity to the product. This has the benefit of
finding problems early in the development stage when they are easier to identify and to fix, so that
by the time all the software is complete, the majority of problems will already have been
addressed. The final testing phase should consist little more than verification that all the parts
work together as expected, since all the individual parts should already have been debugged.
Finally, time has been implicitly provided for any changes in design that may become necessary
as we learn more about how to share code with STIS and about the data from the instrument itself.
Table 5. Proposed Development Time Line
Develop Time
Develop HSTIO Line-by-Line I/O Functions
Complete Date
1 May 1998
Write CALACS and ACSCCD tasks
4 weeks
1 June 1998
Test CALACS and ACSCCD tasks
1 week
8 June 1998
Write ACS2D
4 weeks
6 July 1998
Test ACS2D with CALACS
2 weeks
20 July 1998
Revise CALSTIS2 into ACSREJ
4 weeks
24 Aug 1998
Test ACSREJ
2 weeks
11 Sept 1998
Revise CALSTIS8 into ACSSUM
2 weeks
25 Sept 1998
Test Full CALACS Package
2 weeks
9 Oct 1998
Revise Full CALACS package based on testing
2 weeks
23 Oct 1998
Support File Requirements for Development
The testing schedule provides the basis for specifying what files need to be delivered at what
time. The first phase of CALACS development focuses on building the CALACS task itself for
controlling the operation of the package and on revising CALSTIS1 code to create ACSCCD.
This will only require the following reference files by 1 June 1998:
Tables similar to STIS files:
ATODTAB
BPIXTAB
24
CCDTAB
New for ACS:
OSCNTAB
provides details on overscan regions for
Bias-level subtraction and trimming
Reference File:
BIASFILE
The second phase for CALACS involves building the ACS2D task, containing the majority of
the basic reduction steps for both CCD and MAMAs. This will require the following reference
files by 6 July 1998 for testing:
MLINTAB - MAMA linearity tables
Darks (MDRKFILE, CDRKFILE)
Flat-fields (LFLTFILE, PFLTFILE, DFLTFILE)
SHADFILE
PHOTTAB
APERTAB
In addition, some test MAMA images would be necessary for testing of the MAMA specific
routines.The next phase will be the building of the cosmic-ray rejection task, ACSREJ. The following reference file will be needed by 24 Aug 1998 for testing:
CRREJTAB
Some CR-SPLIT CCD images would be necessary for this testing as well.
The format of the reference tables, rather than the specific values in them, will be critical to
insure that CALACS can properly read and interpret the contents of each reference table. Those
tables which have STIS equivalents could simply be STIS tables with headers changed to reflect
ACS specific information. The actual values in the tables are expected to be volatile, and are not
critical to the initial testing. The ACS CCD images in FITS format needed to conduct the basic
testing will most likely need to be hand-tailored by to include all the necessary keywords and
proper values for CALACS in their headers, both the primary and extension headers.
9. Summary
The development of CALACS can be described as a simplified version of CALSTIS, with different memory usage and image I/O. Initial work on CALACS would start with a complete copy
of the necessary CALSTIS tasks; namely, CALSTIS0, CALSTIS1, CALSTIS2, and CALSTIS8.
Unnecessary code related to spectroscopy would then be removed, and variable definitions would
be trimmed to fit the needs of ACS. At this point, the line-by-line I/O and association handling
25
would be added to produce the final product. This development will take place on a snapshot of
the CALSTIS software and evolve separately to support ACS without any concern for sharing
functions or tasks with STIS.
The overall design of CALSTIS makes this development efficient and easy to implement.
CALACS will not just build on this modularity, but also provide a basis for efficiently implementing the desired enhancements in the future. In addition, only minimal enhancements to HSTIO
have been found to be required to support this calibration software for ACS. Therefore, CALACS
should be easier to develop than previous calibration software, and no more difficult to maintain
or enhance in the future.
26
Appendix A: CALSTIS Structures and Variables
This section describes the primary structures and variables used within CALSTIS. These
structures are used primarily to determine what processing needs to be performed, and to keep
track of the results of the processing so that the image headers can be updated properly. Throughout CALSTIS, similar structure names are used for the structure containing the relevant header
keywords: StisInfo in CALSTIS0, StisInfo1 in CALSTIS1, and StisInfo8 in CALSTIS8.
CALSTIS0 Structure
CALSTIS0 acts as a primary switching station for the observations, sending the data to the
tasks for processing appropriate for the observation type. The structure StisInfo contains all the
necessary information for making these decisions, as shown in Table 6.
Table 6. Definition of Structure StisInfo.
Variable
Description of variable
input, outroot, and temporary file names
char rawfile[STIS_LINE];
uncalibrated science data
char outroot[STIS_LINE];
root name for output
char wavfile[STIS_LINE];
input wavecal file
char crjfile[STIS_LINE];
CR rejected, flat fielded science
char fltfile[STIS_LINE];
flat fielded science
char x1dfile[STIS_LINE];
extracted 1-D spectrum
char x2dfile[STIS_LINE];
rectified 2-D spectrum
char sx2file[STIS_LINE];
summed, rectified 2-D
char sx1file[STIS_LINE];
summed, extracted 1-D
char sflfile[STIS_LINE];
summed, not rectified 2-D
char blv_tmp[STIS_LINE];
blevcorr, then CR flagged
char crj_tmp[STIS_LINE];
CR rejected, summed
char fwv_tmp[STIS_LINE];
flat fielded wavecal
char cwv_tmp[STIS_LINE];
FF, source-subtracted wavecal
char w2d_tmp[STIS_LINE];
2-D extracted wavecal
char rootname[STIS_CBUF];
root name for set of obs
info about input science or wavecal file
int detector;
integer code for detector
27
Variable
Description of variable
int obstype;
spectroscopic or imaging
int nimages;
number of “groups” in file
Info for comparing the binning and gain of the science file
and wavecal. If they’re the same (only relevant for the CCD),
the samebin flag should be set to one.
int scibin[2], wavbin[2];
binning factors
int scigain, wavgain;
ccdgain values
int samebin;
calibration switches
int sci_basic_2d_a;
do calstis1a? (dqicorr or blevcorr)
int sci_basic_2d;
do calstis1 for science file?
int sci_expscorr;
do calstis1 on CR flagged but non-summed?
int sci_crcorr;
do cosmic-ray rejection for science file?
int sci_rptcorr;
combine repeatobs science data?
int sci_wavecorr;
do calstis11, calstis4, and calstis12?
int sci_2d_rect;
do calstis7 for science file?
int sci_1d_extract;
do calstis6?
int sci_geocorr;
do 2-D image rectification?
int wav_basic_2d;
do calstis1 for wavecal?
int wav_subscicorr;
do calstis11? (not a header switch)
CALSTIS1 Structure
CALSTIS1 controls a large number of processing steps and requires a great deal of information from the image header to manage that calibration steps. This results in a structure which
contains more keywords and header information than the similar structure in CALSTIS0. The definition of this structure is given in Table 7. In addition to the keyword values, this structure also
contains specific information about the format of the observation, information such as binning
amount, whether the observation is a subarray, and so on. Since most functions require a large
amount of input information, this structure provides a simple way to pass all this information
around.
28
Table 7. Definition of CALSTIS1 Structure StisInfo1.
Variable
Description of Variable
input and output image names
char input[STIS_LINE];
input image to be calibrated
char output[STIS_LINE];
output calibrated image
char outblev[STIS_LINE];
output text file for bias levels
char rootname[STIS_CBUF];
root name for set of obs
char assoc_typ[STIS_CBUF];
ACCUM, or WAVE for a wavecal
command-line flags
int printtime;
print time after each step?
int verbose;
print additional info?
keywords and file names for reference files
RefFileInfo *refnames;
info about input image
char det[STIS_CBUF];
name of detector
char aperture[STIS_CBUF];
aperture name
char opt_elem[STIS_CBUF];
name of grating or mirror
char obsmode[STIS_CBUF];
e.g. ACCUM or TIMETAG
int detector;
integer code for detector
int ncombine;
number previously summed together
int nimages;
number of “groups” in file
double exptime;
exposure time
short sdqflags;
serious data quality values
coordinate info
int subarray;
is current image a subarray?
int bin[2];
size of pixel in detector coordinates
int dispaxis;
dispersion axis, 1 ==> X, 2 ==> Y
int dispsign;
sign of CD1_1 (dispersion), +1 or -1
MAMA-specific info (regarding linearity)
double global_limit;
count rate for 10% global nonlinearity
double tau;
time constant for global nonlinearity
29
Variable
Description of Variable
double local_limit;
count rate for 10% local nonlinearity
float expand;
radius for flagging local nonlinearity
MAMA-specific info (regarding Doppler shift)
double globrate;
global count rate
double expstart;
exposure start time (MJD)
double doppzero;
Doppler shift zero phase time (MJD)
double doppmag;
Doppler shift magnitude (high-res pixels)
double orbitper;
Assumed HST orbital period (seconds)
CCD-specific info
float bias_rej;
number of sigma for rejecting outliers
char ccdamp[STIS_CBUF];
CCD amplifier read out (A,B,C,D)
int ccdgain;
commanded gain of CCD
int ccdoffset;
commanded offset
int binaxis[2];
BINAXIS1, BINAXIS2 from header
float atodgain;
actual gain of CCD
float ccdbias;
CCD bias offset number
float readnoise;
readout noise
float saturate;
CCD saturation level
calibration flags (switches)
int doppcorr;
Doppler convolution needed (flats, etc)?
int lorscorr;
convert MAMA data to low-res
int dqicorr;
data quality initialization
int atodcorr;
analog to digital correction
int blevcorr;
subtract bias from overscan
int biascorr;
subtract bias image
int glincorr;
global nonlinearity correction
int lflgcorr;
flag local nonlinearity
int darkcorr;
subtract dark image
int flatcorr;
apply flat field
int pfltcorr;
apply pixel-to-pixel flat
30
Variable
Description of Variable
int dfltcorr;
apply delta flat
int lfltcorr;
apply low-order flat
int shadcorr;
correct short exposures for shutter time
int photcorr;
add photometry header keyword values
int filtcorr;
include filter throughput
filtcorr is not a switch; it’s associated with photcorr
int noisecorr;
initialize error array? (yes)
int statcorr;
compute statistics? (yes)
calibration images and tables
RefImage bias;
bias image
RefImage dark;
dark image
RefImage pflt;
pixel-to-pixel flat field
RefImage dflt;
delta flat
RefImage lflt;
low-order flat
RefImage shad;
shutter shading correction image
RefTab bpix;
bad pixel table
RefTab ccdpar;
CCD parameters table
RefTab mlin;
MAMA nonlinearity info table
RefTab atod;
analog to digital correction table
RefTab phot;
photometric throughput table
RefTab apertab;
aperture (filter) throughput table
31