Space Telescope Science Institute
WFC3 PIPELINE CDR
Daryl Swade
Post SM4 Science Data Volume
Handling in OPUS and HDA
October 16, 2001
Data Volume Handling
1 of 8
Space Telescope Science Institute
WFC3 PIPELINE CDR
Daryl Swade
Science Data Volume Input to OPUS
 As shown in earlier presentation: WFC3 data volume
and HST total data after SM4 are estimated in ISR
WFC3 2001-02 (Lisse, Henry, Knezek, and Hanley).
 Typically daily downlink for WFC3 – 11.7 Gbits in 50
exposures.
 22.6 Gbits/day downlink volume from all science
instruments.
October 16, 2001
Data Volume Handling
2 of 8
WFC3 PIPELINE CDR
Daryl Swade
Space Telescope Science Institute
WFC3 Typical File Sizes
Exposure type POD
raw
ima
flt/crj
(MB)
drz
(min)
(MB)
drz
(max)
(MB)
(MB)
(MB)
(MB)
UVIS
ACCUM
38
34
N/A
164
~180
~461
IR
MultiAccum
38
33
168
17a
~12b
~35
aMultiple
readouts combined
bNumber of image extensions reduced from 5 to 3
October 16, 2001
Data Volume Handling
3 of 8
WFC3 PIPELINE CDR
Daryl Swade
Space Telescope Science Institute
WFC3 Typical Dataset Sizes
Observation type
OPUS
(pod+raw+cal)
(MB)
Into HDA
(pod+raw)
(MB)
Distribution
(cal + raw)
(MB)
Expansion
factor*
UVIS:
1 image
416
72
378
10.9
UVIS:
CR-SPLIT=2
488
144
450
6.4
UVIS:
CR-SPLIT=2 DITHER=4
1432
576
1394
6.3
IR:
1 image
268
71
230
7.1
IR:
RPT-OBS=2
541
142
503
7.1
IR:
RPT-OBS=2 DITHER=4
2131
568
2093
9.3
October 16, 2001
Data Volume Handling
4 of 8
Space Telescope Science Institute
WFC3 PIPELINE CDR
Daryl Swade
Science Data Processing Working Disk Space
 Assumptions
–
–
WFC3 and ACS dominate data volume (21.6 of 22.6 Gbits
downlinked per day).
ACS expansion factor similar to WFC3 (ACS/WFC and
WFC3/UVIS detectors similar, same drizzle package, etc.)
 22.6 Gbits/day downlink
 Expansion factor = 10
 28 GBytes working disk space per day
*Expansion factor = size(pod) + size(raw) + size (cal)
#exposures * size(pod)
October 16, 2001
Data Volume Handling
5 of 8
WFC3 PIPELINE CDR
Daryl Swade
Space Telescope Science Institute
WFC3 Calibration Reference File Disk Space
Static (GB)
6 year (GB)
UVIS
IR
28
7
87
40
TOTAL
35
127
Very preliminary estimates – depend
on on-orbit behavior of instrument and
need for updates.
October 16, 2001
Data Volume Handling
6 of 8
Space Telescope Science Institute
WFC3 PIPELINE CDR
Daryl Swade
DADS Data Ingest
 Only pod and raw datasets written to archive optical
media.
 For WFC3 and ACS, this is about 72 MB per exposure.
 Estimate 50 exposures/day for WFC3 and a similar
number for ACS.
 7 GB/day into HDA
 HST science data ingest rate for August 2001 was 2.8
GB/day.
October 16, 2001
Data Volume Handling
7 of 8
Space Telescope Science Institute
WFC3 PIPELINE CDR
Daryl Swade
DADS Data Distribution
 In August 2001, external users retrieved HST science data at a
rate of 10.9 GB/day in 557 datasets/day (4-5 times dataset ingest)
 These retrievals average 20.0 MB per dataset.
 WFC3 dataset size ranges from 230 – 2093 MB in examples on
previous slide.
 We could anticipate a factor of 10 – 100 increase in HST science
data flow to external users if WFC3 and ACS come to dominate
retrievals.
–
To be mitigated by use of CD-ROMs and DVDs for large data volume
retrievals.
October 16, 2001
Data Volume Handling
8 of 8