E15 Data Processing Issue
6/18/12
UVIS Team Meeting
1
Problem description
• No match between the FUV data and the
minimum ray high from geometer.
– several seconds discrepancy.
• Good match of the HSP data with geometer.
• We were expecting 379 integrations and only
371 could be found in DAPS.
• Manual processing the data with the
(modified) ATLO code shows the code was
skipping one integration every 38 integrations.
6/18/12
UVIS Team Meeting
2
Problem Description
• The problem seems to have been introduced
by a code fixe done 11/98 after ICO.
• The problem is not restricted to the E15 data
set.
• The problem potentially affects all FUV and
EUV data sets.
6/18/12
UVIS Team Meeting
3
Problem Description
• The data processing code skips integrations.
• The frequency at which integrations are
skipped depends on the instrument setup.
• For most of the 1-window data set it is 1
integration every 471 continuous integrations
or 157 continuous integrations
– The E15 set up was a 2-window data set
6/18/12
UVIS Team Meeting
4
Which Data Sets Are Affected
• All the data set taken continuously with a
short integration time can be affected:
– Rolling IPHSURVEY (12.5 or 25 seconds integration
time for 9 hours).
– Some aurora imaging (8.0 seconds integration
time for several hours).
– All occultation data sets (1 second to 5 seconds
integration time).
6/18/12
UVIS Team Meeting
5
Examples
• Jupiter/Sigma Leo (C23, 2001-07)
– 1 sec integration, 882 integrations, missing 1
integration (#471)
• UVIS_157SW_IPHSURVEY004_RIDER (S71, 2011251)
– 12.5 sec integration, 2585 integrations, missing 5
integrations (471, 942….)
• UVIS_157SA_AURSTARE001_PIE (S71, 2011-327)
– 8 sec integration, 1087 integrations, missing 2
integrations (471, 942)
6/18/12
UVIS Team Meeting
6
Examples
• UVIS_138SA_USUNOCC001_PRIME (S70, 2011-255)
– 1 sec integration, 3349 integrations, 7 integrations missing
6/18/12
UVIS Team Meeting
7
Since Last Meeting (1)
• A yearly list of affected data sets was provided
last March (see backup slides) for 2004 to
2011
– Not included in the list are the IPHSURVEYs when
the S/C is rolling (that is a large number of data
set)
– As expected: atmospheric occultation, Icy satellite
occultation, some ring occultation, early system
scan, some aurora scan
– But also all the IMPACT
6/18/12
UVIS Team Meeting
8
Since Last Meeting (2)
• The code has been fixed
• Regression testing is in progress (looks Ok so far)
• Regression test step 1
DAPS
Compare
level 1 data
set
Selected un-affected
level 0 files
Test successful if
1. Metadata identical
2. Data identical
Test DB
6/18/12
UVIS Team Meeting
9
Since Last Meeting (3)
• Regression testing step 2
DAPS
Compare
level 1 data
sets
Selected affected level
0 files
Test successful if
1. Metadata identical
2. Missing integrations
are recovered
Test DB
6/18/12
UVIS Team Meeting
10
What’s Next
• Decide what to re-process
• Re-process as needed (DAPS)
– Alain and Dave
• Re-deliver the PDS volumes that are affected
– Dave
6/18/12
UVIS Team Meeting
11
Reprocessing Overview
1. Copy current DAPS database to a
development database
2. On a sequence basis (or yearly basis)
– Delete the affected data sets
– Re-process the affected data sets
– Verify the integrity of the database
3. When all done copy the development
database to the DAPS database
6/18/12
UVIS Team Meeting
12
Backup Slides
6/18/12
UVIS Team Meeting
13
2004
6/18/12
UVIS Team Meeting
14
2005 (1)
6/18/12
UVIS Team Meeting
15
2005 (2)
6/18/12
UVIS Team Meeting
16
2006 (1)
6/18/12
UVIS Team Meeting
17
2006 (2)
6/18/12
UVIS Team Meeting
18
2007
6/18/12
UVIS Team Meeting
19
2008 (1)
6/18/12
UVIS Team Meeting
20
2008 (2)
6/18/12
UVIS Team Meeting
21
2009
6/18/12
UVIS Team Meeting
22
2010
6/18/12
UVIS Team Meeting
23
2011
6/18/12
UVIS Team Meeting
24