SCIAMACHY Algorithm Development and Data Usage Meeting
May 28-29, 2002
DLR Oberpfaffenhoffen
Agenda
May 28
1330 Opening Remarks (SADDU Chairman)
 Review of Data Product matrix
1400 Brief summary of SCIAMACHY performance (ESA)
1415




Overview of the SCIAMACHY L12 off-line processor (DLR)
General remarks
Implementation
Status
Documentation
1530 Coffee
1600






Status of limb algorithm development
Status of prototype (SAO)
Operational implementation of prototype (DLR)
Requirements and developments on limb p,T retrieval (DLR)
New approaches for p,T retrieval (SAO)
KNMI limb retrieval algorithm (KNMI)
U. Bremen limb retrieval algorithm (IFE)
May 29
0900 Limb work-around solution for early validation period (DLR)
0945 Limb data requirements for early validation period (KNMI)
 Coordination with the validation activities
 Mission planning
1030 Coffee
1100 Data policy issues




ESA data policy for L1, L2, and higher products (ESA)
Agency data policy for L1, L2, and higher products (DLR, NIVR)
Publication during validation period (SAO)
Operational versus scientific data products (SAO)
1200 Lunch
1330 Availability of data for algorithm validation (ESA, DLR)
1400 Commissioning phase algorithm refinements (SAO)
1430 Algorithm updates - examples and discussion (ESA)
 GOME Data Processor V3 Upgrade - User Consultation, Decision
Process, Implementation, Validation, Results
1500 AOB
 Proposal for SCIAMACHY data inventory (SAO)
 Time and place of next meeting (All)
Attending:
Kelly Chance
Albrecht von Bargen
Hans Schrijver
Ahilleas Maurellis
Klaus Bramstedt
Christian Muller
Johannes Frerick
Claus Zehner
Wolfgang Balzer
Andreas Richter
Johannes Kaiser
Tom Rother
Sander Slijkhuis
Ankie Piters
Jeroen van Gent
Piet Stammes
Achim Friker
SAO
DLR-IMF
SRON
SRON
IFE/IUP Bremen
B.USOC-BIRA-IASB
ESA-ESTEC
ESA-ESRIN
DLR-MF
IUP Bremen
IUP Bremen
DLR
DLR-IMF
KNMI
KNMI
KNMI
DLR-RD-RE
+1-617-4957389
+49-8153-282742
+31-30-2538728
+31-30-2535745
+49-421-2184584
+32-2-3730372
+31-71-5655457
+39-06-94180544
+49-8153-281893
+49-421-2184474
+49-421-2184352
+49-8153-281404
+49-8153-281412
+31-30-2206433
+31-30-2206799
+31-30-2206449
+49-228-447397
Action Items
kchance@cfa.harvard.edu
albrecht.von-bargen@dlr.de
h.shrijver@sron.nl
a.n.maurellis@sron.nl
klaus.bramstedt@iup.physik.uni-bremen.de
christian.muller@oma.be
johannes.frerick@esa.int
czehner@esa.int
wolfgang.balzer@dlr.de
richter@iup.physik.uni-bremen.de
johannes@uni-bremen.de
tom.rother@dlr.de
Sander.slijkhuis@dlr.de
piters@knmi.nl
gentjvan@knmi.nl
stammes@knmi.nl
Achim.friker@dlr.de
S. Slijkhuis: make a narrow-swath proposal to the SOST team, to be sure we
have enough information from different scan strategies to later consider
optimizing scanning strategy (June 17).
A. Piters: Send SCIAVALIG and NILU protocols to J. Frerick. (June 17.)
J. Frerick: Check that protocols and ESA requirements are consistent. (June
24).
W. Balzer and J. Frerick review the list of software tools on the validation
website for possible update. (June 24).
K. Chance: email A. Richter, A. von Bargen, W. Balzer with current
understanding of the process for algorithm refinement during commissioning
phase. (June 17).
K. Chance: Determine why the absorbing aerosol index implemented for
SCIA is different from the TOMS AAI algorithm. (Closed – the DPM has
been updated to show that they are the same).
All: Provide suggestions to K. Chance for the form and content of a
SCIAMACHY data inventory for scientific use. (Next SADDU meeting).
Minutes
Review of Data Product matrix
It is noted that the cloud algorithm implemented for launch is the DLR
OCRA cloud fraction. No aerosol algorithm has yet been proposed except
for the UV TOMS-type absorbing aerosol index (AAI) which has been
implemented now for several years.
The current OL Data product matrix is available with these minutes.
Brief summary of SCIAMACHY performance
J. Frerick presented an overview of SCIAMACHY performance
(presentation available with these minutes): SCIAMACHY is, so far, “fully
functional,” but problems with the Envisat Payload Module Computer
(PMC) have delayed the full SCIAMACHY turn on.
At this time, no l1b data have yet been processed and disseminated and there
is thus also no l2 data available. 0-1 parameters are now fixed and we should
be ready for the next data, when the satellite is back up. The hope is that the
final door will be open and that there will be l1b data in June.
Overview of the SCIAMACHY L12 off-line processor
Both OL and operational support are done with DLR (Oberpfaffenhofen)
internal funding. NRT 0-1 and 1-2 processing is supported through the route
DLR-Bonn → ESA → NRT developer.
The 1-2 OL operational baseline for O3 is GDP version 2.4, with data input
from version 2.7, except FM spectra, which are from SCIA (ready July 1).
Other nadir UV/vis products are also ready July 1. Limb UV is ready
November 30.
Prototypes that will be ready November 30 include GDP 3.0, PTH limb, and
the line-by-line code for limb.
Status of limb algorithm development
Wavelength windows for limb measurements are currently expected to
change over three pressure height regimes.
For limb ozone measurements the current default is the use of the SCIA FM
cross sections, over the 20-45 km range (at least at the start) and (variable)
wavelength windows in the 320-360+ nm range. There are lots of trials of
the limb algorithm for ozone at DLR, using the 0-1b prototype processor to
generate “calibrated” radiances and fitting to limb scan ratios. The trials
include comparisons to the DLR NGONG algorithm. Fitting so far, for
cloud-free (Sahara) scenes gives ~1% rms values.
Q: Do we have enough information from different scan strategies to later
optimize scanning strategy?
A: S. Slijkhuis should make a narrow-swath proposal to the SOST team.
K. Chance presented a summary of the off-line algorithm prepared by R.
Spurr (available with these minutes).
K. Chance presented the knee algorithm developed for OSIRIS and modified
for SCIAMACHY by C. Sioris, now of the SAO.
J. van Gent presented the KNMI/SRON SCIARALI radiative transfer model
for SCIAMACHY limb. It is estimated to be ca. 2 months from working.
The presentation is available with these minutes.
J. Kaiser presented IFE/IUP progress in limb algorithm development
(presentation available with these minutes), which includes both the
SCIATRAN/CDIPI and SCIARAYS radiative transfer approaches and
retrievals by three different methods.
Limb work-around solution for early validation period
As the first limb algorithm is not scheduled for operational implementation
until November 30, 2002 and the validation measurements start substantially
before that (in fact, this summer) it has been necessary to produce a “limb
work-around” (LWO) procedure, whereby limited limb products will be
produced by hands-on work with the limb prototype. The LWO products are
not “official” data products.
It is estimated that the implementation of the LWO will require ~5 FTE
weeks of work, starting in early July and finishing by August 1. 5 FTE
weeks is only an estimate, based on preliminary knowledge of the minimum
necessary limb data products for early validation (the actual amount is
highly correlated to the number of orbits to be processed by hand).
SCIAVALIG will send a list of states to DLR, which will then revise the
estimate of labor for preparation to run the LWO products. The focus will be
on O3 and NO2, with CH4 perhaps eventually also being produced (although
this was thought by some to be very unlikely).
It was not clear whether the estimated 5 FTE weeks of effort would impact
the November 30 operational target date by a similar amount (DLR
estimates that the limb target date will be affected by a similar amount).
Limb data requirements for early validation period
K. Bramstedt presented an overview of early validation activities for which
such data will be necessary between July and October (presentation
available with these minutes).
C. Muller noted that 26 July is the nominal commissioning date, although
that is now likely to change with delays due to the PMC.
J. Frerick noted that the minimum requirement for commissioning is that
SCIA is operational and taking data, rather than successful processing at
first. It is more a case of “verification” than “validation.”
A. Piters inquired about the possibility to process the preliminary data
products with different algorithm and input parameter settings and make
these available to the validation scientists. Validation scientists can aid in the
fine tuning of the algorithm parameters, by comparing their measurements
with SCIA data processed with different parameters. S. Slijkhuis answered
that this should in principle be possible.
Data policy issues
C. Zehner gave a presentation on ESA data policy (available with these
minutes). He reiterated the categories of users:
Category 1 – Research (free products)
Category 2 – Commercial
ESA does not currently plan to produce any level 3 data products. The list of
users is available at http://projects.esa-ao.org.
It is now possible to apply for Category 1 status at any time (as opposed to
the past requirement to respond to an AO). Currently, AO responders receive
data free, including production costs; outside the AO it may be necessary to
pay production costs (although the Program Board may decide to make them
totally free).
A. Piters asked about the procedure for new cal/val proposals. C. Zehner
replied that in principle the deadline has passed, but individual cases might
be included in the cal/val when it is considered to be essential. (E. Attema
notes that after the commissioning phase and after the transfer of the mission
to from ESTEC to ESRIN the ACVT structure will survive for at least
another 9 months.)
Category 1 data users are not restricted from producing and distributing their
own L2 data, except that distribution must be limited to other Category 1
users if an ESA product also exists.
J. Frerick states that publication (including web sites, conferences and
meetings) during the commissioning and validation periods should be
cleared in advance through G. Levrini and E. Attema of ESA.
A. Piters argued that all AO PIs have signed several protocols in which these
issues are cleared. She does not recall needing “prior written permission”
from ESA for publication, although of course proper acknowledgements and
copyright statements are required. She will send the NILU, SCIAVALIG,
and ESA protocols to J. Frerick, who will check into the situation. (Question
from the chair: do they require “prior written permission” from
SCIAVALIG and/or NILU?) E. Attema suggests in reviewing the minutes
that if there is any problem with protocol inconsistency, the ESA one be
used because of its contractual significance.
C. Muller noted that the Belgians plan to use the Netherlands data center for
re-distribution (with ESA permission).
C. Zehner noted that the AO countries (DE, NL, BE) can re-distribute data
to anyone within their own country.
The FODP is not yet signed. At issue is who gets what data under what
conditions, particularly whether there would be an agreed list versus
informing ESA after the fact.
The Netherlands is willing to re-distribute data products to other countries
when required, and with ESA's approval.
Algorithm development, new SCIAMACHY products
New data products are highly encouraged (A. Friker) but may take
substantial time to implement (P. Stammes, with the example of the
FRESCO cloud algorithm, which is at DLR but is not implemented). P.
Stammes asked what funding is available for such upgrades. (FRESCO is P4
– peer-reviewed, published, publicly-available, and in the processor!)
OL updates get a Δ-validation
NRT updates get a δ-validation
Also on the topic of validation, W. Balzer noted that DLR validates
algorithms with a test data set as part of the verification process.
A. Piters stated that scientific development of SCIAMACHY algorithms, in
parallel to the operational algorithm improvement, is essential for the
scientific output of SCIAMACHY. The validation groups will be
encouraged to participate in the validation of such products.
A. Friker noted that some of the longer-term validation funds were used up
by the extended pre-launch.
C. Muller Noted that ESA has no chemistry mission scheduled for the next
10 years. Envisat could be requested to be the GMES “big project.”
Availability of data for algorithm validation
J. Frerick noted that data are currently limited to the Kiruna orbits. This will
eventually improve. Ground rules during this commissioning period are
defined by the ESA GSOCP document.
W, Balzer, in collaboration with J. Frerick, will maintain a website to enter
results, objectives, tasks, etc.
A small amount of l1b data will initially be made available through the
ATMOS website (button below SOST) – perhaps 1 or 2 orbits worth (A.
Richter suggests one orbit with both continents and ocean); this will come
after the instrument I fully operational and will use a nominal orbit state.
The date for this will be end of June to mid-July (in late June the in-flight
calibration auxiliary files for 0-1b processing may or may not be done).
LWO limb data will also be made available on the ATMOS web server.
A. Richter asked in which format the lv2 limb off-line product will be
provided to the validation groups. J. Frerick answered that the format will be
the ENVISAT PDS format, but that EnviView does not currently support
this specific format. He added that, to his knowledge, implementation of the
lv2 limb off-line products into EnviView is currently not foreseen. A.
Richter pointed out, that supplying data in a complex format without tools to
convert it to standard formats (ASCII, HDF, NetCDF,...) is not acceptable,
and that validation groups will not be able to develop PDS reading tools on
their own. Therefore, without a data conversion tool, validation of the lv2
limb off-line products will not be possible. W. Balzer noted that he has
written a LINUX tool to convert the limb off-line products to ASCII format,
and that he is willing to provide it to the validation groups with the data. It
was agreed that implementing lv2 limb off-line products into EnviView has
a high priority, in order to facilitate SCIAMACHY data usage by the
scientific community. (This point was also stressed post-meeting by other
scientists.)
W. Balzer and J. Frerick are requested to review the list of software tools on
the validation website for possible update.
Commissioning phase algorithm refinements
A. Richter proposes that minimization of χ2 and reduction of artifacts as
found in GOME be primary criteria for optimization of fitting.
We will spend about a month or less optimizing, after first getting the test
orbits. Choices will apply to both OL and NRT.
The timing is a bit tricky; the September 9-13 commissioning phase review
is very late to optimize the fitting; however, the final flight states are only
determined at the end of SODAP (end of July), and at least a few weeks are
needed to optimize and verify the baseline choices. The goal is to finalize
the algorithm baseline by September 9, just before the SODAP and
commissioning phase reviews.
K. Chance will email A. Richter, A. von Bargen, and W. Balzer with his
current understanding of the process for algorithm refinement during
commissioning phase.
Algorithm updates - examples and discussion
A. Piters showed a viewgraph from the FODP on the Change Control Board.
C. Zehner have a presentation on “GOME Data Processor V3 Upgrade User Consultation, Decision Process, Implementation, Validation, Results”
as an example of the update process (available with these minutes).
It was generally acknowledged that at least one year will be required to
implement algorithm improvements, although many would prefer faster
implementation.
Any Other Business
P. Stammes presented a sensitivity study on aerosols, and asked why the
absorbing aerosol index implemented for SCIA is different from the TOMS
AAI algorithm. K. Chance took an action to investigate.
T. Kurosu proposes that there be a data inventory for SCIAMACHY users,
containing such useful information as geolocation, time, measurement mode,
for convenient lookup of desired data products. DLR has something similar
now on the web from B. Aberle, and J. Frerick also has something. A group
action was set to suggest what should go into an improved inventory.
Time and place of next meeting
October 8/9, 2002 at RAL