Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
1 of 16
EarthCARE Level 2 Documentation
ACM-CAP
Cloud and Precipitation Best Estimate
Software Requirements Specification
(SRS)
VARSY Project
Code:
Issue:
Date:
Reference:
L2-ACM-CAP-SRS
0.1
27/08/2012
Name
Function
Prepared by
Robin Hogan
Project Scientist
Reviewed by
Pavlos Kollias
Project Scientist
Approved by
Pavlos Kollias
Project Manager
Signatures and approvals on original
Signature
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
2 of 16
This page intentionally left blank
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
3 of 16
Document Information
Contract Data
Contract Number:
4000104528/11/NL/CT
Contract Issuer:
ESA-ESTEC
Internal Distribution
Name
Unit
Copies
Pavlos Kollias
McGill University
1
Aleksandra Tatarevic
McGill University
1
Wanda Szyrmer
McGill University
1
Internal Confidentiality Level
Unclassified

Restricted
Confidential


External Distribution
Name
Organisation
Copies
Tobias Wehr
ESA-ESTEC
1
Michael Eisinger
ESA-ESTEC
1
Dulce Lajas
ESA-ESTEC
1
Robin Hogan
University of Reading
1
LATMOS
1
Gerd-Jan van Zadelhof
KNMI
1
David Donovan
KNMI
1
University of Leicester
1
Julien Delanoë
Alessandro Battaglia
Archiving
Word Processor:
MS Word 2003
File Name:
VARSY-READING-SRS-001
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
4 of 16
Document Status Log
Issue
01-draft
Change description
Draft version.
Date
Approved
25/08/2012
Contents
1. Purpose and Scope.............................................................................................................................. 6
1.1. Applicable Documents ................................................................................................................. 6
1.2. Reference Documents.................................................................................................................. 7
1.3. List of Abbreviations .................................................................................................................... 8
2. Compliance Matrix .............................................................................................................................. 9
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
5 of 16
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
6 of 16
1. PURPOSE AND SCOPE
This document specifies the software requirements associated with the implementation of the
ACM-CAP L2b algorithm.
1.1. Applicable Documents
Table 1: Applicable Documents
Reference
Code
Title
Issue
[SOW]
EC-SW-ESA-SY-0310
Statement of Work: VARSY - 1-Dimensional
VARiational Retrieval of SYnergistic
EarthCARE Products
1.0
[CC]
Appendix 2 to
AO/1-6823/11/NL/CT
Draft Contract (attachment to SOW)
1.0
[AD 1]
EC-SW-ESA-SY-0152
EarthCARE Level 2 Processor Development
1.0
General Requirements Baseline
[AD 2]
EC.ICD.ASD.SY.00004
EarthCARE Product Definitions. Vol. 0:
Introduction
[AD 3]
EC.ICD.ASD.SY.00005
EarthCARE Product Definitions. Vol. 1:
1.0
Common Product Definitions
[AD 4]
EC.ICD.ASD.ATL.00021
EarthCARE Product Definitions. Vol. 2b:
1.0
ATLID level 1
[AD 5]
EC.ICD.ASD.BBR.00022
EarthCARE Product Definitions. Vol. 3b:
1.0
BBR level 1
[AD 6]
EC.ICD.ASD.MSI.00023
EarthCARE Product Definitions. Vol. 4b:
1.0
MSI level 1
[AD 7]
ECSIM-DMS-TEC-ICD01-R
ECSIM Simulator Interface Control Document
[AD 8]
PE-TN-ESA-GS-0001
Ground Segment: File Format Standard
1.0
[AD 9]
EC-TN-ESA-GS-0218
Tailoring of the Earth Explorer File Format
Standard for the EarthCARE Ground Segment
2.0
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
7 of 16
1.2. Reference Documents
Table 2: Reference Documents
Reference
Code
Title
[RD1]
ECSIM-DMS-TEC-SUM-01-R
ECSIM System User Manual
[RD2]
ECSIM-KNMI-MAD01-R
ECSIM Model and Algorithms Document
[RD3]
EE-MA-DMS-GS-0001
Earth Explorer Mission CFI Software:
Issue
General Software User Manual
[RD4]
EOP-SM/1567/TW
[ATLAS-FR] EC-FR-KNMI-ATL-027
EarthCARE Mission Requirements Document
ATLAS Final report
[ATLASACM-TC]
EC-TN-KNMI-ATL-ACM-TC-024 L2b Classification ATBD
[ATLASEBD]
EC-TN-KNMI-ATL-ATBD-A-EBD- L2a ATLID Extinction, Backscatter and
021
Depolarization algorithm ATBD
[ATLAS-FM] EC-TN-KNMI-ATL-ATBD-A-FM- L2a ATLID Feature mask ATBD
010
[RATEC-FR] RATEC-FR-READING-1
RATEC Final Report
1.0
1.2
13/03/08
1.1
27/04/09
2.2
1.0, April
2011
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
8 of 16
1.3. List of Abbreviations
Table 3: List of abbreviations
Abbreviation
Name
1D-VAR RS
1-dimensional variational retrieval scheme
ATLID
Atmospheric Lidar (The EarthCARE lidar)
CASPER
Cloud and Aerosol Synergetic Products from EarthCARE retrievals
CPR
Cloud Precipitation Radar (The EarthCARE radar)
EarthCARE
The Earth Clouds, Aerosols and Radiation Explorer
ECSIM
EarthCARE Simulator
HSRL
High-Spectral Resolution Lidar
MSI
Multi-spectral Imager (The EarthCARE imager )
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
9 of 16
2. COMPLIANCE MATRIX
The following table lists all the requirements in the General Requirements Baseline [AD1] related
to the software and the degree of compliance with each requirement.
*C=compliant, P=partly non-compliant, N=not compliant, NY=not yet compliant but will be,
NA=Not applicable
Reference Requirement
Implementation Compliance*
Design Requirements
DES-1
The level 2 processor shall be implemented as a
set of independent executable entities, referred to
below as level 2 modules, each one implementing
a specific algorithm generating a specific level 2
data product.
The ACM-CAP
is implemented
as a standalone
executable
C
DES-2
No proprietary libraries or code shall be used
Libraries used: C
GNU Scientific
Library (GPL),
LIDORT
(permissive
open source
license),
“Adept”
automatic
differentiation
library (GPL),
Multiscatter
library (LGPL);
note that Hogan
is copyright
holder for
“Adept” and
“Multiscatter”;
Mishchenko’s
T-matrix code
(permissive
BSD license);
BHMIE does
not specify a
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
10 of 16
license so have
emailed Bruce
Draine.
DES-3
Only portable libraries or code shall be used.
In this context, "portable" means that any
libraries or code invoked within a level 2 module,
including bespoke libraries and code, must not
depend on any special features of, e.g., the
operating system, the compiler, the specific
programming language implementation on the
development or target platform.
The ACM-CAP N
currently uses
an extension of
the GNU C++
compiler in
permitting
variable-length
arrays (a C
feature) in
C++.
Functional Requirements
FUN-1
Level 2 modules shall derive geophysical
See associated
quantities (level 2 data products) from
ATBD
EarthCARE level 1b data products. Synergistic
(level 2b) modules may use EarthCARE level 2
data products on input as well (see also INT-10).
C
FUN-2
Level 2 modules shall implement the algorithm
specified in the corresponding ATBD.
C
FUN-3
Level 2 modules shall compute and store into
products: geophysical parameters, error
descriptors and product confidence data (i.e.,
quality flags).
FUN-4
Level 2 modules shall check whether all required
data are available before running.
FUN-5
Level 2 modules shall respond in a controlled
way to instrument degradation or failure or
unexpected values in the input data as per INT10.
FUN-6
Convergence criteria shall be established for
The
C
each algorithm. These criteria shall be monitored convergence
during the execution of the algorithm. Abortion
criterion is that
See associated
PDD
C
C
Meaningful
C
errors are
reported and
program
gracefully exits
if appropriate
in response to
missing or
corrupt data
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
11 of 16
criteria shall be established and applied for
deviation from accepted convergence profile.
Respective error information shall be produced.
the L2 norm of
the gradient of
the cost
function falls to
less than unity.
If this is not
reached in a
specified
number of
iterations, the
algorithm
aborts. Error
and
convergence
information is
reported
FUN-7
Level 2 modules shall provide a trace of their
execution in a single Log File for each execution
run.
C
FUN-8
The configuration files of a level 2 module shall See associated
allow to configure the logging level of the level 2 ICD
module.
C
FUN-9
The following logging levels shall be supported
for a level 2 module: Error, Warning, Info,
Debug, Progress.
See associated
ICD
C
FUN-10
An Error shall be logged when an unexpected
condition was detected and the
level 2 module cannot perform any error
recovery and must exit.
See associated
ICD
C
FUN-11
A Warning shall be logged when an unexpected
condition was detected but error recovery is
possible.
See associated
ICD
C
FUN-12
An Info shall be logged when a significant
condition occurred that directly or indirectly
affects the level 2 module outputs. An info shall
also be logged at module startup and module
termination.
See associated
ICD
C
FUN-13
Debug should be used as needed for the
development of the Level 2 module.
See associated
ICD
C
FUN-14
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
12 of 16
Progress shall be used to log the progress of the
simulation from 0% to 100%, exploiting the
processing steps of level 2 modules.
See associated
ICD
C
Interface Requirements
INT-1
The Unix operating system shall be used. The
Code developed C
actual hardware/operating system
under Linux
configuration shall be PC (multi-core/multiprocessor system if needed)/Linux. Details
(including the Linux distribution and its version)
shall be agreed with the agency.
INT-2
Coding shall be performed in a high-level
programming language such as C/C++
or Fortran-90. Details (including compiler
versions) shall be agreed with the Agency.
INT-3
Compilation options and compiler versions shall GNU compiler
be documented and justified.
collection is
needed
C
INT-4
Each level 2 module shall be able to run standalone.
C
INT-5
Besides being able to run stand-alone, Level 2
Not yet
modules shall be compatible with and run within
the EarthCARE end-to-end simulator ECSIM.
NY
INT-6
Input to and output from level 2 modules shall be
performed via files.
C
INT-7
In addition, level 2 modules shall use Unix
standard output for status messages.
INT-8
Level 2 modules shall not use Unix standard
input.
C
INT-9
Level 2 modules shall not produce graphical
output. Graphical output (e.g. for quicklook
results) may be produced from additional tools.
C
C (1990
C
standard), C++
(1998 standard)
and Fortran
(1977 and 1990
standards) are
required
Not yet,
NY
currently sends
some messages
to standard
error
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
13 of 16
INT-10
Level 2 modules shall read the following files on
input: 1. one EarthCARE level
1b data product (in case of synergistic
processors: one or more EarthCARE level 1b
data products from different instruments, and/or
one or more level 2 data products) covering
up to 2 orbits worth of data, 2. a configuration
file, 3. support files (e.g., meteorological data
from ECMWF, look-up tables, topographic
databases, etc.) as required.
C
INT-11
Level 2 modules shall write the following files on Currently no
output: 1. one level 2 data product, 2. a log file. log file is
written but this
will be fixed
NY
INT-12
All files (including configuration and
ntermediate input/output files) shall be in XML
or NetCDF format. The default format shall be
XML while NetCDF shall be used for data
products and large files only (i.e., where using
XML would significantly degrade performance).
Details (including the NetCDF version) shall be
agreed with the agency.
Output data is NY
in netcdf but
configuration is
via an ascii
format that is
much more
easily edited.
INT-13
The functions of the Earth Explorer mission CFI
software shall be used wherever possible.
What are these
?
N
INT-14
The reference systems defined in the Earth
Explorer mission CFI software shall
be used throughout, in particular for all outputs.
???
N
INT-15
All processing parameters including the
environment of level 2 modules (e.g.,
paths) shall be configurable in external
configuration files without code updates or
recompilation.
C
INT-16
Level 2 modules shall read their configuration
file(s) at run-time at start-up.
C
INT-17
Level 2 products shall follow the general
EarthCARE file format as specified in [AD1].
Not sure the
general
baseline
document
defines a
NY
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
14 of 16
format?
INT-18
Level 2 modules shall generate one level 2 output
data product per level 1 input data product.
C
INT-19
Each level 2 module, complete with reference
configuration and support files, shall be
delivered as a single tar file, optionally
compressed by gzip.
C
INT-20
No unused or otherwise unneeded files shall be
delivered with a level 2 module.
C
INT-21
Each level 2 module shall be delivered with a
script that plugs it into the ECSIM framework.
NY
INT-22
Only relative path names shall be used in the
installation scripts of Level 2 modules.
C
INT-23
The tar file used to deliver a level 2 module shall
conform to the naming convention described in
Section 6.7 of the General Requirements
Baseline Document [AD?]
NY
Testing Requirements
TST-1
Test entry points shall be defined to facilitate
testing and debugging. They shall allow running
selected parts of the processing chain by
providing breakpoints for reading intermediate
input data and writing intermediate output data.
TST-2
ECSIM shall be used for end-to-end performance
testing using appropriate ECSIM models
(including scenes, forward models, instrument
and L1b simulators). Furthermore, realistic
scenes shall be used, derived from existing
measurements.
The ACM-CAP
processor ins
indivisible
N
NA
Configuration management requirements
CFG-1
All elements and sub-elements composing the
level 2 processor shall be put under
configuration management.
CFG-2
At least the following configuration items shall
be managed under configuration
management:
What is
configuration
management?
N
N






Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
15 of 16
Software modules (source code and
object code)
Static configuration and support data
Acceptance Test Data Sets
Procedures and scripts
Any static data, e.g., delivered tar files
Software and hardware environment
documentation
CFG-3
Widely supported Software Configuration
Management tools shall be used.
N
CFG-4
It shall be possible to know, for any discrete
point in time, the software
configuration of the level 2 processor.
?
CFG-5
It shall be possible to retrieve, for any discrete
point in time, the level 2 processor
in this configuration.
?
CFG-6
To maintain software integrity throughout the life
cycle, a proper and systematic
control of software changes shall be provided.
NY
CFG-7
For all versions of any software module, the
modifications with respect to the
previous version shall clearly be identified and
documented in the software module itself.
NY
CFG-8
All versions of all software modules shall be
archived with a version identifier
and the time from which the software module was
in use.
NY
CFG-9
For each delivery of any Configuration Item, a
Change Log detailing all changes
since the previous delivery shall be provided.
NY
CFG-10
Each version of an Level 2 module shall be
uniquely identified through an
Overall Version Number in the following format:
VERSION = Major Version.Minor Version, each
over two digits
C
Coding Rules
COD-1
Common code should be separately packaged
and reused within a single level 2
C
Code
L2b-ACMCAP-SRS
ACM-CAP SRS
Issue
01
VARSY Project
Date
27/08/2012
Page
16 of 16
module as well as across level 2 modules.
Code duplication should be eschewed as far as
possible.
COD-2
Code related to the interfaces (input/output) shall
be separated (in different
subroutines and different source code files) from
code implementing the algorithm.
C
COD-3
File or path names shall not be used inside the
code.
C
COD-4
Redundant calculations in Level 2 modules
should be avoided.
C
COD-5
It shall be shown that code does not include any
uninitialised variables, unused variables,
memory leaks, and dead (i.e., unused) code.
All issues can
be verified
rigorously
using various
compiler
options, except
for memory
leaks. The
absence of any
potential leaks
would be
difficult to
rigorously
demonstrate
P
COD-6
Standard headers shall be used for subroutines.
Details shall be agreed with the agency.
Define
“standard
header”
?
COD-7
Ample use of comments shall be made.
C
COD-8
Comments shall be used to describe the structure
of the code.
C
*C=compliant, P=partly non-compliant, N=not compliant, NY=not yet compliant but will be,
NA=Not applicable