Euclid
Consortium
Interaction between SWG and SGS-OUs
Fabio Pasian & Marc Sauvage
(EC SGS Project Office)
The presented document is Proprietary information of the Euclid Consortium. This document shall be used and disclosed by the receiving Party and its related entities (e.g.
contractors and subcontractors) only for the purposes of fulfilling the receiving Party's responsibilities under the Euclid Project and that identified and marked technical data shall
not be disclosed or retransferred to any other entity without prior written permission of the document preparer .
OU-LE3/GC-SWG
22.03.12
Rationale
Euclid
Consortium
The SWG and the SGS will constantly interact throughout the
mission phases.
During the development of the SGS, the SWGs set
requirements on the algorithmic objectives and defines the
validation testing of the pipeline module.
During operations, the SGS systems produce the data
products that form the basis of the SWG work.
The science objectives of Euclid are quite ambitious, and
require a complex data processing system.
The interface between the SGS and the SWGs must be
clearly defined to avoid biasing the SGS products, or
overlap and/or duplication of work between the SGS and
the SWGs.
OU-LE3/GC-SWG
22.03.12
Documentation flow-down
Science
Management
Plan
(SMP)
Science
Operations
Concept
(SOCD)
input from
EC SWGs
Science
Implementation
Requirements
(SIRD)
SOC Science
Implementation
Plan
(SOC SIP)
EC Science
Implementation
Plan
(EC SIP)
EC SGS
Documentation
Tree
OU-LE3/GC-SWG
22.03.12
Data
Processing
Requirements
(GDPRD)
EC SGS WP
Breakdown and
Description
(WPBD)
Euclid
Consortium
Science
Requirements
Document
(SciRD)
Mission
Requirements
Document
(MRD)
Payload
Elements
Requirements
(PERD)
Legacy Data
Processing
Requirements
(LDPR)
Science
Analysis
Implementation
(SAID)
EC SGS
SWGs-SGS
Interface
Control Doc
EC
SWGs
Requirements flow-down
Science
Management
Plan
(SMP)
Science
Operations
Concept
(SOCD)
Science
Requirements
Document
(SciRD)
Science
Implementation
Requirements
(SIRD)
SOC Science
Implementation
Plan
(SOC SIP)
EC Science
Implementation
Plan
(EC SIP)
EC SGS
Documentation
Tree
OU-LE3/GC-SWG
22.03.12
Euclid
Consortium
Data
Processing
Requirements
(GDPRD)
EC SGS WP
Breakdown and
Description
(WPBD)
Mission
Requirements
Document
(MRD)
Payload
Elements
Requirements
(PERD)
Legacy Data
Processing
Requirements
(LDPR)
Science
Analysis
Implementation
(SAID)
SWGs-SGS
Interface
Control Doc
Euclid
SWG Development
OU Development
Consortium
SUR phase - Definition of the science user requirements
CV phase – Concept validation
TR phase - Transfer of the software to operations
Science Inputs
SUR
Proof of concept
Method
Publication
CV
OU Inputs
GDPRD
LDPRD
Next Cycle
Software System
TR
SPMP
SCMP
SVVP/Reviews,
PO
Standard
Document
UR
OM
URD
SVVP/ST Plan/IT Plan
SR/AD
STD
AT plan
SSD
SVVP/ST spec
DD
SDC Development
UR phase - Definition of the user requirements
SR phase - Definition of the software requirements
AD phase - Definition of the architectural design
DD phase - Detailed design and production of the code
OM phase - Operations and maintenance
OU-LE3/GC-SWG
22.03.12
from the EC SIP 2.2 draft
CODE
SUM
SPMP - Software Project Management Plan
SCMP - Software configuration management procedures
SVVP - Software Verification and Validation Plan
AT Plan - Acceptance Test
ST Plan - System Test Plan
IT Plan - Integration Test Plan
URD - User Requirements Document
DPRD - Data Processing Requirements Document
SSD
- Software Specification Document
SUM - Software User Manual
STD
- Software Transfer Document
SWGs-SGS ICD
Euclid
Consortium
Science
Management
Plan
(SMP)
Science
Operations
Concept
(SOCD)
Science
Requirements
Document
(SciRD)
Science
Implementation
Requirements
(SIRD)
SOC Science
Implementation
Plan
(SOC SIP)
EC Science
Implementation
Plan
(EC SIP)
EC SGS
Documentation
Tree
OU-LE3/GC-SWG
22.03.12
Data
Processing
Requirements
(GDPRD)
EC SGS WP
Breakdown and
Description
(WPBD)
Mission
Requirements
Document
(MRD)
Payload
Elements
Requirements
(PERD)
Legacy Data
Processing
Requirements
(LDPR)
Science
Analysis
Implementation
(SAID)
SWGs-SGS
Interface
Control Doc
The SWGs-SGS ICD
The ICD originates from
discussions between OU-LE3
and the SWGs.
Draft 0.5, distributed to the
OU+SDC leads, and SWG
coords was iterated inside the
SGS PO.
Comments are expected and
welcome.
OU-LE3/GC-SWG
22.03.12
It's short (only 7 pages of meaningful text)!
Euclid
Consortium
The interface entities
Euclid
Consortium
The ground segment Organization Group:
The SGS PO (manager, scientist, system team lead + support team).
The OU leads.
The SDC managers.
The Science Working Groups coordinators
The leads of the two primary cosmology probes SWGs (4 persons).
2 rotating representatives for leads of the legacy, theory and simulation
SWGs.
These two representative bodies are in charge of taking care of
the interface issues. See the EC Management Plan for names
(wiki).
OU-LE3/GC-SWG
22.03.12
EC SGS Management
OU-LE3/GC-SWG
22.03.12
Euclid
Consortium
A key concept: avoid confirmation bias
Euclid
Consortium
How? try to maintain cosmology "independence" and/or "blindness"
in the SGS pipeline.
SGS codes should refrain from relying on cosmology assumptions
("independence"), and their validation should not be done against
a given cosmology ("blindness").
Requests to depart from that, (e.g. use or produce absolute
quantities) should be made known to the Organization Group, to
be discussed with the SWG coords, and use the reference
cosmology maintained by the SWG coords.
This is/will be a highly debated item in discussions between the
SGS and the SWG.
Is it enough to avoid a confirmation bias? likely not.
OU-LE3/GC-SWG
22.03.12
Requirement driven development
Euclid
Consortium
General principle for SGS data processing pipeline
development:
SWG formulates requirement for the production of data with
specified properties, as well as tests that have to be run to
validate that part of the SGS pipeline.
OU researches algorithms that fulfill the requirement, and
once the SWG has accepted that the algorithm is an
acceptable candidate, passes to the SDC-Dev the algorithm
description and a test "plan" for verification and validation.
OU algorithms suggestions can only be refused by SWG
because they do not meet the requirement.
Each SGS WP should be connected, directly or through other
WPs to a requirement on the SGS.
"Free-floating" SGS WP could be a worry and should be
tracked down.
However. WPs are management tools.
Management WPs, or finer-grained
subdivision of work (e.g. across countries)
OU-LE3/GC-SWG
are a matter of SGS decision.
22.03.12
Code practice
Euclid
Consortium
No coding standards at OU level
OUs are not required to produce code at all (algorithms,
prototypes + test data)
Coding standards/uniformity required at SDC level
codes need to be scalable, maintainable, moveable.
codes will adhere to some form of GPL, but intellectual
"property" should be maintained by applying to the codes
rules similar to publications (policy to be set).
Request by SWG to use the SDC to run their codes should be
handled by the SGS organization group.
The primary mission of the SDCs is to develop and run the
SGS pipeline.
SWG SAID is a good read for SGS to anticipate these
demands.
OU-LE3/GC-SWG
22.03.12
Data access
Euclid
Consortium
Question: is it always harmless to use Euclid data to develop
SGS modules?
Data access should stay compatible with the primary missions
of the SGS.
Some issues remain to be cleared out, e.g. regarding simulated data produced by the SGS.
OU-LE3/GC-SWG
22.03.12
Conclusions
Euclid
Consortium
Currently, the SWGs-SGS ICD is a living document. Feedback is
needed.
SWGs-SGS interrelations are important, key to the success of the
mission.
ICDs are built to define the interfaces between groups.
Interrelations ≠ Interfaces
IMPORTANT: let’s not build useless/harmful fences
between (virtual) groups!
OU-LE3/GC-SWG
22.03.12
Euclid
Consortium
OU-LE3/GC-SWG
22.03.12