SoW - DSI - Emits
advertisement
ESA UNCLASSIFIED – For Official Use esrin Via Galileo Galilei Casella Postale 64 00044 Frascati Italy T +39 06 9418 01 F +39 06 9418 0280 www.esa.int Statement of Work for the Data Consolidation and Bulk Processing Service Initiative Prepared by Reference Issue Revision Date of Issue Status Document Type Distribution EOP-GTO EOEP-GSOP-EOPG-SW-12-0003 1 12-j 19/11/2012 SOW ESA UNCLASSIFIED – For Official Use Title Issue 1 Revision 12-j Author Date 19/11/2012 Approved by Date Reason for change Issue Issue 1 Reason for change Revision 12-j Date Pages Page 2/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j Revision Date Paragraph(s) ESA UNCLASSIFIED – For Official Use Table of contents: 1 INTRODUCTION ............................................................................................................................. 6 1.1 Definitions and abbreviations ........................................................................................................................................... 6 1.2 Related documents ............................................................................................................................................................. 6 1.2.1 Applicable Documents ..................................................................................................................................................... 6 1.2.2 Reference Documents ...................................................................................................................................................... 7 2 BACKGROUND, APPROACH, OBJECTIVES AND SCOPE ................................................................. 7 2.1 Background and approach ................................................................................................................................................. 7 2.2 Objectives ...........................................................................................................................................................................8 2.3 Scope of the Service............................................................................................................................................................ 9 3 CORE SERVICES ........................................................................................................................... 10 3.1 Data collection and analysis ............................................................................................................................................ 10 3.1.1 Description ..................................................................................................................................................................... 10 3.1.2 Scope ...............................................................................................................................................................................11 3.1.3 Activities ..........................................................................................................................................................................11 3.1.4 Inputs ..............................................................................................................................................................................11 3.1.5 Outputs ........................................................................................................................................................................... 12 3.1.6 Requirements ................................................................................................................................................................. 12 3.2 Data consolidation ...........................................................................................................................................................23 3.2.1 Description .....................................................................................................................................................................23 3.2.2 Scope ..............................................................................................................................................................................24 3.2.3 Activities ......................................................................................................................................................................... 25 3.2.4 Inputs ............................................................................................................................................................................. 25 3.2.5 Outputs ...........................................................................................................................................................................26 3.2.6 Requirements .................................................................................................................................................................26 3.3 Integration of Processing Systems ................................................................................................................................. 38 3.3.1 Description .................................................................................................................................................................... 38 3.3.2 Scope ............................................................................................................................................................................. 40 3.3.3 Activities ........................................................................................................................................................................ 40 3.3.4 Inputs ............................................................................................................................................................................. 41 3.3.5 Outputs ...........................................................................................................................................................................42 3.3.6 Requirements .................................................................................................................................................................42 3.4 Bulk processing / reprocessing/ reformatting operations ............................................................................................ 48 3.4.1 Description .................................................................................................................................................................... 48 3.4.2 Scope ............................................................................................................................................................................. 50 3.4.3 Activities ........................................................................................................................................................................ 50 3.4.4 Inputs ............................................................................................................................................................................. 51 3.4.5 Outputs ........................................................................................................................................................................... 51 3.4.6 Requirements ................................................................................................................................................................. 51 4 SUPPORT SERVICES .................................................................................................................... 59 4.1 Managerial and technical approach ................................................................................................................................ 59 4.1.1 Overall responsibility ..................................................................................................................................................... 59 4.1.2 Customer and User orientation ..................................................................................................................................... 59 4.1.3 Risk contention .............................................................................................................................................................. 59 4.1.4 Cooperation ................................................................................................................................................................... 60 4.1.5 Autonomy and pro-activeness ...................................................................................................................................... 60 4.1.6 Skills, know-how and experience required .................................................................................................................. 60 4.2 Data Repatriation/delivery .............................................................................................................................................. 61 4.2.1 Description ..................................................................................................................................................................... 61 4.2.2 Scope .............................................................................................................................................................................. 61 4.2.3 Activities .........................................................................................................................................................................62 Page 3/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 4.2.4 Inputs .............................................................................................................................................................................63 4.2.5 Outputs ...........................................................................................................................................................................63 4.2.6 Requirements .................................................................................................................................................................63 4.3 Data Information and Configuration Management........................................................................................................69 4.3.1 Description .....................................................................................................................................................................69 4.3.2 Scope ..............................................................................................................................................................................70 4.3.3 Activities .........................................................................................................................................................................70 4.3.4 Inputs .............................................................................................................................................................................70 4.3.5 Outputs ........................................................................................................................................................................... 71 4.3.6 Requirements ................................................................................................................................................................. 71 4.4 Service and Project Management .................................................................................................................................... 79 4.4.1 Operating model ............................................................................................................................................................ 79 4.4.2 Use of documented procedures .................................................................................................................................... 80 4.4.3 Support Processes (Availability, Performance and Capacity Management) ............................................................... 81 4.4.4 Service Continuity Management ...................................................................................................................................85 4.4.5 Security Management ................................................................................................................................................... 86 4.4.6 Asset Management......................................................................................................................................................... 87 4.4.7 Issue Management ......................................................................................................................................................... 87 4.4.8 Service Infrastructure Configuration and Change Management .................................................................................99 4.4.9 Release management ................................................................................................................................................... 101 4.4.10 Service Level Management .......................................................................................................................................... 106 4.4.11 Performance Management Measurement and Reporting ..........................................................................................115 4.4.12 Communication Management ..................................................................................................................................... 119 4.4.13 Interfaces to Third Parties ........................................................................................................................................... 121 4.4.14 Escalation procedure ................................................................................................................................................... 123 4.4.15 Tools ............................................................................................................................................................................. 124 4.4.16 Project Management .................................................................................................................................................... 125 4.5 Continual Improvement ................................................................................................................................................ 127 4.6 Risk Management .......................................................................................................................................................... 129 4.7 Quality management ...................................................................................................................................................... 130 4.8 Additional Services......................................................................................................................................................... 130 4.8.1 Non-standard activities ............................................................................................................................................... 130 5 GOVERNANCE ............................................................................................................................. 131 5.1 Customer and Demand Management ........................................................................................................................... 132 5.2 Roles and Responsibilities ............................................................................................................................................. 133 5.3 Meetings and Deliverables ............................................................................................................................................. 134 5.3.1 Meetings ....................................................................................................................................................................... 134 5.3.2 List of deliverables ....................................................................................................................................................... 134 5.3.3 Use of deliverables and CFIs ....................................................................................................................................... 135 5.3.4 Review and acceptance procedures of the deliverables ............................................................................................. 135 5.4 Yearly Service Review .................................................................................................................................................... 137 6 FINANCIAL MANAGEMENT ........................................................................................................ 137 7 TRANSITION MANAGEMENT ...................................................................................................... 139 7.1 Phase-in .......................................................................................................................................................................... 139 7.2 Phase-out ........................................................................................................................................................................ 141 8 ESA UNDERTAKINGS AND CFI’S ................................................................................................. 143 9 ANNEXES ....................................................................................................................................144 Annex 1 Definitions and Abbreviations .................................................................................................................................. 144 Definitions ............................................................................................................................................................................... 144 Abbreviations ........................................................................................................................................................................... 148 Annex 2 Initial basket of Projects ........................................................................................................................................... 149 Annex 3 Quality Matrix ............................................................................................................................................................151 Annex 4 Roles and Responsibilities .........................................................................................................................................151 Page 4/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use ESA and related players .......................................................................................................................................................... 152 Contractor Players ................................................................................................................................................................... 158 Overview of the principal interfaces ....................................................................................................................................... 161 Annex 5 Deliverable Item List (DIL) ...................................................................................................................................... 162 Annex M – Cost Model ............................................................................................................................................................ 162 Annex Q Quality and documentation management ............................................................................................................... 164 Annex S Security Management ............................................................................................................................................... 166 Page 5/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 1 INTRODUCTION Within the Directorate of Earth Observation Programmes (D/EOP) of the European Space Agency – ESA (http://www.esa.int) - the Ground Segment and Mission Operations Department (EOP-G) includes among its responsibilities the management of various activities concerning EO data, with the objective of responding to the data needs of ESA’s EO Data Users. Some of these activities are performed systematically upon acquisition of the data. Others are performed if and when considered necessary in a later time frame. These include - data collection from various media, - data consolidation, - integration of processing systems into the operational environment; - data bulk-processing/reprocessing/format conversion (referred to generically as processing) This document is the Statement of Work (SoW) for the procurement as a service of these activities as well as the related management and control activities. The full document is to be considered as requirements, and not only the requirements which for ease of reference have been numbered. Note: the roles and responsibilities referred to in the document are described in Annex 4 1.1 Definitions and abbreviations See Annex 1 [Definitions and Abbreviations] 1.2 Related documents 1.2.1 Applicable Documents Applicable documents are considered to be part of this SoW and the proposals are expected to be compliant with the Applicable Documents. [ADS 1] GMGT-SENE-EOPG-RS-09-0002 Network and ICT Security requirements for the EO PDGS v1.1 [ADS 2] ESA Security Directives - Non Disclosure Agreement [ADS 3] GMGT-SENE-EOPG-PD-10-0004 ESA_EOP-G-NW Policy Implementation-v1.2 Page 6/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 1.2.2 Reference Documents Reference documents, though not formally part of this document, amplify or clarify its contents. The documents listed in the table of Annex 2 describing the processors and the datasets are reference documents to this SoW. [RDS 1] GMGT-SENE-EOPG-TN-10-0006 EO PDGS data categorization guidelines and security rules v.1.0 [RDS 2] GMGT-SENE-EOPG-TN-10-0007 EO PDGS User Classification v.1.0 [RDS 3] Overall ESA Security Directives - Admin-ipol-2008-006e [RDS 4] GMGT-SENE-EOPG-PD-12-0011 EOP-G Security Incident Handling Policy [RD PDSC]: Long Term Data Preservation – Earth Science Preserved Data Set Content, v4.0 Ref. LTDP-GSEGEOPG-RD-11-0003 [RD NAMING] Products standard naming convention PGSI-GSEG-EOPG-TN-06-0001 v2.0 [RD EOCODES] EO Parameters Codes PGSI-GSEG-EOPG-TN-07-0001 [RD STATION CODES] IODI-GSEV-EOAD-TN-02-0001 ODISSEO Station Code Reference List [RD IDEASICD] IDEAS – Landsat IDEAS ICD, ref. IDEAS-SER-MGT-SPE-1063 – This document although specific to one mission is representative of the interface with the IDEAS service. [RD EO-INV] STFC-RAL-LTDP-6.2 EO Inventory Data Model Technical Note v 3.7 of 21/06/2012 [RD IPF-ICD] Generic IPF Guidelines Issue 1 revision 8, MMFI-GSEG-EOPG-TN-07-0003 2 BACKGROUND, APPROACH, OBJECTIVES AND SCOPE 2.1 Background and approach The demand for the activities mentioned above has been increasing steadily, user needs for reprocessed and bulk-processed data have increased significantly and so has the volume of data to be handled in these activities. At the same time, there is a trend in the evolution of IT technology infrastructure that enables the deployment of computing power and storage on-demand at continuously decreasing costs. These trends provide an opportunity to improve the efficiency the operating model and optimize usage of funds for the “on-request” activities through : - Increased standardisation of the related processes and interfaces; Page 7/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - Increased exploitation of the most cost-effective combination of commercially available infrastructure; Leverage of competition to increase value for money; Economies of scale through the grouping of similar, standardised activities into a single contractual framework. This has led ESA to adopt for the scope of this SoW a service approach, where the Contractor executes the activities in scope under agreed levels of quality and performance. Not only the performance of the core activities, but also the selection, deployment and management of the resources, means and competences, as well as the technical architecture and implementation, are fully the responsibility of the Contractor, who is expected to put forward and uphold the most effective technical and managerial approach to meet the requirements and, beyond, achieve the satisfaction of ESA and of the data users. ESA will not impose or provide any technological platform or technical support and will limit the CustomerFurnished Items to the processing systems concerned (in the broad sense, including any processing system required eg prototype/IPF, format converter, QC tool etc) and the data itself (including the auxiliary data). The major competences that the Contractor is requested to deploy over the duration of the Contract are: a) Understanding of Earth Observation data and related knowledge necessary to perform the services in scope, particularly data consolidation and bulk-processing /reprocessing / format conversion; b) Efficient utilization of Information Technology, to deploy and maintain a cost-effective infrastructure able to store and process the data at short notice and with the necessary performance. The Contractor is also required to deploy and uphold a strong ability to manage service processes and coordinate the contributions of the various units under its management in order to fulfil its overall responsibility. 2.2 Objectives The overarching objective of this initiative is to achieve the satisfaction of ESA and of the data users. More specifically, ESA has 3 major families of objectives: Strategic objectives • Optimal responsiveness to user needs • Focus on data and user needs • Outsourcing of infrastructure & technology and a Service approach • Synergies among activities • Overall coherence with other data management activities Operational objectives • High throughput / turn-around in activities • Capacity adaptable to increasing or changing demand Page 8/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use • • • Increased process standardisation Simple contractual scheme High level of delegated responsibility Economic objectives • Cost efficiency • Cost model in line with service approach Only if the Contractor understands and shares these objectives will the initiative be successful and the scope of the business have a chance to extend to further projects. 2.3 Scope of the Service The scope of this SoW is defined by: - the missions and corresponding data undergoing the activities: o ESA missions (ERS, ENVISAT, Earth Explorers); o ESA Third Party Missions (eg Landsat,, etc.) o Historical, present and future missions are covered, however future ESA missions are limited to Earth Explorers (Sentinels are not in the scope of the present procurement); - the activities: o so-called coreactivities, executed as projects (including a period of guarantee and follow-up as indicated in the task descriptions) . The types of projects covered in this procurement include: Data Collection: reception of media from ESA, uploading of the data into the Contractor’s storage, assessment of the contents and production of an inventory, entry into the internal CCM system, preparation of the data for subsequent activities of consolidation and processing; Data Consolidation: different stages covering data cleansing (removal of corrupted data and exact duplicates, merging data of same instrument from different sources into a single dataset and assessment of data completeness); Processing system Integration: where the processing system is installed, integrated and tested in the operational environment, ready for operational processing; Reprocessing / Bulk processing / Format Conversion (collectively referred to as Processing): execution of the processing itself. o Service support activities, which are: Data Repatriation / delivery: to deliver data to ESA or other centres (PACs or future LTDP centres) upon request (dissemination to end-users is not in the scope of this SoW) and to provide on-line access to data for specific purposes; Data Information and Configuration Management: to collect, generate and manage the information necessary for the efficient performance of the core services; Project and Service Process management and other supporting services. o Governance and financial management; o Transition management, to drive the phase-in and phase-out periods of the contract. Page 9/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The core activities (projects) in this SoW are performed upon explicit request of an internal Customer. The four types of projects are not always required for a given dataset (for example, consolidation may or may not be required, processing may or may not be required …). A project of data collection is always the starting point. A planned scope (referred to as the Initial Basket and shown in Annex 2) is known at the beginning of the Contract, however the actual scope (with projects removed from and/or added to the Initial Basket) will be defined in the course of the contract. For planning purposes, the Contractor will have early visibility of what projects are intended (even though they may later not be confirmed) and the evolution of the confirmation status. This contract will be in principle the chosen approach for the core activities highlighted above. This will however not be systematic, depending on: - the Contractor’s performance and quality of outputs; - the knowledge (esp. on the data) required for a given activity vs available in the Contractor’s team; - the comparative costs of the Service vs. available alternatives; - external factors (notably constraints linked to Intellectual Property Rights on the processors). The Service support activities are continuous activities available to support the Core activities (projects). Out of default scope of this procurement are activities of: - Scientific data validation of processed data; - EO data processor and quality control tool development; - Higher level products generation (>L2); - On-demand (not systematic) production of L1, L2 products; - Dissemination of processed data to end users; - Long-term Archival/Preservation; - Handling of Sentinel missions data. 3 CORE SERVICES 3.1 Data collection and analysis 3.1.1 Description This WP consists in collecting and preparing the data for the main activities (consolidation / processing) by collecting the specified data, taking a working copy into the Service infrastructure storage, analysing it and storing the data-related information into the data CCM DB and Data Information System in a way that further WP’s can be activated on this data or that the data can be delivered back to ESA. The collection of data may be limited to receiving data on HDD as CFI from ESA via the ESA Project Manager (which will be the most common approach in particular for the first projects to be handled), or it can involve fetching data from an on-line source or receiving and transcribing media. In some cases the environmentfriendly destruction of the original media is also requested. Page 10/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use In addition to the HDD, a wide range of media is potentially covered by this WP, including but not restricted to: - Exabyte including 8200 and 8500 - Optical disks - DLT - LTO - Etc. Note: in the case of physical media, the media as input to the activity will normally be shipped to the Contractor. Note: ESA will deliver the input data with any existing information and reports on completeness and consistency, which may be available. 3.1.2 Scope The scope of the WP will be indicated project by project. The data concerned in each project may represent various combinations of: - data of given missions / instruments - data currently stored in a certain media / format - data currently stored in a given location - Etc. The scope of the WP always includes data in the broad sense, including primary, auxiliary and ancillary data as well as mission documentation. The mission documentation concerned is however limited to the documentation necessary to carry out the activities of consolidation / processing. Data consolidation is not in the scope of this WP and is the subject of a separate WP. 3.1.3 Activities The activities include: - Receive and classify various media from ESA or a Third Party, or collect the data from an on-line repository - Transcribe data from various media to the Service repository, analyse the data and populate the CM DB and Data Information System (see ch 4.3) - Destroy / archive / ship back the original media Note: depending on the situation (esp. the source of the data) the activities included in each specific project may vary slightly.The transcription activity will always be included in each project. 3.1.4 Inputs For activity “Receive and classify media” Page 11/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use o o Media shipped / delivered to the Service Accompanying inventory (if available) of the media and their contents For activity “Transcribe data from various source media to the Service storage” o Media labelled and stored safely o Inventory of the received package o Updated CCM data base For activity “Destroy / archive / ship back the original media” o Media shipped / delivered to the specified destination o Instructions from the ESA Project Manager 3.1.5 Outputs For activity “Receive and classify media” o Acknowledgement of receipt of the package o Media labelled and stored safely o Inventory of the received package o Updated Data Information System and CCM data base (with newly received media included) For activity “Transcribe data from various source media to the Service storage” o Contractor’s service storage containing the data transcribed o Transcription report o Updated CCM data base and Data Information System (to reflect the transcription status) o Upon request, HDD containing a duplicate of the data and report on the duplication For activity Destroy / archive / ship back the original media o For media specifically requested by the ESA Project Manager to be destroyed, confirmation of the environmental-friendly destruction; o For media to be archived: media safely stored in the Contractor’s repository; o For media to be shipped back: original media shipped to the specified destination and acknowledgement of receipt of the media by the destination o In all cases: updated CCM Data Base and Data Information System Any ancillary tools used / developed by the Contractor during the course of the Collection exercise to read / convert / transcribe / analyse / inventory the data etc. Verification documentation 3.1.6 Requirements Note: the requirements are listed in sequence, reflecting the natural succession of the activities. The Contractor is however encouraged whenever practicable to perform the activities not in a strict waterfall but in a “continuous cycle” approach in order to reduce the project duration. Page 12/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 3.1.6.1 Receive and classify media This activity consists in receiving (and acknowledging receipt of) a delivery or shipment of media containing EO data (in the broad sense), including all types of media (paper, tapes, HDD, optical disks, NAS, etc) and types of data ( primary, auxiliary, ancillary, documents), and complementing the collection of missionrelated data from the various sources available. The activity also includes the identification and labelling of the individual media items received and their presumed contents, the update of the Contractor’s CCM system , and the preparation and delivery to ESA of an inventory of the package received. 3.1.6.1.1 Reception of the delivery [Req COLL-REC-10] Upon notification by the ESA Project Manager, the Contractor shall liaise with the party responsible for the shipment / delivery and if applicable with the Courier in order to facilitate the tracking of the shipment and the delivery. [Req COLL-REC-20] Upon delivery of the package, the Contractor shall perform an initial visual inspection of the contents and acknowledge receipt of the package within 1 NWD to the party doing the shipment and to the ESA Project Manager , flagging any evident anomaly with the package. 3.1.6.1.2 Configuration Management and Inventory Generation of the physical items [Req COLL-REC-30] The Contractor shall place each of the physical items of the package under Configuration Management, in compliance with the Data CCM Plan (see Ch. 4.3), including the assignment to each of a unique Configuration Item Id. [Req COLL-REC-40] The Contractor shall affix a label on each item with the Configuration Item id. [Req COLL-REC-50] In case any items already carry a label, the Contractor shall leave the existing label in place in addition to affixing its own. [Req COLL-REC-51] The Contractor shall capture the contents of the original label in the Data Information System and CCM DB. [Req COLL-REC-60] The Contractor shall record in the Data Information System and CCM DB at least the following information regarding each item: - shipment id, shipment origin and delivery date - Configuration Item id assigned to the item, - nature, - type, - technology, - capacity, - status as revealed by visual inspection, Page 13/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - presumed contents, reading of any label which may already be on the item at the time of reception for documents, the reference, document reference, mission, document format (paper or electronic, and if electronic source system e.g. MS-Word 97), document title, author and version if available any information regarding the item included in the inventory prepared by the party doing the shipment (if there is such an inventory) any known information on the contents of the item. [Req COLL-REC-70] Pending further action as described below, the Contractor shall store the items received in an appropriate storage area, compliant with the Security requirements of this SoW. [Req COLL-REC-80] The Contractor shall extract from the Data Information System and CCM DB an Inventory of the items received, showing all the items received in the package and for each at least the information listed above. The package received will in principle contain an inventory prepared by the party doing the shipment. This however may not be guaranteed in all cases. [Req COLL-REC-90] In case the package received is accompanied by an inventory prepared by the party doing the shipment, the Contractor shall: - annex the original inventory to its own Inventory and store it in its Data Information System and CCM DB; - compare the expected contents of the package with the effective contents of the package; - record any differences in its own inventory. [Req COLL-REC-100] In case the package received is accompanied by an inventory prepared by the party doing the shipment and there are any differences between the original inventory and the effective contents of the package, the Contractor shall liaise with the party doing the shipment to clarify and rectify the differences, and shall report the progress at least once per week to the ESA Project Manager. [Req COLL-REC-110] The inventory prepared by the Contractor shall be delivered as part of the overall Data Collection Report. However, the Contractor shall send the Inventory in advance in electronic form to the ESA Project Manager. [Req COLL-REC-120] The Contractor shall complete the registration of the items received into the Data Information System and CCM DB, the storage into an appropriate area, and the delivery to ESA of the Inventory within the time not exceeding the threshold indicated in the SLA. 3.1.6.2 Transcribe data from various source media to the Service storage This activity consists in the transcription of data from various source media to the storage used by the Contractor for the purpose of the Service. The activity also includes the recovery of failures that may occur during the transcription and the delivery of a transcription report as part of the Data Collection Report. The transcription report is an important management tool which will in particular support ESA’s decision to destroy or not the source media. Page 14/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use In some cases, it may in addition be required to make a copy of the transcribed data to a HDD and ship the HDD, accompanied with an inventory of the data contained, to a destination specified by the ESA Project Manager. Note: the physical format of the data (which is linked to the media support) will obviously be affected by the transcription exercise. The logical format (internal structure of the files) is not affected except in case of explicit requirements for format conversion which may be put forward on a case by case basis. The logical structure of the data (directory tree) is driven by the overall organisation of the data in the Contractor’s storage. 3.1.6.2.1 Collection of additional mission documentation The collection of mission data and documentation goes beyond the material actually shipped to the Contractor, and includes the pro-active gathering of additional data and documents necessary to perform any intended activities of consolidation and reprocessing. Additional material may need to be collected from: - ESA and centres managed by ESA, taking advantage of the data and document repository already accumulated and managed by the Data Librarian - sources indicated by the Data Librarian - sources indicated by the Contractor itself and agreed with the ESA Project Manager. [Req COLL-ADD-10] The Contractor shall identify any additional material (data and documentation), which may be needed for the intended activities to be performed on the data being collected (eg consolidation, reprocessing). [Req COLL-ADD-20] The Contractor shall collect any additional material identified, which is available directly from the ESA Project Manager. [Req COLL-ADD-30] The Contractor shall liaise with the sources indicated by the ESA Project Manager and collect the additional material identified which is available from these sources. [Req COLL-ADD-40] The Contractor shall pro-actively search for and propose additional sources of additional material identified related to the data being collected. The ESA Project Manager will confirm the use of the additional material and the approach by the Contractor of the corresponding sources. [Req COLL-ADD-50] The Contractor shall liaise with the ESA Project Manager on the best way to approach the proposed additional sources, and liaise accordingly with these sources to collect any additional material identified. Note: the ESA Project Manager and other ESA roles as needed will interface with the sources in question at organisational level, while the Contractor will interface at operational level. Page 15/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 3.1.6.2.2 Data transcription [Req COLL-TRA-10] The Contractor shall transcribe the data from the input media to the storage infrastructure of the Service. [Req COLL-TRA-20] The physical location and organisation of the data in the storage infrastructure of the Service shall be compliant to the prescriptions of the Service CCM plan as prepared by the Contractor and accepted by the ESA Technical Officer. [Req COLL-TRA-30] The Contractor shall append to the original file name a unique signature of the file. Upon proposal of the Contractor, this requirement may be waived by the Project Manager in consultation with the Data Librarian. Note: the unique signature to be used should be the MD5 checksum. In principle, the contents of the media provided by ESA will be homogeneous (eg data of the same mission / instrument) and will match the indications given by an inventory delivered with the media. However this cannot be guaranteed in all cases. In all circumstances, the transcription process must collect the full contents of the original media in order to avoid any loss of data, eg in the case that some tapes cannot withstand more than one additional use. [Req COLL-TRA-40] The Contractor shall make sure that the transcription covers all the data contained on the media regardless of: mission, instrument, type of data , etc correspondence or not with any inventory information which may have been delivered together with the media. [Req COLL-TRA-50] In addition to the label affixed in the course of the previous activity (“Receive and classify media”), the Contractor shall affix a transcription label to each source media item with the following information: - date of transcription; - indication of fully successful transcription or not; - reference of the transcription report. Paper documents are treated as media items for which all the same requirements apply, with in addition the scanning of the documents into electronic form: [Req COLL-TRA-60] The Contractor should scan the paper documents into PDF/A format. Note: scanning to PDF is not required. It is a nice feature to have for a small number of documents to be scanned occasionally. It is not intended to cover large amounts of documents. Note: PDF/A is defined in ISO 19005-1:2005, ISO Standard published on October 1, 2005: Document Management – Electronic document file format for long term preservation – Part 1: Use of PDF 1.4 (PDF/A1). This is specifically aimed to “digital preservation of electronic documents”. PDF/A differs from PDF by omitting features ill-suited to long-term archiving, such as font linking (as opposed to font embedding). PDF/A-1 is based on the PDF Reference Version 1.4 from Adobe Systems Inc. (implemented in Adobe Acrobat 5 and latest versions) Page 16/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req COLL-TRA-70] The Contractor shall check the completeness and legibility of the scanned documents through sample comparisons with the original. Note: in some cases, the data to be transcribed will not be recorded on media delivered to the Contractor but will be accessible on a repository on-line. In such cases, the same requirements (including concerning the checksum on the data) apply. 3.1.6.2.3 Storage of the data Performing the services required in this SoW requires the storage of the data by the supplier. This is not considered an official archival centre, since the official repositories of data for archival are the PACs or LTDP centres. However reliable storage is required to perform the tasks of this SoW including the delivery (repatriation) of data to ESA / PACs / LTDP centres after collection and analysis, consolidation and/or processing. Although the data will be repatriated to ESA / PACs / LTDP centres after consolidation / processing, the Contractor may be required to keep a working copy of the data in its storage infrastructure, in order eg to perform follow-on activities, multiple processing, multiple repatriation exercises, etc. Storage of the data [Req COLL-STO-10] The Contractor shall store the data collected into its data storage. [Req COLL-STO-20] The Contractor shall store the media/equipment holding the data in a storage area which is: compliant with the security requirements of this SoW (see Annex S); compliant with the technical requirements of this SoW (see section 4.4.3.2); compliant with the environmental prescriptions of the media/equipment manufacturer (in terms of temperature, humidity and other parameters); protected against fire and other disasters; protected against entry by non-authorised personnel; reserved for the storage of ESA data (not mixing of ESA and other data in the same [area of] physical storage) clearly marked as containing ESA property for the sole purpose of this project. [Req COLL-STO-30] The Contractor shall ensure, through the use of appropriate technology, processes, procedures, controls etc. that the data in the archive suffers no loss or alteration over the entire duration of the contract. This applies in particular to any transcription of the data performed within the Contractor’s storage, eg to replace ageing media. 3.1.6.2.4 Detection and Recovery of anomalies Page 17/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Several standard failure situations may occur while transcribing the data to the Contractor’s storage infrastructure. The Contractor is requested to detect failure situations and in those cases to take appropriate corrective action including at least (but not limited to) those described in the following requirements. Missing media in a series [Req COLL-ANOM-10] The Contractor shall detect cases of missing media in a series which may not have been visible at the stage of receiving the media (eg due to missing labels). [Req COLL-ANOM-20] In case of a missing media in a series, the Contractor shall liaise with the ESA Project Manager , and with the party having provided the media in order to recover the missing media. Media-related anomalies [Req COLL-ANOM-30] The Contractor shall detect cases of media-related anomalies, including: - mechanical failure in the source media (eg tape); - failure to read the source media; - failure to read part of the data on the source media; - discrepancy between the media directory and the readable contents; - discrepancy between any file descriptor (header, checksum, directory entry etc) and the file contents; - other error situations traceable to the source media. [Req COLL-ANOM-40] In case of a media-related anomaly, the contractor shall attempt to recover by applying at least the following corrective actions: - repeat the reading process on the same infrastructure; - repeat the reading process on an alternative infrastructure (eg different tape reader); - in the case of a tape media, spool the tape and repeat the reading process. [Req COLL-ANOM-50] In the case that failures in the reading of the source media cannot be solved using any of the steps defined above, the Contractor shall propose for each remaining failure a specific action plan. The specific action plan may include actions potentially damaging or irreversible for the media item if considered helpful. Such actions shall be flagged together with the expected likelihood of success and impact on the media. The specific action plans proposed, and the records of ESA’s decisions, shall be delivered as part of the overall Data Collection Report. [Req COLL-ANOM-60] The Contractor shall execute the specific action plan proposed for the specific media item, after explicit authorisation of the ESA Project Manager. The ESA Data Curator will be the ultimate authority for any action potentially damaging for the media. The ESA Project Manager will convey the authorisation to the Contractor. [Req COLL-ANOM-70] No destructive or irreversible corrective action shall be attempted on a media item without the prior written approval of ESA, communicated by the ESA Project Manager. Transcription-related anomalies Page 18/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req COLL-ANOM-80] The Contractor shall detect cases of differences between the data on the source media and the copy made in the target environment, including at least any errors detectable through the MD5 checksum calculated on the transcribed file vs the same checksum stored on the source media (if available); the file size of the transcribed file vs the file size on the source media. [Req COLL-ANOM-90] In case of errors in the checksum check, the Contractor shall repeat the copy and repeat the check of the failed reading. 3.1.6.2.5 Update of the Data Information System and CCM DB During the activity of receiving and classifying the media, the Contractor has populated the CCM data base with information regarding the media Configuration Items. In this activity, the Contractor populates the Data Information System and CCM DB with the information related to the data being collected. This can be done in the course of the transcription process, or as part of a subsequent analysis of the data, or a combination. Depending on the existence and level of detail of an inventory delivered together with the media, part of the information may already tentatively be known and will be confirmed or corrected as part of this activity. All the data in the broad sense including documents shall be covered by this exercise. [Req COLL-CCM-10] The Contractor shall determine the information and knowledge regarding the data transcribed, necessary to populate the CCM DB and Data Information System with the information deriving from reading the data, including: - information on the data contents, format, temporal and geographical coverage - preliminary assessment of the completeness and quality parameters - information on the transcription process, including provenance of the original media, errors recovered and not recovered, etc. Information on the data (esp. origin and lineage of the data, which is typically not included in the media delivered) shall be collected through various sources including the Contractor’s own corporate knowledge and sources, web sites, access to on-line catalogues, interviews with the ESA Data Librarian, ESA officers with specific knowledge of the data concerned, officers of the Facility who shipped the data, etc. The information collected from other sources than ESA shall always be submitted to ESA for rubberstamping before it is considered authoritative and included in the CCM DB and Data Information System. [Req COLL-CCM-20] The Contractor shall populate the Data Information System and CCM DB with the information and knowledge regarding the data newly ingested into the Service storage through the transcription activity. [Req COLL-CCM-30] In case of inventory delivered together with the data, the Contractor shall nonetheless determine/confirm autonomously the information and knowledge regarding the data, and shall store the delivered inventory for the record in the Data Information System and CCM DB. Page 19/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 3.1.6.2.6 Provision of a transcription report [Req COLL-REP-10] Throughout the duration of the transcription exercise, the Contractor shall keep the ESA Project Manager regularly informed about: - the progress of the exercise in terms of media transcribed; - the list of the failures encountered, corrective actions taken and outcome, and further proposals for resolving open issues. [Req COLL-REP-20]The Contractor shall produce a transcription report including the following information: - a listing of the media transcribed; - a summary of the files / data resulting from the transcription, reflecting the location and structure of the data on the target storage; - a summary of the failures occurred during the transcription of the source media, including: o id of the faulty media; o corrective actions taken in order to read the data, and corresponding outcome; o in the case of corrective actions with a potentially damaging or irreversible effect on the media, the reference to the ESA authorisation and description of the effective impact on the media; o suggested further corrective actions available if any; - a summary on the data which could not be (partially or totally) transcribed due to an unrecoverable failure, showing all the available information on the data including: o functional information: mission, instrument, product type, cycle, orbit, date/ time, etc. o technical information: file name, dimension, location on the media, etc The transcription report shall be delivered as part of the overall Data Collection Report. [Req COLL-REP-30] The Contractor shall make sure that the information reported in the transcription report is stored in the Data Information System / CCM DB. The contractor is expected to minimise its workload by generating the transcription report as an extract of the Data Information System and CCM DB. Note: the requirements on Data Information and Configuration Management become applicable as soon as the media are delivered to the Contractor to start the transcription and ingestion process, and remain applicable until the data is dismissed from the scope of the Service. 3.1.6.2.7 Throughput of the transcription [Req COLL-PERF-10] The Contractor shall complete the transcription of the input media within a time in line with the throughput agreed in the Project Proposal. The throughput required may vary between projects depending on the technology and age of the input media, and the needs of the project. Page 20/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 3.1.6.2.8 On-request copy to HDD In case a copy of the transcribed data (normally including documents) on HDD is requested by the ESA Project Manager , the Contractor shall activate the Data repatriation / delivery WP accordingly. 3.1.6.3 Destroy / archive / ship back the original media After the transcription process, the input media, depending on the situation and the instructions of the ESA Project Manager , shall be destroyed, archived, or shipped back to a destination specified by the ESA Project Manager (by default, the entity which supplied the tapes in the first place). [Req COLL-ORIG-10] In all cases, the Contractor shall fully update the CCM DB and the Data Information System with a record of the decisions taken and the corresponding actions wrt destroy / archive / ship back the original media. The following requirements apply depending on the situation. 3.1.6.3.1 Storage of the input media [Req COLL-ARC-10] After the transcription, the Contractor shall keep the source media in its storage area, compliant with the above requirements for physical storage and the Security Requirements, until further notice. 3.1.6.3.2 Destruction of the input media [Req COLL-DESTR-10]The Contractor shall obtain from ESA and archive a written statement of the decision to destroy the input media. [Req COLL-DESTR-20] The Contractor shall dispose of the media in compliance with the local environmental regulations and best practices. [Req COLL-DESTR-30] Any media not successfully transcribed shall be saved from destruction and safely kept in the Contractor’s storage area or shipped back to ESA as indicated by the ESA Project Manager. [Req COLL-DESTR-40] The Contractor shall wait for 1 month after ESA’s written confirmation before proceeding to destroy the media in question. [Req COLL-DESTR-50] The Contractor shall keep the media properly stored in its storage environment until the destruction. 3.1.6.3.3 Shipping of the input media Page 21/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The same requirements listed for the shipment of HDDs as part of the Data Repatriation/delivery WP apply. See chapter 4 Data Repatriation/delivery Section 4.2.6.3 Data distribution on HDD Sub-section Shipment of the HDD 3.1.6.4 Verification of the Data Collection activities [Req COLL-VERIF-10] The Contractor shall prepare the Verification portion of the Verification and Validation plan (including the test specifications, test scenarios and any tools used in the testing exercise and covering non regression tests if applicable), perform a Verification exercise on the basis of the plan and document the outcome in a Verification Test Report. Note: for brevity, the Verification portion of the Verification and Validation plan is also referred to as the Verification plan. It is however not a separate deliverable but an integral part of the V&V Plan deliverable. [Req COLL-VERIF-20] The test scenarios of the Verification Plan shall cover at least the following: - audit of the transcription report against actual data inserted - sample checks of the data transcribed vs records in the Data Information System and CCM DB - sample checks on the checksum of the files transcribed vs the original files - sample checks on the original media vs the data transcribed - any other approach depending on the specific project and data. [Req COLL-VERIF-30] The Contractor shall execute the Verification Plan and document the outcomes in the Verification portion of the Verification & Validation report. Note: for brevity, the Verification portion of the Verification and Validation report is also referred to as the Verification report. It is however not a separate deliverable but an integral part of the V&V Report deliverable. The Verification plan and report will be subject to the approval of the ESA Project Manager. After the successful execution of the Verification plan, the Data Release Management process shall be activated. 3.1.6.5 Data Collection guarantee and follow-up The activities of data collection as described in this WP are treated on a project basis. The project terminates with the completion of all the activities required, based on the data delivered to the Contractor as input and any additional data collected by the Contractor itself. The completion of the project is marked by the successful completion of the Verification and of the Validation (see Chapter “Release Management”). Page 22/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use However, there is a significant likelihood that additional data which is part of the same mission / instrument / time and geographical coverage as the input data will become available for collection after the project activities are underway including after the completion of the project. In addition, despite the care taken in the Verification and Validation exercises, ESA or the Contractor may discover, either autonomously or following notification of further recipients of the data (eg PACs / LTDP Centres) inaccuracies in the outcome of the activities performed in the course of the project. [Req COLL-GUA-10] The Contractor shall continue to perform / repeat the project activities on an incremental basis to cover: a- cases of errors and omissions by the Contractor in the Project b- the possible discovery of additional input data. [Req COLL-GUA-20] The additional collection of further input data (case b above) shall be considered incremental (and therefore fall under the guarantee and follow-up activities) up to a yearly amount of 3% of the original data. Beyond this threshold, further activities will be considered as a new project. [Req COLL-GUA-30] The guarantee and follow-up period shall last for 1 year after the completion of the project (or until the end of the Contract). The guarantee and follow-up activities are an integral part of the project from the funding point of view, ie no additional cost will be paid by ESA for these activities. However from the operational point of view they will be carried out as continuous activities managed under the overall service and not any more as project activities. 3.2 Data consolidation 3.2.1 Description For the purpose of this SoW, Data consolidation is a process consisting of 3 stages, as represented in the Figure hereunder: - Stage 0: data cleansing, which consists in producing from one or several input datasets a single dataset which is devoid of corrupted files, aligned on the same naming convention and file format, and devoid of exact duplicate files; - Stage 1: analysis of overlaps, which consists in analysing the overlaps remaining inside the files and proposing a strategy to reduce the redundancy; - Stage 2: advanced consolidation, which consists of ad-hoc interventions which require advanced instrument and scientific knowledge of the data and is performed by SPPA (see Annex 4). Depending on the case, it may be appropriate to apply Stage 0 only or Stage 0+1, or all three stages. This WP consists in - Executing Stage 0 of the consolidation - Facilitating Stage 2 of the consolidation (which is always performed by SPPA). Page 23/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Stage 1 is not in the scope, the Contractor may be required to perform it as part of a separate contractual arrangement. The WP is supervised by the ESA Project Manager, however SPPA will contribute to the supervision of the work (eg checking preliminary outputs, participating in meetings, etc). The intermediate decisions / assessments and validations of proposals by the Contractor are performed by the DSI-CCB, in particular the DSI-CCB is entitled to designate which among the various datasets is the master dataset for a given mission / instrument / time period. - Inventory of sources - Removal of corrupted files - Harmonisation of naming convention - Harmonization of format - Removal of exact duplicates - Analysis of remaining gaps - Analysis of overlapping portions - Proposal of a strategy to reduce redundancy - Further consolidation based on measurement / instrument / scientific and other advanced data knowledge Consolidation Stage 0: “Data cleansing” Consolidation Stage 1: “Analysis of overlaps” Consolidation Stage 2: “Advanced Consolidation ” Data Consolidation: breakdown into stages Note: for each specific project, there may be some tuning to the requirements in this WP, following inputs / guidance by SPPA depending on specific situations. 3.2.2 Scope The data covered by this WP is potentially any data including L0 or higher and auxiliary data. However it is expected that the target will in most cases be L0. In some cases, SPPA will provide data already consolidated, particularly for auxiliary data. Depending on - the initial status of the source datasets available, their degree of completeness and quality and other factors, - the recommendations of the Contractor supported by its expertise and/or a preliminary analysis of the data, - the advice of SPPA, - the decision of the Customer, the scope of the WP covers: - Stage 0 of the consolidation - Upon explicit request, Stage 1 of the consolidation - Facilitation of Stage 2 of the consolidation Page 24/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The 3 stages are described in the requirements hereunder. The scope does not include - the rescue of the files from old media, which is part of the “Data collection” WP; - the actual execution of Stage 2. Mission documentation is covered to the extent necessary to support the exercise. It is not in the scope of this WP to perform a systematic collection of a mission documentation, however collecting the documentation necessary to perform the consolidation (along with any mission-related documents which may be required for the subsequent activities to be performed on the data eg bulk processing reprocessing) is part of the scope. 3.2.3 Activities The activities required include: - Stage 0 (Data cleansing): o assessing the available sources of data, possibly collecting additional sources and triggering the “Data collection” WP for the sources selected; o collecting additional resources necessary for the exercise; o detecting and eliminating corrupted files; o aligning / homogenising the data, including for File Naming and Format / packaging, possibly by means of ancillary tools developed for the purpose; o merging of data from different sources and eliminating exact duplicates; o reducing the redundancy across the files. - Stage 1 (Dealing with overlaps): o analysing the overlaps inside the files, possibly by means of ancillary tools developed for the purpose; o proposing a strategy to reduce / eliminate the overlaps within the files. - Facilitation of Stage 2: o liaising with SPPA (or a third party designated by SPPA) to facilitate their intervention on the data; o providing assistance to SPPA or the third party in question to facilitate the further consolidation of the data. - In conjunction with all 3 Stages: o producing various statistics on the consolidation process; o assessing the resulting completeness; o liaising with DSI-CCB for the validation of the outputs and the designation of the master dataset. 3.2.4 Inputs Data holdings from various sources including the DSI Service itself Page 25/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Possible additional sources of information and data Additional resources (as explained below) 3.2.5 Outputs Individual datasets resulting from each of the Consolidation Stages and candidate for being labelled Master Regular deliveries of the Consolidation Technical Note, documenting the various steps, trade-offs, decisions, etc. Updated Data Information System and CCM DB Any ancillary tools used / developed by the Contractor during the course of the Consolidation exercise to read / convert / assess redundancy / assess completeness etc. Verification documentation 3.2.6 Requirements 3.2.6.1 Stage 0: Data Cleansing 3.2.6.1.1 Collection of input data There are sometimes many sources available for the same data, including: main archives hosting a complete dataset, and therefore which should be targeted first, such as Processing and Archiving Facilities; alternative archives hosting a backup or a large percentage of a dataset, in particular for low level data, ground stations, re-transcription centres, Processing and Archiving Facilities using the files as input; other sources : hosting a limited amount of the data: backup ground stations (such as the stations used after ERS on-board recorder failure), users holding copies of files (less frequent for low level data), etc. It is important to identify all the possible sources because: in some cases no archive contains all the data and completeness can only be achieved by merging several sources; some files may be corrupted and have to be replaced by another copy some files may be stored on obsolete media for which reading devices are no more available or not for the latest hardware the access or restoration of the products may involve more time or manpower depending on the source. [Req CONS-STG0-10]The Contractor shall identify any additional sources in addition to the input datasets provided by ESA as CFI, which may be used in the consolidation process and improve the quality and completeness of the output. Page 26/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req CONS-STG0-20] The Contractor shall retrieve from the selected sources any input data / media which is neither provided by ESA as input to the WP nor already in the possession of the Contractor. The data collection activities shall not affect the overall project schedule as agreed with ESA. The ESA Project Manager will confirm the use of the additional sources and the approach by the Contractor of the corresponding sources. [Req CONS-STG0-30] The Contractor shall recommend a prioritisation of the sources, indicating which should be ingested for consolidation and which (if any) should be left aside, on the basis of a cost benefit trade-off between the cost of extracting the data and the amount of data expectable from each source. [Req CONS-STG0-40] The Contractor shall provide an estimate of the completeness and redundancy levels which can be expected by including data from each source at Stage 0 and by applying respectively Stage 0 and Stage 1 consolidation to the selected input data. Note: the Contractor is expected to base this estimation on its knowledge and experience of the mission / data in question and of similar activities. No specific level of accuracy is required, but a rational estimation approach supported by previous experience. [Req CONS-STG0-50] The Contractor shall collect and ingest into its storage infrastructure the data from the various input sources (or selected portions) in compliance with the process and requirements of WP “Data collection”. Depending on the input sources and the packaging of the data it may be necessary to separate the measurement data (or more generally the target data) from other data possibly contained in the inputs: [Req CONS-STG0-60] The Contractor shall separate in the input sources the target data from the other data contained. This includes splitting up compound products (made up of eg a measurement file and an xml file of meta-data) and separating the measurement data file from the remaining files. Note: no data shall be discarded and any files not involved in the consolidation exercise shall be kept in the service storage and inventoried in the Data Information System and CCM DB. 3.2.6.1.2 Collection of additional resources In addition to the data itself, data consolidation requires some critical information in particular to assess the contents of the data, to homogenise the data and to evaluate its completeness. In particular, the following documentation is typically necessary: Accurate and complete format description, for all versions of files : this must be available both for the analysis of the file content and extent, and then for end-usage of the files. Unambiguous and extensive file naming convention. History of processing software changes will be required to understand the differences between redundant or various generation files and eventually set up a selection strategy. In addition, some mission- and spacecraft-related ancillary data is necessary for homogenisation and completeness analysis; this may include typically: Page 27/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use mission phases : a clear definition of the mission phases (start/end orbit and start/end times); if applicable, mission cycles : proper and clear definition of the cycles ( start/end orbit and start/end times ); orbits : clear definition of all mission orbits ( start/end orbit and start/end times ) information on payload activity and anomalies : unavailability or anomaly notifications for platform, instrument, telemetry or ground stations is essential to assess the level of completeness of the data results of previous assessments of the data (eg for completeness analysis) Detailed Mission Operations Plan (DMOP): reporting the planning input against actual time series. This explains when periods with no measurements were planned (e.g. MIPAS duty cycle not 100% throughout the entire mission). [Req CONS-STG0-100] The Contractor shall determine the list of additional resources (documentation and ancillary data) in addition to the information and documentation provided by ESA as CFI, which may be helpful for the consolidation exercise, and the authoritative sources for these resources. The ESA Project Manager will confirm the use of the additional resources and the approach by the Contractor of the corresponding sources where these additional resources are available. [Req CONS-STG0-110] The Contractor shall collect the additional resources from their authoritative sources. [Req CONS-STG0-120] In case the Contractor fails to collect any of the inputs required for the exercise, the Contractor shall: - justify in the Consolidation Technical Note the reason for the unavailability and the work-around attempted; - analyse in the Consolidation Technical Note the alternative approach and the impact on the endresult of performing the exercise without the inputs in question. [Req CONS-STG0-130] The Contractor shall incorporate the additional sources into the Service CCM and Data Information System. [Req CONS-STG0-140] The Contractor shall document in the Consolidation Technical Note the analysis of the additional resources considered necessary for the consolidation exercise, the availability at the start of the project, the resources collected as part of the project, and any resources not available together with the potential impact of this unavailability on the overall consolidation project. 3.2.6.1.3 Removal of corrupted files [Req CONS-STG0-200] The Contractor shall detect any files which may be corrupted, including: - files with contents or size not compliant with the header/meta-data; - files with size / duration obviously not in line with reasonable expectations considering the average file population; - files with field values not coherent with the nature of the field; - files with a checksum not equal to the checksum possibly contained in the file title (if available). [Req CONS-STG0-210] The Contractor shall eliminate the corrupted files from the target dataset. Page 28/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Note: this requirements only applies to fully corrupted files, ie without intact portions from which useful results may still be obtained. [Req CONS-STG0-220] The Contractor shall replace the corrupted files with intact versions if present in another source, if available. Note: ESA will deliver a new version of files flagged by the Contractor as corrupted, if available. [Req CONS-STG0-230] The Contractor shall keep in the target dataset the files only partially corrupted (eg with portions of corrupted data) and flag them in order to be able to closely monitor the subsequent utilisation (eg processing) of these files. 3.2.6.1.4 Alignment / homogenisation of the data The files collected for the consolidation may be heterogeneous in terms of: - file naming; - format; - packaging eg including or not an XML metadata file for each file into a compound product . The data needs to be homogenised in order to facilitate the consolidation as well as the further utilisation of the data. File Naming [Req CONS-STG0-300] The Contractor shall propose (in cooperation with the ESA Project Manager ) and implement a naming convention for the consolidated data, based on: - any ICDs and other applicable documents; - the expectations (ICD) of the processor envisaged for subsequent processing of the data; - the most frequently used naming convention used in the sources being consolidated; - the most recent file naming convention used for the mission; - the most widely used file naming convention by the users of the data; - the most suitable file naming convention. The proposed naming convention should be an existing standard naming convention. Unless necessary to satisfy the required criteria, no new naming convention should be invented. [Req CONS-STG0-310] The Contractor’s proposed naming convention shall have no dependency on other resources, ie all the fields of the file name (eg orbit number, start and stop time, checksum code) shall be extracted from the contents of the file itself. [Req CONS-STG0-320] The Contractor’s encoding approach for the receiving stations should be compliant with [RD STATION CODES]. Page 29/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req CONS-STG0-330] The Contractor’s proposed naming convention should be based on the standard definition of orbit (and corresponding time frame) = “from equator crossing in ascending node (ANX) to next equator crossing I ascending node (ANX+1)”. Other definitions (eg from pole to pole) should be avoided. [Req CONS-STG0-340] The Contractor’s proposed naming convention shall be unambiguous (eg in case a checksum is included in a file name, the choice of calculation algorithm shall be indicated explicitly). [Req CONS-STG0-350] The Contractor’s proposed naming convention shall be fully documented: - as part of the dataset information collected in the Data Information System and CCM DB for the dataset resulting from the consolidation; - in the Consolidation Technical Note. [Req CONS-STG0-360] The Contractor’s proposed naming convention shall ensure that files with any differences in their contents are assigned different names, and that files with identical contents are assigned identical names. Note: this is expected to involve a checksum being part of the file name. The DSI-CCB, in consultation with other ESA stakeholders, shall make the final decision in the choice of the file naming convention of the consolidated data. Format and packaging [Req CONS-STG0-370] The Contractor shall propose (in cooperation with the ESA Project Manager ) and implement a format and packaging for the consolidated data, based on: - any ICDs and other applicable documents; - the expectations (ICD) of the processor envisaged for subsequent processing of the data; - the most frequently used format and packaging used in the sources being consolidated; - the most recent file format and packaging used for the mission; - the most widely used file format and packaging by the users of the data; - the most suitable file format and packaging for the long term preservation of the consolidated data. The proposed file format and packaging should be an existing standard file format and packaging. Unless necessary to satisfy the required criteria, no new file format or packaging should be invented. [Req CONS-STG0-380] The Contractor’s proposed file format and packaging shall be fully documented: - as part of the dataset information collected in the Data Information System and CCM DB for the dataset resulting from the consolidation; - in the Consolidation Technical Note. The DSI-CCB, in consultation with other ESA stakeholders, shall make the final decision on the format and packaging of the consolidated data. Homogenisation [Req CONS-STG0-390] The Contractor shall align the data from all the sources contributing to the consolidation to the chosen file names, format and packaging. Page 30/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use In the case that reading / conversion tools are not fully available among the material collected, the homogenisation of the data may require the development by the Contractor of reading / conversion tools. Tools and configuration management [Req CONS-STG0-400] The Contractor shall validate and place under Configuration Management any reading / conversion / packaging etc. tools developed and/or used in the scope of this WP. [Req CONS-STG0-410] The Contractor shall update the Service Data Information System and CCM DB to reflect the naming / formatting / packaging harmonisation operations performed. Any reading / conversion / packaging etc. tools developed in the scope of this WP are deliverable to ESA, including the related documentation. 3.2.6.1.5 Merging of data from different sources [Req CONS-STG0-500] The Contractor shall merge the data from the various sources into a single dataset. [Req CONS-STG0-510] The Contractor shall eliminate from the merged dataset the cases of: - exact duplicates (same file name and content bit-by-bit). 3.2.6.1.6 Reduction of redundancy [Req CONS-STG0-600] The contractor shall make a preliminary assessment and a recommendation concerning the redundancies in the resulting dataset, and propose a course of action among: a) no further action, eg in case there are few redundancies; b) a file-based approach, where full files are kept or discarded without modifying /stitching the contents of the files, eg in case the redundancies are concentrated in some of the files; in this case the Contractor shall recommend and justify proposed criteria for selecting highly redundant files to be discarded; c) a record-based approach involving further analysis and in view of potential modification/snipping/stitching of the contents of the files (Stage 1). [Req CONS-STG0-610] The Contractor shall document the assessment and recommendation on the reduction of redundancy in the Consolidation Technical Note. The ESA Project Manager, through the DSI-CCB and other ESA stakeholders, will assess the recommendations and decide on a course of action. [Req CONS-STG0-620] In case the ESA Project Manager , upon recommendation of the DSI-CCB and other ESA stakeholders, opts for a file-based approach (case b above), the contractor shall produce a cleansed dataset by discarding the files responding to the selected criteria. Page 31/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 3.2.6.1.7 Throughput requirements [Req CONS-STG0-700] The Contractor shall complete Stage 0 of the Consolidation in a time not exceeding the throughput indicated in the Project Proposal. 3.2.6.2 Stage 1: Analysis of overlaps Stage 1 consists of the analysis of the overlaps within the files. It is only part of the WP of Data Consolidation if explicitly required. In this case it will be covered by data engineering activities to be specified by SPPA. It is not in the standard scope of this WP. The requirements shown in this section are typical requirements which may apply to Stage 1 consolidation, however the actual requirements will be specified by SPPA case-by-case. [Req CONS-STG1-10] The Contractor shall check that the presumed file start and end times (according to the file name / header) match the actual contents, and if not shall correct the start and end times. [Req CONS-STG1-20] The Contractor shall compute for each file the overlap length and percentage with the previous and next files in chronological order. [Req CONS-STG1-30] The Contractor shall analyse the overlapping sections of two respective files to determine if the corresponding data is fully redundant or not, and on this basis produce an index of redundancy. The analysis shall include at least whether the following criteria are identical in the two overlapping files / file sections: - number of data records; - record times; - acquisition station / processing centre; - processor version; - data in the overlapping records: meta-data, ancillary information, quality fields, measurement. [Req CONS-STG1-40] On the basis of this analysis, the Contractor shall propose a strategy (eg of stitching of files and removal of overlapping records) for the elimination / reduction of the redundancy in the data. [Req CONS-STG1-50] The proposed strategy proposed by the Contractor shall: - be executable through a defined process applied to the full dataset (a case by case / file by file action is not in the scope of Stage 1 and may be in the Scope of Stage 2 to be performed by SPPA); - include rules for associating the newly formed primary and auxiliary data files resulting from the stitch-and-remove process; - show any trade-off proposed between the removal of redundant data and the loss of unique data. [Req CONS-STG1-60] The Contractor shall submit to SPPA the proposed redundancy-reduction strategy and shall revise the strategy according to the feedback of SPPA. Page 32/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req CONS-STG1-70] The Contractor shall document in the Consolidation Technical Note the calculation of the index of redundancy and the proposed and revised redundancy reduction strategy. SPPA will decide if the proposed redundancy-reduction strategy and/or a different strategy is effectively applied to the data. In the case that SPPA decides to take action, the corresponding implementation may be performed by SPPA (and its supporting contractual structure) outside the scope of this contract, or it may be entrusted to the Contractor in the scope of this Contract. This can take the form of a processing project (eg applying a consolidation tool like ECONS to the data) or be covered by data engineering activities to be specified by SPPA. It is in all cases outside the scope of the consolidation WP. [Req CONS-STG1-100] The Contractor shall update the Data Information System and CCM DB to reflect the redundancy-reduction strategy proposed, the validation of the strategy by SPPA, and any redundancyreduction operations performed, either by the Contractor or by a third party providing visibility to the Contractor. Any redundancy-analysis or redundancy-reduction tools developed in the scope of this WP are deliverable to ESA. 3.2.6.2.1 Throughput requirements [Req CONS-STG1-110] The Contractor shall complete Stage 1 of the Consolidation in a time not exceeding the throughput indicated in the Project Proposal. Note: there may be parallelism between Stage 0 and Stage 1. 3.2.6.3 Facilitation of Stage 2: Advanced Consolidation The Contractor’s involvement in Stage 2 is in essence limited to data transfer to and from SPPA and provision of information on an advisory basis. Stage 2 is in the standard scope of the Consolidation WP however it may be waived on a case-by-case basis if SPPA decides not to perform Stage 2 activities. The data repatriation activities follow the same processes and requirements listed in Ch 4.2 Data Repatriation/delivery however the data volumes involved come on top of the volume estimates which are indicated in Ch 4.2. Stage 2 of the consolidation consist in ad-hoc treatment of the data by SPPA or a SPPA-designated third party (generically referred to as SPPA in this section) based on measurement-related, instrument-related, scientific or other advanced knowledge. Note: this may in principle apply only to ADFs or Level 1 products, as Level 0 products are not changed by SPPA only analysed. In some cases, what may be needed is a Level 1 processing in order to judge the quality of the input Level 0. The Contractor will be requested to interface SPPA (or a SPPA-designated third party) for the provision of the data for Stage 2 consolidation and the recuperation of the data afterwards for further custody in the Page 33/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use service storage and processing. The Contractor will also be required to give the assistance that SPPA may require for its Stage 2 consolidation, “on an advisory basis” ie in terms of knowledge of the particular dataset and its lineage. [Req CONS-STG2-10] The Contractor shall facilitate the access to the data by SPPA by the means selected by SPPA amongst: - selecting a subset of the data if requested, by applying filtering criteria specified by SPPA; - delivering a copy of the data to SPPA by means of the same media used for the on-media bulk delivery; - putting a copy of the data on-line at the disposal of SPPA by means of the same access mechanism used for the on-line data delivery. [Req CONS-STG2-20] The Contractor shall give assistance to SPPA on the data to be consolidated by providing knowledge about the data on an advisory basis. [Req CONS-STG2-30] The Contractor shall liaise with SPPA and recuperate the data at the end of SPPA’s intervention, through the same means used to deliver the data to SPPA for its Stage 2 consolidation. [Req CONS-STG2-40] The Contractor shall put the data at the disposal of SPPA within 2 NWD of the request and recuperate the data within 2 NWD of SPPA’s notification of availability. [Req CONS-STG2-50] The Contractor shall collect from SPPA the full information about the operations performed and corresponding outcomes. [Req CONS-STG2-60] The Contractor shall update the Service CCM and Data Information System to reflect the Stage 2 consolidation operations performed. 3.2.6.4 Statistics, completeness and quality assessment 3.2.6.4.1 Production of statistics on anomalies and redundancies [Req CONS-STAT-10] The Contractor shall produce statistics on the Stage 0 Consolidation exercise including (before and after the cleansing exercise): - list, number, percentage of files detected as fully corrupted (ie cannot be read); - list, number, percentage of files detected as clones ie exact duplicates (including clones revealed as a result of the renaming operation); - list of file renamings, showing the old name and the new name) per source, number and percentage of files renamed per source / overall; - list of different formats encountered per source and list, number, percentage of files per format and of files translated to a different format; - if applicable as part of the approved redundancy-reduction policy: list, number, percentage of files rejected because they were overlapping with other files and having lower quality or fewer records, amount of overlap before and after the overlap reduction, data completeness before and after the overlap reduction. Page 34/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req CONS-STAT-20] In the case that Stage 1 is activated, the Contractor shall produce statistics on the Stage 1 Consolidation exercise including: - list, number, percentage of overlapping portions of files before and after the redundancy-reduction exercise; - proportion of overlaps (index of redundancy) in the output dataset. [Req CONS-STAT-30] The Contractor shall produce the statistics in a way which allows automatic filtering for specific statistics: - per phase of the mission; - per type of input media; - per input source; - according to other TBD criteria. [Req CONS-STAT-40] The Contractor shall include / append the statistics produced to the Consolidation Technical Note. 3.2.6.4.2 Assessment of completeness The assessment of completeness of the dataset consists in detecting the missing data (with respect to planning – DMOP) and qualifying these gaps either as platform/instrument/station anomalies or data loss. In principle, all missing data should be related to a platform or instrument unavailability (such as manoeuvre, orbit or phase change) or anomaly during data acquisition in the ground segment. Any “lost data” must be considered as suspect and solved either by improving the list of anomalies or recovering the still missing data (in some cases to be justified there may be a remaining part of “unexplained gaps” as sometimes the corresponding info cannot be retrieved anymore). An index of completeness is then calculated to qualify the output dataset. [Req CONS-STAT-60] The Contractor shall detect: - inter-file gaps, ie elapsed time between the end and the start of two chronologically consecutive files; - intra-file gaps, time between two consecutive records within the same file larger than a threshold commensurate with the mission / instrument /type of file. [Req CONS-STAT-70] The Contractor shall determine the duration and geographical location of each gap. [Req CONS-STAT-80] The Contractor shall produce a list of the gaps, each associated with its duration and the corresponding geographical location. Crossing with unavailabilities / anomalies [Req CONS-STAT-90] The Contractor shall establish a list of the data which can be expected to be present considering all the availability factors, ie: - mission planning if applicable Page 35/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - instrument unavailabilities platform unavailabilities station unavailabilities subject to consultation with the ESA Data Librarian or DSI-CCB, the “inter-phase” time ie the time elapsed in between two consecutive phases, in which measurements may be available but are not considered significant. [Req CONS-STAT-100] The Contractor shall produce a “net” list of the gaps, each associated with its duration, orbit identification and time, and the corresponding geographical location, limited to the gaps not explained by an unavailability. [Req CONS-STAT-110] The Contractor shall compute an index of completeness of the consolidated dataset based on: - the data which was expectable considering the unavailabilities; - the data which is indeed present in the dataset. [Req CONS-STAT-120] The Contractor shall provide a justification for any gap in the data not explained by a known unavailability, including: - possible reason why the sources of data in input do not contain the data in question - possible additional sources of data able to fill the gap; - possible presence of the data in the sources rejected following the cost benefit trade-off performed at the start of the exercise; - possible additional unavailabilities. [Req CONS-STAT-130] The Contractor should use catalogue information from the various organisations managing and distributing the data being consolidated in order to support the differentiation between data which was never acquired and data which was lost. [Req CONS-STAT-140] Upon request of the ESA Project Manager , the Contractor shall propose a revised (enlarged) input set of data to be consolidated, in the form of a delta consolidation including additional data previously excluded, together with an estimate of the gain expectable from a new iteration including this extra data. [Req CONS-STAT-150] The Contractor shall fully document the completeness assessment and related statistics and further proposal the Consolidation Technical Note. 3.2.6.4.3 Update of the data storage and Data Information System and CCM DB [Req CONS-STOR-10] The Contractor shall store in the Service storage and keep under CM the data and other outputs resulting from all the activities of consolidation performed. [Req CONS-STOR-20] The Contractor shall keep at all times the Data Information System and CCM DB up to date by to recording the operations performed as part of the consolidation and the resulting evolution of the statistics and designation of master dataset. Page 36/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req CONS-STOR-30] The Contractor shall ensure that the final outputs of the consolidation exercise are fully recorded in the Data Information System and CCM DB within 3 NWD of the delivery of the outputs to the ESA Project Manager. 3.2.6.5 Verification of the Data Consolidation activities [Req CONS-VERIF-10] The Contractor shall prepare the Verification portion of the Verification and Validation plan (including the test specifications, test scenarios and any tools used in the testing exercise and covering non regression tests if applicable), perform a Verification exercise on the basis of the plan and document the outcome in a Verification Test Report. Note: for brevity, the Verification portion of the Verification and Validation plan is also referred to as the Verification plan. It is however not a separate deliverable but an integral part of the V&V Plan deliverable. [Req CONS-VERIF-20] The test scenarios of the Verification Plan shall cover at least the following: - sample checks and statistical evidence of the presence of corrupted / duplicated files / inconsistent file naming / inconsistent file format - sample checks and statistical evidence of the presence of unjustified gaps - sample checks and statistical evidence of the presence of overlaps among the files - the recalculation of selected samples of the statistics produced, using the same tools as in the consolidation exercise or alternative tools - sample checks of the data storage contents vs records in the Data Information System and CCM DB - any other approach depending on the specific project and data. [Req CONS-VERIF-30] The Contractor shall execute the Verification Plan and document the outcomes in the Verification portion of the Verification & Validation report. Note: for brevity, the Verification portion of the Verification and Validation report is also referred to as the Verification report. It is however not a separate deliverable but an integral part of the V&V Report deliverable. The Verification plan and report will be subject to the approval of the ESA Project Manager. After the successful execution of the Verification plan, the Data Release Management process shall be activated. 3.2.6.6 On-request copy to HDD In case a copy of the consolidated data (normally including documents) on HDD is requested by the ESA Project Manager , the Contractor shall activate the Data repatriation / delivery WP accordingly. Page 37/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 3.2.6.7 Data Consolidation guarantee and follow-up The activities of data consolidation as described in this WP are treated on a project basis. The project terminates with the completion of all the activities required, based on the data delivered to the Contractor as input and any additional data collected by the Contractor itself. The completion of the project is marked by the successful completion of the Verification and of the Validation (see Chapter “Release Management”). However, there is a significant likelihood that additional data which is part of the same mission / instrument / time and geographical coverage as the input data will become available for consolidation after the project activities are underway including after the completion of the project. In addition, despite the care taken in the Verification and Validation exercises, ESA or the Contractor may discover, either autonomously or following notification of further recipients of the data (eg PACs / LTDP Centres) inaccuracies in the outcome of the activities performed in the course of the project. [Req CONS-GUA-10] The Contractor shall continue to perform /repeat the project activities on an incremental basis to cover: a- cases of errors and omissions by the Contractor in the Project b- the possible discovery of additional input data. [Req CONS-GUA-20] The additional consolidation of input data (case b above) shall be considered incremental (and therefore fall under the guarantee and follow-up activities) up to a yearly amount of 3% of the original input data. Beyond this threshold, further activities will be considered as a new project. [Req COLL-GUA-30] The guarantee and follow-up period shall last for 1 year after the completion of the project (or until the end of the Contract). The guarantee and follow-up activities are an integral part of the project from the funding point of view, ie no additional cost will be paid by ESA for these activities. However from the operational point of view they will be carried out as continuous activities managed under the overall service and not any more as project activities. 3.3 Integration of Processing Systems 3.3.1 Description This WP consists in: - the deployment by the Contractor of the necessary infrastructure to operate a Processing System (see definitions in Annex 1), its installation and integration into the infrastructure so that the processing exercise is technically ready to start, Page 38/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - the technical verification and validation of the Processing System as integrated in the operational environment, - the readiness of the infrastructure to start the processing. Note: In order to avoid any confusion on the different types of verification and validation regarding processors, attention is drawn to the definitions of verification and validation in Annex 1. While these definitions apply to all the WP’s in this SoW, it may be useful to further clarify the situation concerning the WP of Integration of Processing systems in the case of a data processor: - a processor is in principle delivered as CFI to the Contractor after undergoing Scientific Validation by SPPA, meaning that the algorithm as implemented produces scientifically correct results; - the act of integrating the processor in the Contractor’s infrastructure is a technical task which requires verification and validation to ensure that the processor has been integrated properly (eg the processor software executes smoothly, uses the right input files with the specified orchestration, etc). This is ensured via a process of (technical) verification of the integration by the Contractor, followed by the (technical) validation of the integration by ESA; - in some cases, the processor delivered has undergone an initial phase of scientific validation but is not fully scientifically validated. Indeed a full scientific validation requires the processing of a large amount of data (up to half the data of the mission) and the scrutiny of the processed outputs by specialised scientific teams. It may therefore happen that a processor is integrated, and subsequently data is processed, in the scope of this SoW, for the purpose of supporting the full scientific validation. Clearly in such a case the technical verification (by the Contractor) and validation (by ESA) of the integration is performed normally. - The scientific validation is never in the scope of this SoW and is performed by the competent scientific organisation under the control of SPPA. The agility, rapidity, low cost and high pro-activeness of the Contractor in performing the integration will be a major factor in the satisfaction of ESA. Various degrees of challenges may need to be considered which depend on: - the type and complexity of the Processing System architecture and its interfaces - the complexity of the orchestration model to be implemented and input data selection rules - the complexity of the workflow and dataflow of the processing system - processing system parallelisation capabilities and constrains - the number of different input data and their volume - the number of different output products (which drives the complexity of the validation) and their volume - the degree of effective compliance of the Processing System to the ICD and other documentation - the degree of completeness of the processing system documentation, in particular information contained in ICD and user/operations manual - the performance requirements. Page 39/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Note: setting up fast and easy access to datasets for external interfaces (SPPA) required for QC and validation, is also an important factor. This is covered as part of the repatriation / delivery WP. Note: the definitions in Annex 1 explain the terms Processing System and Processor as used in this SoW. 3.3.2 Scope The scope of this WP includes all the activities necessary between the delivery of the Processing System including its accompanying documentation (and possibly sample data) and the readiness to start the processing exercise with the required performance. This includes in particular the implementation of the required orchestration model. The scope also includes the activities to keep the Processing System operational (or in a position to be made operational again at short notice) for a period of time after the end of the main processing campaign as well as the integration in the operational environment of minor upgrades to the Processing System. 3.3.3 Activities The activities required include: - receive the Processing System from ESA (SPPA), including related documentation and sample/test data and test scenarios - place the Processing System under Configuration Management for the scope of the activities under this SoW (obviously the configuration management related to the development and maintenance of the processing systems is not in the scope of this SoW) - deploy the appropriate infrastructure necessary to perform the processing (infrastructure tailored to the needs of the provided processing system) with the required performance (to remain available for the period of time required) - install and integrate technically the Processing System into the operational environment; the technical integration includes the setup / implementation of the orchestration model specified for the processor; - verify the technical integration of the Processing System into the operational environment, including by o processing the sample data provided with the processor or already stored in the Contractor’s repository and specified as sample data according to agreed test scenarios; o comparing the outputs to the output test data; - deliver the output of the test processing for further checks by ESA / organisation owning the Processing System (data delivery is in scope of this WP and follows the processes and requirements of the Data Repatriation WP) - generate CR/PR against the Processing System - receive upgrades of the Processing System and perform delta activities of integration. Page 40/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 3.3.4 Inputs Specific project requirements (Annex 2) including start and end dates of processing and additional period of processing capability) Processing system software and related documentation. The document set available may vary with the specific processor, but will in general include the following or equivalent documents: - Tailored ICD of the specific processor and if available compliance matrix with ESA Generic IPF ICD specifications - ESA Generic IPF ICD specifications [RD IPF-ICD] - Processing System Architectural overview and data flow description (possibly part of tailored ICD) - Processing strategy document, indicating recommended/possible processing sequence and any constrains. Typically, the Processing strategy document provides or confirms such inputs as: o reprocessing window (start orbit end orbit) o processor version is applicable for the reprocessing campaign (and which product baseline is implemented) o which aux data is to be used and which version of aux data is applicable (for instance for precise orbit information) o which products shall be generated during reprocessing and delivered to whom (e.g. whether also intermediate products shall be delivered) o reprocessing sequence (e.g. first all L1 calibration products chronologically and sequentially orbit by orbit, then L1 science products (L1b, FBR) etc) o which time periods shall be processed first (e.g. July-Aug 2010 CryoVex Arctic) o SPPA will perform data quality management - If available, recommendations on parallelisation of processor instances and concurrent execution capabilities (possibly part of tailored ICD) - Operations Manual Installation Manual Software Release Note (highlighting implemented changes and PR fixes with respect to previous s/w version) - Procedure to raise PRs and CRs on the processing system to ESA/the Contractor responsible for the maintenance of the processing system (to be documented as part of a Service Interface Document between the two organisations) - User Manual - Orchestration model Performance and scalability requirements Sample data to be processed including sample job order files and test scenarios (which may be provided together with the Processing System or already be stored in the Contractor’s repository) Sample output products provided with the processor software Note: depending on the nature of the processing system (processor, format converter, QC tool, etc) the list of inputs may be different or a subset. The above list refers to the case of a processor and is the most complex case. Page 41/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 3.3.5 Outputs Operational infrastructure / environment for the processing in place Processing System integrated, with orchestrator implemented, verified and technically validated in the operational environment Verification plan Verification test report and test data Problem Reports and Change Requests against the Processing System and accompanying documentation and sample data In case of non-compliances in the Processing System and accompanying material, delta project plan for the integration. Processing system integration report Processed sample data 3.3.6 Requirements 3.3.6.1 Technical Integration [Req INT-SYST-10] Upon request of the ESA Project Manager , the Contractor shall receive / accept / fetch the Processing System (with accompanying documentation and sample data, and other related inputs) from the indicated source, either on-line or through a widespread type of media. The source is in principle ESA itself (or an entity mandated by ESA). The Processing System and related material will be delivered accompanied by a CIDL packaged as a set of Configuration Items. The Contractor is free to adopt the same or a different breakdown in its own Configuration Management system, provided that: 1) the material received is fully kept under CCM, 2) the original identification and version number of the processing system delivered are maintained; 3) CCM is applied to the material received and to any material generated by the Contractor in the scope of this WP 4) at the end of the Integration WP the material is packaged and baselined as part of the Data Information System and CCM DB. Note: the Contractor must under no circumstances make any changes to the processing systems received as CFIs, which are maintained / modified under separate contractual contexts under the supervision of the ESA entity providing them. Therefore no change management (in the sense of change approval cycle and change implementation control) is necessary on the processing systems. Only configuration management and baselining are required as CCM activities on these items. [Req INT-SYST-20] The Contractor shall place the material received under its own CCM system, in compliance with the approved Data CCM Plan. Page 42/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req INT-SYST-30] The Contractor shall apply the defined CCM process throughout the WP, covering the material received from external sources as well as any material generated by the Contractor itself, in compliance with the approved Data CCM plan. [Req INT-SYST-40] The Contractor shall deploy the operational environment necessary to run and operate the Processing System with the required performance and scalability and without impact on the performance of other concurrent processing projects possibly sharing the same infrastructure. [Req INT-SYST-50] If a management/orchestration layer is required as part of the processing (case of a processor), the Contractor shall select / procure / implement and deploy as part of the processing infrastructure a suitable management/orchestration layer to control, orchestrate and monitor the processor in compliance with the specified performance and scalability requirements. [Req INT-SYST-60] The infrastructure and operational environment deployed by the Contractor shall allow to cope with a rapid increase and high peaks in the processing demand in line with the scalability requirements, e.g. by allowing dynamic allocation and re-allocation of resources. [Req INT-SYST-70] The Contractor shall deploy and maintain the operational environment in compliance with the performance requirements specified as input to the WP, namely: - project duration, corresponding to the time to make the infrastructure available (including installation and integration of processor and V&V activities) - due delivery date of the processed data and consequent necessary throughput (number of times real-time) of processing - volume of data which will need to be processed - duration of the availability of the processing capability at the required throughput of processing (to cover the processing campaign required) - subsequent duration of the availability of the processing capability with reduced (10%) performance (to cover any additional processing – see section 3.3.6.5). [Req INT-SYST-90] The Contractor shall integrate the Processing System into the operational environment in a way that ensures: - the use of the various types of input data according to the orchestration model provided as part of the Processing System documentation - the compliance with the specified performance/throughput and scalability requirements - the compliance with the processing strategy and processing sequence specified in the input documentation (taking into full account and balancing any dependencies/processing constraints of the processing system). [Req INT-SYST-100] In case of non-compliances in the Processing System provided including the accompanying documentation and data, the Contractor shall generate the corresponding Problem Reports (PR) and deliver them to the designated interface responsible for the maintenance of the processing system. Note: the designated interface may be the ESA Contractor in charge of processing system corrective maintenance or the ESA Project Manager or another interface, depending on the specific project. Page 43/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req INT-SYST-110] In case of significant non-compliances in the interfaces/documentation/operation of the Processing System provided including the accompanying documentation and data, the Contractor shall determine the feasibility of overcoming the non-compliances through technical or procedural work-arounds and document the findings in a delta project plan. Note: the only intervention possible by the Contractor is at the level of technical work-arounds in the operational environment without changes to the Processing System itself in order to avoid any alterations of the data processing performed (and to the data orchestration). [Req INT-SYST-120] In case of approval by the ESA Project Manager of the delta project plan, the Contractor shall put in place and document the necessary work-arounds in order to overcome the non-conformances. [Req INT-SYST-130] In case that the Processing System provided including the accompanying documentation and data, presents opportunity for significant improvements to the operability, the Contractor shall generate the corresponding Change Requests (CR) and deliver them to the designated interface. [Req INT-SYST-140] The Contractor shall deliver the CR and PR to the designated interface in electronic format according to a standard template. Note: the CR and PR generated by the Contractor will be considered for implementation. However there is no guarantee whatsoever that they will be implemented, and in case they are implemented there is no guaranteed release time for the upgrades. The Contractor is expected to run the processing system despite any imperfections that it may present. Only demonstrated major(blocking) anomalies of the processing system will be accepted as a basis to revise the project schedule and/or outputs. [Req INT-SYST-150] The Contractor shall, on request of the ESA Project Manager , liaise with the designated interface for the resolution of the PR and CR submitted. [Req INT-SYST-160] The Contractor shall document in the Processing System Integration Report the technical aspects of the Processing System in the operational environment and in particular: - the CR and PR submitted to the designated interface, and the feedback received - the data orchestration approach and implementation (including selection of what primary and auxiliary data files are processed together) - the performance - the scalability of the system and capabilities for concurrent execution and dynamic allocation of resources. 3.3.6.2 Update of the data storage and Data Information System and CCM DB For reasons of traceability and in order to allow any required troubleshooting of the data resulting from the use of the processing system as integrated by the Contractor, it is fundamental to store baselines of the processing environment. Page 44/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req INT-CCM-10] The Contractor shall keep under formal configuration management the Processing System as integrated in the operational environment, including the accompanying documentation, sample/test data, installation/integration records including technical specifications of the infrastructure deployed, and other related material, to the extent necessary to guarantee: - the ability to reconstitute demonstrably the processing environment used to generate a resulting dataset, in order to support troubleshooting or justify the correctness of the data produced; - the ability to restore the operational capability of the Processing System within the allowed time, with the required performance, for the required length of time (eg in order to perform an additional processing campaign on the same processing system). [Req INT-CCM-20] The Contractor shall store the processing system including SW, documentation, test data and all other CI in the Data Information System and CCM DB: - as a baseline at the end of the integration WP and - incrementally at each subsequent update of the processing system (see Follow-up activities). 3.3.6.3 Verification of the Integration activities [Req INT-VERIF-10] The Contractor shall submit to the ESA Project Manager the Verification portion of the Processing System integration Verification and Validation plan. In the case of test scenarios and specific test data provided by ESA together with the processing system, the Contractor shall include the provided test scenarios and use of the test data into the verification portion of the Processing System integration Verification and Validation plan. Note: for brevity, the Verification portion of the Verification and Validation plan is also referred to as the Verification plan. It is however not a separate deliverable but an integral part of the V&V Plan deliverable. [Req INT-VERIF-20] The Contractor shall include in the Processing System integration Verification Plan the comparison of sample output products delivered together with the processing system with products generated by the integrated processing system, and the analysis of any differences. Note: in the nominal situation (correct integration of the processor) the outputs resulting from processing the sample inputs may not be fully identical to the corresponding sample outputs provided by ESA (eg due to different processing date / times). The verification report is required to include an analysis of the differences and exclude justifiably any cause due to incorrect integration. The final assessment will be performed by SPPA as part of the validation process. [Req INT-VERIF-30] The Contractor shall execute the Processing System integration Verification Plan and document the outcomes in the Verification portion of the Processing System integration Verification and Validation report. Note: for brevity, the Verification portion of the Verification and Validation report is also referred to as the Verification report. It is however not a separate deliverable but an integral part of the V&V Report deliverable. Page 45/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req INT-VERIF-40] The Contractor shall run the processing on the required sample/test data and deliver the output to the ESA Project Manager , and to the designated interface. The processed sample data shall be attached as an integral part to the Verification report. The Processing System integration Verification Plan and Report are subject to approval by the ESA Project Manager. [Req INT-VERIF-50] The Contractor shall deliver the output of the processing of the sample/test data (including output of any post-processors and QC tools) either on-line (e.g. through an FTP server) or through any of the most widespread types of media, as specified by the ESA Project Manager. Note: the delivery of the output of the processing of the sample/test data is an integral part of this WP. However it follows the same approach and requirements described in WP Data Repatriation. For further details refer to the description of that WP. Note: Depending on the nature of the Processing System , the output of the processing may also include output file listing, working directory composed of the Job Order, diagnostic log files, L0, L1, L2, auxiliary data used and size of products generated. After the successful execution of the Verification plan, the Data Release Management process shall be activated. 3.3.6.4 Processing System Integration guarantee and follow-up The activities of Processing System Integration as described in this WP are treated on a project basis. The project terminates with the completion of all the activities required, based on the material delivered to the Contractor as input. The completion of the project is marked by the successful completion of the Verification and of the Validation (see Chapter “Release Management”). However, there is a significant likelihood that, following the analysis of the output of the processing, the owner of the Processing System will produce updates in order to improve the scientific behaviour and/or implement some of the CR / PR produced by the Contractor during the integration or by ESA and/or make other improvements. In addition, despite the care taken in the Verification and Validation exercises, ESA or the Contractor may discover, either autonomously or following notification of further recipients (eg PACs / LTDP Centres) of the data processed following the integration exercise, technical problems in the outcome of the activities performed in the course of the project. [Req INT-GUA-10] The Contractor shall continue to perform /repeat the project activities on an incremental basis to cover cases of: a- any remaining errors in the Processing System integration by the Contractor b- the possible delivery of patches of the processing system. Page 46/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req INT-GUA-20] Upon request of the Project Manager or the owner of the Processing System (normally SPPA in the case of this requirement), the Contractor shall perform a “delta loop” of integration consisting in: - receive / accept / fetch updates/patches of the Processing System (with accompanying documentation and sample data) either on-line or through any of the most widespread types of media; - install the updates of the Processing System in the operational environment; - perform adequate regression testing of the integration including running the processing on the required sample data and test scenarios and deliver the output to the ESA Project Manager or directly to SPPA - deliver the output of the processing either on-line or through any of the most widespread types of media, as specified by the ESA Project Manager. Depending on the circumstances, there may be several delta loops of integration after the main integration activity. [Req INT-GUA-30] Any updates of the Processing System (case b above) which require no evolution of the infrastructure and consist in the mere replacement / addition of CI’s of the Processing System (eg with no data migration or other additional activities) shall be considered updates/patches and therefore fall under the guarantee and follow-up activities, up to a yearly amount of 3 rounds of regression testing. [Req INTL-GUA-40] The guarantee and follow-up period shall last for 1 year after the completion of the integration project (or until the end of the Contract). The guarantee and follow-up activities are an integral part of the project from the funding point of view, ie no additional cost will be paid by ESA for these activities. However from the operational point of view they will be carried out as continuous activities managed under the overall service and not any more as project activities. 3.3.6.5 Continued availability of the processing capability Once the integration of the processing system is complete, the Contractor is required to maintain the processing environment operational availability for a period of time specified in the individual project requirements. Typically, there will be a first period (for the main processing exercise) with full capacity/performance followed by a longer period of reduced (10%) capacity/performance. [Req INT-CONT-10] The Contractor shall maintain the operational capability to process additional data with a reduced (10%) level of performance as specified in the project requirements, and in any case for no less than 6 months after the completion of a processing project. [Req INT-CONT-20] After the end of the subsequent period of availability required for additional processing, the Contractor shall make the processing capability available again upon request within: - 2 weeks (reduced performance, 10% of full performance) - 4 weeks (full performance as defined for the original integration). Page 47/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req INT-CONT-30] In case it is required to restore the operational capability of the Processing System after the Contractor has made any changes to the operational environment (eg to redeploy the computing resources after a period of inactivity of the Processing System), the Contractor shall demonstrate the continued validity of the operational environment deployed (including for performance) either by redeploying the exact same original environment or by demonstrating the equivalence, including by a round of regression testing. [Req INT-CONT-40] In case it is required to restore the operational capability of the Processing System after the supplier of the processing system has delivered upgrades/patches, the Contractor shal install the upgrades/patches and perform adequate regression testing of the integration including running the processing on the required sample data and test scenarios and deliver the output to the ESA Project Manager or directly to SPPA. 3.4 Bulk processing / reprocessing/ reformatting operations 3.4.1 Description The purpose of this WP is to execute reprocessing, bulk processing and re-formatting operations on designated EO datasets in response to customer needs. One major aspect of ESA’s objectives is to provide high quality while guaranteeing a fast turn-around in (re)processing campaigns. This includes operating ESA approved scientific processors and software applications for quality control and data format conversion, provided by ESA as CFI and integrated by the Contractor into its operational processing infrastructure. The specific parameters (data, processing system identification, time window etc) of each processing project will be indicated at the time of requiring the project proposal. See Annex 2 for the specific parameters of the initial basket of projects. The ESA-defined strategy for processing may vary across processing projects. In some cases, selected geographical areas and data acquisition time windows will need to be processed with priority, in other cases processing will be done from beginning of mission to end of mission or the other way around. Note: the word “processing” refers generically to an elaboration of data in form of re-processing, bulkprocessing or reformatting. The processing system may consist in a single process or in a combination of pre-processing, processing, post-processing, QC processes. Page 48/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - Pre-processing steps might be needed to prepare input data to the format actually requested by the processing system The processor is in charge of the main processing Post-Processing steps might be applied to the output data to apply a naming convention, a format conversion or to generate quick-looks and other metadata from the output products A basic Quality Control step is generally part of the Processing task, integrated within the post processor functionality or based on a separate s/w application which performs basic quality control on the formal validity of the produced data, i.e. cloud coverage, naming convention, data format, congruency between input data files and output ones, etc. A further aspect of Quality Control is performed outside the scope of this contract with the scientific validation of the output data by SPPA (supported by external organisations). The output of the scientific validation is a go/no-go decision by SPPA whether the dataset produced can be put at the disposal of end users. The scientific Quality Control step may trigger a further activation of the processing tasks, as issues detected in the output products may result in an updated processor s/w to be integrated and to be operated or different input data to be used. The processing system (including any QC tools) will be developed as part of separate contractual arrangements and delivered to the Contractor as CFI. The Contractor shall integrate the processing system into its own processing infrastructure (see WP on ‘Integration of Processing Systems’) and execute the processing in its own infrastructure as part of this WP. The technical validation of the processing chain is part of the integration WP. The Scientific validation of the output data is performed by SPPA or a third party designated by SPPA. The Contractor supports the quality assessment of processed datasets by enabling data accessibility as part of WP data repatriation and delivery. Processing projects are, like all projects in scope of this contract, initiated by the ESA Technical Officer or Project Manager on behalf of the Customers for these projects. In the case of processing done for the purpose of a scientific validation of a processor, the trigger for processing operations, including the designation of what input data should be processed, the start and stop of operations, and the analysis of the resulting output data, will be delegated to SPPA (and/or organisations tasked by SPPA for processor development and validation) who will interact directly with the Contractor. Page 49/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use This WP requires as pre-requisites: the completion of tasks specified under WP ‘Integration of Processing Systems’ namely the successful integration of the processing system into the Contractor’s processing environment; the completion of the tasks specified under WP ‘Data collection’; the completion of tasks specified in WP ‘Data Consolidation’ (unless a dataset is considered by ESA to be ready for processing without going through this WP). 3.4.2 Scope In Scope of this WP are reprocessing, bulk processing or re-formatting operations executed on ESA EO datasets using ESA supplied tools (including quality control tools supplied by ESA to facilitate the task of data quality assessment of a processed dataset). The processing in scope of the WP includes one or more rounds of processing of the data to generate L1 and L2 but also the repetition of the processing on part (up to 20%) of the same input data in order to support the scientific validation of the processor. Included into the WP is the delivery of the reprocessed, bulk-processed and/or reformatted dataset to SPPA or other ESA specified entities for quality control and verification or other purposes. The output of the processing (including reprocessed, bulk-processed data and reformatted datasets together with metadata, quick-looks and QC reports as applicable) is delivered to its destination as part of the Data repatriation WP. Note: ESA may be willing to convert L0 data into SAFE format in the scope of this contract, without providing a SAFE generator as a CFI. In this case, ESA will provide the full SAFE format specification and ask the Contractor to develop and deliver the SAFE generator. A specific contractual arrangement will be made for this purpose. 3.4.3 Activities The activities required include: Preparing the processing and readiness review Executing Test processing and planning of the operational processing Executing the operational processing of the required dataset Monitoring the processing for performance, processing failures, production of statistics Verifying the outputs of the processing Facilitating the Scientific Validation by providing (access to) output data including log files Updating the data store and CCM DB / Data Information System Page 50/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 3.4.4 Inputs Project parameters of the processing to be performed (see Annex 2); Contact points for the third parties designated by SPPA for the maintenance/ anomaly handling/configuration management of the Processor and QC tool and for the Scientific validation of the data; Processor documentation including Processing Strategy document (including sequence of operations, priorities etc.); Processing System Operations Manual/user manual, and Operational Procedures and any other available documentation; Contractor-supplied processing environment integrated with the applicable ESA processors and/or QC tool and/or format conversion tools All suitable input data (in particular master data sets) collected, consolidated and ingested within the operational processing environment (ingestion report) of the Contractor, Final Transcription Report of the data (all data types) as ingested into Contractor’s operational environment 3.4.5 Outputs Reprocessed, bulk-processed or re-formatted datasets, metadata, quick-looks associated to the data products having passed technical verification and validation and ready for scientific validation by external entities. In the case of QC tools being run together with the processor, outputs of the QC tools. Regular deliveries of the Processing Technical Note documenting the various steps, the progress and performance, etc. During processing operations, regular (weekly) statistics and reports on production progress 3.4.6 Requirements 3.4.6.1 Preparation of the processing and readiness review Prior to the start of a processing, a readiness review of the input data and processing infrastructure are mandatory. Readiness review of input data: The processing will generally be preceded by a consolidation exercise. In this case, the master dataset will normally be the output of the consolidation exercise. If not, the preparation / definition of the a master dataset will be done as part of this WP. Page 51/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use In any case, the processing is always preceded by a data collection WP. [Req PRO-PREP-10] The Contractor shall prepare the dataset to be processed, by selecting the master input datasets from the service storage for the primary data as well as the auxiliary data needed for the processing. [Req PRO-PREP-20] In case there is no identified master dataset for some or all of the input data available in the storage area, or if the master dataset does not cover the full spectrum (eg time coverage) required for the processing, the Contractor shall propose an approach for selecting the most suitable dataset (primary and auxiliary) for the processing, and document the proposal and its rationale in the Processing Technical Note. [Req PRO-PREP-30] The Contractor shall liaise with the ESA Project Manager and/or DSI-CCB to confirm the selection of the input data sets (primary and auxiliary). [Req PRO-PREP-40] In case any of the datasets (primary or auxiliary) selected as input is not already marked as the master dataset, the Contractor shall liaise with the ESA Project Manager, and/or DSI-CCB to select the appropriate course of action among: confirm the designation of the master dataset; stop the processing project until a master dataset is designated or made available; any other appropriate course of action. [Req PRO-PREP-50] The Contractor shall review ESA’s processing strategy document (if available) and ensure that the project plan, input data and processing infrastructure are prepared in compliance to requested processing strategy, sequence and priority. [Req PRO-PREP-60] The Contractor shall document the selection of the input data, processing strategy and related decisions in the Data Information System and CCM DB as well as the Processing Technical Note. Readiness review of processing infrastructure: The processing project will normally be preceded by a project to integrate the processing system to be used, culminating in the verification and validation of the integration. However the readiness of the processing system needs to be verified before the start of the processing for various reasons eg this processing exercise may be taking place a long time after the successful integration of the processing system and the initial processing exercise. The ESA Project Manager may waive the corresponding requirements in case the integration of the processing system has been completed or checked recently. [Req PRO-PREP-70] The Contractor shall make sure that the readiness of the processing environment is assured, by proposing and performing regression tests commensurate with the current status of the processing environment compared to the status at the time of validating the integration of the processing system (see WP Integration of Processing Systems). [Req PRO-PREP-80] The Contractor shall make sure as part of the readiness of the processing system that the end-to-end performance is in line with the project requirements. Page 52/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req PRO-PREP-90] The Contractor shall tune/customise the operational procedures delivered with the processing system, according to: - the most suitable processing strategy for the specific project - the specificities of the Contractor’s infrastructure in compliance to the Project’s performance and quality requirements. [Req PRO-PREP-100] The Contractor shall document in the Processing Technical Note the readiness check of the processing system. [Req PRO-PREP-110] In case the readiness of the processing system is no longer assured, the Contractor shall prepare, document in the Processing Technical Note and execute a plan to establish / restore the readiness of the processing system. [Req PRO-PREP-120] The Contractor shall ensure that the Project Proposal and Project Plan take into account any necessary plan to establish / restore the readiness of the processing system. [Req PRO-PREP-130] The Contractor shall include in the Preparation chapter of the Processing Technical Note a recapitulative list of criteria referred to the data and the processing system and the status for each criterion. The Preparation chapter of the Processing Technical Note, including the list of criteria and the status of each criterion, will be subject to the approval of the ESA Project Manager. [Req PRO-PREP-140] The Contractor shall start the operations specified within this WP only after the readiness criteria are all confirmed (or alternatively waived) by the ESA Project Manager. 3.4.6.2 Execution of the processing The purpose of the processing may be the finalisation of the scientific validation of the processor, or the production of the final output data. In the former case, the originator of the processing is typically SPPA, however for the day-to-day operations (perform a subset of processing, deliver the output…) the Contractor may be in touch directly with the entity (eg QWG, ESL) in charge of the scientific tasks on behalf of SPPA. In this case, the planning of the processing is driven by the scientific analysis of the outputs of the processing. The scientific validation of the processor requires several loops of tests before the final, operational processing. According to the strategy adopted for the validation, the test loops may cover all the products over a specific area, all the data acquired over a specific time window, the data acquired in a limited time window repeated over the cycles of the mission, etc. In the case of the production of the final output data, the initiator of the processing is the ESA Project Manager , who will liaise with SPPA. SPPA will continue to check the output data from the scientific point of Page 53/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use view and may require alterations in the processing plan and/or the processor itself, leading to a repeat of part of the processing. [Req PRO-EXEC-10] Upon request, the Contractor shall liaise with SPPA or a third party designated by SPPA, for: - the definition of the subsets of data to be processed as part of the test loops (e.g. input data provided with processor software or data to be extracted from Contractor’s storage area); - the start / stop of the processing; and perform the rounds of processing accordingly. [Req PRO-EXEC-20] The Contractor shall perform the required processing on the dataset(s) specified, in compliance with: - the project requirements, including processing speed/throughput and output product characteristics,; - Processing strategy and sequence agreed with the ESA Project Manager and documented in the Processing Technical Note; - the finalised operating procedures of the processing system; - the orchestration model of the processor; - the operating procedures and operational scenario of the Quality Control tool, if such tool is to be operated. In the case of a reprocessing or bulk processing project, the typical outputs of the processing will be higher level output products, quick looks, meta-data associated to the products (e.g. cloud cover votes) , and Quality Control reports (in case that QC tools are run as part of the processing). [Req PRO-EXEC-30] In the case that a Consolidation project has been performed by the Contractor prior to the Processing project, the Contractor shall consider the outcome of the processing in order to fine tune the consolidation. 3.4.6.3 Monitoring of the processing The Contractor is fully responsible for the technical correctness of the processing including the detection and correction of failures in: - the infrastructure and its performance; - the technical compliance of the outputs to the specifications (eg number, format, length, correctness of the choice of files and orchestration method, etc) documented in the Operating documentation of the processing system. [Req PRO-MONIT-10] The Contractor shall ensure that the application of the operational procedure and the monitoring of the operational processing detect 100% of the operational and technical failures in terms of violations of the Operating documentation, and shall repeat the corresponding steps until successful production of the expected outputs. [Req PRO-MONIT-20] In the case of technical failure of the processing (failure due to a technical error in the processing system provided as CFI), the Contractor shall: Page 54/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use raise within 1 NWD of occurrence any operational incidents due to the processing systems supplied by ESA as CFIs to the designated interface responsible for the maintenance of the processing system (as defined in WP Integration of Processing Systems); liaise with the designated interface for the resolution of the problems in question; keep the ESA Project Manager and SPPA fully informed of the occurrence of such failures and corresponding correction by the designated interface. [Req PRO-MONIT-21] In the case that the failure is recoverable by the Contractor, the Contractor shall correct the error situation and repeat the failed processing. This includes eg situations like: - error by the Contractor in the execution of the procedures; - error in the orchestration implementation; - missing input data; - failure of the processor which can be overcome by a workaround; - etc. [Req PRO-MONIT-22] The Contractor shall perform as part of the standard project cost: - any repeat processing due to errors traceable to the Contractor’s operations; - repeat processing up to 20% of the initial amount of data due to processor upgrades (eg due to the scientific validation) and/or errors traceable to the ESA-provided CFI and/or additional material collected by the Contractor with the agreement of ESA. [Req PRO-MONIT-30] Through the design of the operational procedure and infrastructure, the Contractor shall ensure that the throughput of the processing chain (including any repeats due to operational incidents) is compliant to ESA performance requirements and the milestone required in the Processing Project Requirements. [Req PRO-MONIT-40] The Contractor shall monitor closely the processing performance, and in case of performance degradation and/or risk of slippage in processing speed/project schedule shall: 1) Immediately alert the ESA Project Manager ; 2) Draft a plan of corrective action including the restoration of the necessary speed / performance and the recuperation of the backlog; 3) Submit the plan of corrective action to the approval of ESA Project Manager ; 4) Execute the plan as approved; 5) Prepare an internal (Contractor) review and perform a Root Cause Analysis for the loss of performance; 6) Inject the corresponding outputs into the lessons learnt document of the project; 7) Ensure that all subsequent projects follow the lessons learnt. [Req PRO-MONIT-50] The Contractor shall include the reports, logs generated by the ESA processing system (in particular in the case of a processor) and the ESA QC tool as a means to closely monitor the production and to detect any processing failures/erroneous scenes. [Req PRO-MONIT-60] In case of erroneous scenes and/or problems flagged by the processing system or QC tool and traceable to defects in the processing system, the Contractor shall generate the Page 55/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use corresponding problem reports (PR) and deliver them to the designated interface responsible for the maintenance of the processing system. [Req PRO-MONIT-70] In addition to the releases of the Processing Technical Note , the Contractor shall release weekly or fortnightly (as agreed with the ESA Project Manager ) production statistics to the ESA Project Manager , including the global re-processing or bulk-processing status according to the required overall processing time window, total number of scenes/orbits/data cycles processed successfully, number of scenes/orbits/data cycles processed since last report, number of processing failures and reason of failure (scenes/orbits not processed or flagged erroneous). The progress should preferably be displayed graphically. [Req PRO-MONIT-80] Together with the weekly or fortnightly release of the production statistics, the Contractor shall release updated maps of location / density of processed scenes as applicable (eg for optical data). 3.4.6.4 Facilitation of the Scientific Validation A key aspect for the scientific validation is that SPPA (and the entities supporting SPPA) have rapid access to the processed data. [Req PRO-SSCI-10] The Contractor shall facilitate the access to the data for SPPA or other initiator of the processing in compliance with WP Data Repatriation/delivery, including the delivery of data in slices to allow the early scientific validation of the outputs. Note: access to the data by SPPA and related organisations shall be granted throughout the project for any checks that SPPA decides to perform. [Req PRO-SSCI-20] The Contractor shall give assistance to SPPA and related organisations on the output data of the processing by providing knowledge about the input data and processing exercise on an advisory basis. As the outputs of the test loops are analysed, the initiator of the processing will provide either upgraded versions of the processor or modified operating instructions (including on the selection of the input data). [Req PRO-SSCI-30] The Contractor shall collect the feedback from the scientific validation, document the changes required in the Processing Technical Note, and implement the required changes. In particular, the Contractor shall carry out any installation of ESA CFI s/w patches during the operations phase while minimising the impact on the project schedule. Note: see also the section on Processing System Integration Guarantee and Follow-up. Page 56/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 3.4.6.5 Update of the data storage and Data Information System and CCM DB [Req PRO-STOR-10] The Contractor shall store the processing system including SW, documentation, test data and all other CI in the Data Information System and CCM DB as baselined at the start and at the end of the Processing WP. [Req PRO-STOR-20] The Contractor shall store in the Service storage and keep under CM the data and other outputs resulting from the processing. [Req PRO-STOR-30] The Contractor shall assign to any newly produced dataset a new and unique version number and store the dataset in the CCM DB accordingly. [Req PRO-STOR-40] The Contractor shall keep at all times the CCM DB / Data Information System up to date by recording the operations performed as part of the processing and the resulting evolution of the data information, in particular the designation of master dataset. [Req PRO-STOR-50] The completion of the processing exercise and the designation of the master dataset shall be considered major events and as such the Contractor shall publish them as part of the Knowledge Management WP. 3.4.6.6 Verification of the Processing activities The verification of the Processing activities include, beyond the monitoring performed throughout the processing, the reconciliation of the output data with the input data. Note: the scope of the verification and validation of the processing activities in scope of this SoW is the correct operation of the processing system and production of the output data. The scientific validity of the output is taken care of by SPPA and related organisations (supported by the Contractor as explained above) and is not part of the scope of this SoW. [Req PRO-REC-10] Once the processing is complete and by making full use of the detailed processing statistics collected throughout the project, the Contractor shall perform a reconciliation process between the input data and output data, including: - each input file shall be matched to zero, one or more identified output file(s), and vice-versa (e.g. match L1 product to corresponding input L0 orbit); the matching patterns to be expected will in principle be provided by SPPA, otherwise the Contractor shall propose the patterns; - the reason for any mismatch between input and output files shall be determined; - any mismatch between input and output files shall be eliminated. In case the Contractor is not in a position, for reasons outside its control, to eliminate some of the mismatches, the ESA Project Manager in consultation with SPPA may waive this requirement. Note: this process can also be performed in slices in order to anticipate the detection of errors. [Req PROC-REC-20] The Contractor shall submit to the ESA Project Manager the Processing verification portion of the Verification and Validation plan, including the activities of reconciliation. Page 57/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Note: for brevity, the Verification portion of the Verification and Validation plan is also referred to as the Verification plan. It is however not a separate deliverable but an integral part of the V&V Plan deliverable. [Req PROC-REC-30] The Contractor shall execute the Processing Verification Plan and document the outcomes in the Verification portion of the Processing Verification and Validation report, including the outcome of the reconciliation process with the list of mismatches and the explanation and resolution of each, in the Processing Technical Note. Note: for brevity, the Verification portion of the Verification and Validation report is also referred to as the Verification report. It is however not a separate deliverable but an integral part of the V&V Report deliverable. The Verification plan and report will be subject to the approval of the ESA Project Manager. [Req PRO-REC-40] Once the processing and reconciliation are complete, the Contractor shall deliver (as part of the Processing Technical Note) a final report on the completeness of the processed output dataset to the ESA Project Manager , based on the completeness of the input data and the outcome of the reconciliation. As a result of reconciliation, the resulting (higher level) dataset inherits the attribute “master” from the input (lower level) dataset and can in turn be used to produce yet higher level data (e.g. Level 2): the attribute “master” propagates up the stages of processing through the reconciliation exercise. This process is however not automatic and will require confirmation by SPPA through the channel of the DSI-CCB. [Req PRO-REC-50] The Contractor shall seek, capture and document the assignment by SPPA of the “master” label for the output dataset through the DSI-CCB. After the successful execution of the Verification plan, the Data Release Management process shall be activated. 3.4.6.7 Data Processing guarantee and follow-up The activities of data processing as described in this WP are treated on a project basis. The project terminates with the completion of all the activities required, based on the data and processing system delivered to the Contractor as input. The completion of the project is marked by the successful completion of the Verification and of the Validation (see Chapter “Release Management”). However, there is a significant likelihood that additional data which is part of the same mission / instrument / time and geographical coverage as the input data will become available for processing after the project activities are underway including after the completion of the project. This may come about in particular through the follow-up activities of data collection and data consolidation. Page 58/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use In addition, despite the care taken in the Verification and Validation exercises, ESA may discover, either autonomously or following notification of further recipients of the data (eg PACs / LTDP Centres) inaccuracies in the outcome of the activities performed in the course of the project. [Req PRO-GUA-10] The Contractor shall continue to perform /repeat the project activities on an incremental basis to cover cases of: a- errors and omissions made by the Contractor in the Project b- the possible discovery of additional input data (eg for gap filling). [Req PRO-GUA-20] The additional processing of input data (case b above) shall be considered incremental (and therefore fall under the guarantee and follow-up activities) up to an amount of 3% of the original data per year. Beyond this threshold, further activities will be considered as a new project. [Req PRO-GUA-30] The guarantee and follow-up period shall last for 1 year after the completion of the project (or until the end of the Contract). The guarantee and follow-up activities are an integral part of the project from the funding point of view, ie no additional cost will be paid by ESA for these activities. However from the operational point of view they will be carried out as continuous activities managed under the overall service and not any more as project activities. 4 SUPPORT SERVICES 4.1 Managerial and technical approach The technical and management approach to this contract should emphasise the following aspects: 4.1.1 Overall responsibility The Contractor shall be fully responsible for the fulfilment of the requirements of this SoW for services and projects, regardless of the choice of infrastructure and of the organisation of the consortium. 4.1.2 Customer and User orientation The successful delivery of data is a major driver of this procurement and must be emphasised and maximised in the Management of this Service. 4.1.3 Risk contention The innovative aspect of the management of this Service must be accompanied by the careful reduction and mitigation of all risks in order to maintain stakeholders’ confidence in the validity of this model. Page 59/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 4.1.4 Cooperation The execution of this Service requires effective cooperation with Interfacing Services -see Annex 4. A major driver in the successful provision of the Service is a positive and cooperative spirit when dealing with all stakeholders. In addition, the Contractor is required to manage and orchestrate all the various services and WPs in a way that maximises the effectiveness and efficiency of the overall service and the fulfilment of the Service Objectives. In particular the Contractor must make sure that there is synergy and cooperation among the various services and WPs for communication of information and provision of inputs. 4.1.5 Autonomy and pro-activeness The Contractor must be fully autonomous in fulfilling its tasks and reaching the Service objectives. In particular, there will be no contribution / participation of ESA in the setup, operations and support of the technical platform, beyond the provision of requirements through this SoW and the review of the Contractor’s outputs. 4.1.6 Skills, know-how and experience required The profiles required for the Contractor team in the key roles (as defined in Annex 4) are: - strong customer orientation and determination to make a joint success out of a challenging service approach; - no less than 10 years of experience in service and project management, including the coordination and organization of diverse teams in complex organisations; - strong experience of at least 5 years in the technical field of EO data management and processing; - significant experience at least 3 years in the definition and management of KPI’s and service metrics, and in the implementation, monitoring, analysis and improvement of services according to quality parameters; - familiarity with software engineering standards (ISO 12207 or equivalent); - familiarity with project management standards (Prince2 or equivalent); - familiarity with service delivery best practices (ITIL); - presentation and communication skills. The Contractor’s team is required to show: - customer orientation, problem solving orientation and can-do attitude; - skills and experience in the technical aspects linked to the datasets in scope of the service, and desirably beyond; - skills and experience in the modern infrastructure management approaches which allow the flexible deployment of storage and processing power (eg cloud technology) and the related technical challenges (security, networking, etc); - skills and experience in integrating heterogeneous software components in complex environments; - skills and experience in configuration management. Page 60/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use In all cases, a strong operations- and customer-orientation is necessary. 4.2 Data Repatriation/delivery 4.2.1 Description The purpose of this WP is to repatriate/deliver the data in scope of this Contract together with accompanying metadata and mission documentation to various recipients including: ESA internal recipients, in particular o SPPA for processor loop testing, data quality control and advanced data consolidation, etc. o DISS-UNIT for dissemination of the datasets to end –users (external to ESA) o OPS-UNIT for archival Other recipients, e.g. ESA dissemination facility, PAC’s, LTDP centres, Satellite mission operators and national space agencies (e.g. USGS, NASA), etc. Activities of data repatriation/delivery are systematically triggered upon completion of the core WPs, including Data Collection, Data Consolidation and Data Processing. In all cases, the data after elaboration is repatriated to ESA and related archiving centres (PACs / LTDP centres). Moreover, specific data repatriation/delivery activities are required in case of : WP data consolidation: Datasets resulting from project based activities within this WP are distributed to SPPA (and/or supporting organisations) for advanced consolidation involving in-depth scientific expertise; WP bulk-processing/reprocessing/reformatting operations: Dataset resulting from processing/reformatting projects, including datasets processed for the purpose of processor loop testing, are distributed to SPPA (and/or supporting organisations) for scientific data quality control. In case of output datasets being processed as part of nominal reprocessing/bulk-processing projects, they are delivered to SPPA (and/or supporting organisations) for further scientific quality control and for assigning the ESA quality stamp and to DISS-UNIT for implementing the proper mechanism to deliver the data to end-users (once the ESA quality stamp is available). 4.2.2 Scope Entire or large portions of the datasets are either distributed on HDD or are made accessible on-line via an on-line data repository to be set-up and maintained in scope of this Contract. Note: in the context of data repatriation/ delivery, the term HDD is used to represent a fast access memory device and covers NAS’es as well as Hard Disks. Data is also delivered incrementally on-line (and in exceptional cases via HDDs), i.e. small portions of a dataset are delivered, for various uses. Page 61/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Any activities in scope of this WP are triggered/activated with the Contractor on-request of the ESA Technical Officer. The default scope of this WP includes the delivery of data on-line (via FTP/SFTP/FTPS) or off-line via HDD to a limited population (as described above) of recipients including organisation units of ESA and its partners. Data dissemination to the end-users of the data (ie scientists) are not included in the default scope. However, it may be required occasionally and for a limited period of time to disseminate data to a large population of users. This will be handled and funded in a separate context. In principle, datasets distributed within this service: o either have already been validated by ESA through SPPA and are considered conformant to ESA data quality standards (i.e. are quality stamped). In this case, data sets are authorised for release to other ESA facilities and to users external to ESA (managed though DISS-UNIT) , o or are distributed only to specified/authorised expert entities (internal ESA users) for the purpose of data consolidation and data validation/quality stamping or processor loop testing. However, any subset of the data may be required for extraction and/or repatriation/delivery, for whatever recipient and purpose. The amount of data to be repatriated/delivered yearly through this WP is estimated to be: - 2 to 3 times the combined amount of data resulting from data collection, data consolidations and processing projects in scope of the contract (not including the backup data covered in Chapter Service Continuity). 4.2.3 Activities The activities include: For data distribution on HDD - record destination and shipment details in Contractor’s information System - copy ESA EO data on HDD - label media and prepare accompanying inventory - ship to destination For on-line data accessibility - perform basic user management and ensure that any user is registered and thus authorised to enter the Contractor’s data repository - setup the on-line data repository and load the data to it - monitor the access and produce / report statistics In all cases: Produce monthly and upon request a data delivery report. Page 62/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 4.2.4 Inputs - Instructions from the ESA Technical Officer to repatriate / deliver a dataset including identification of the data to be repatriated/delivered, method of delivery, recipients, start and end date of availability (in the case of on-line delivery) 4.2.5 Outputs - Data package shipped on HDD or available on-line to the recipient Record of the delivery (shipment or on-line availability) within the Contractor’s Data Information System Acknowledgement of receipt of the media by the recipient Data repatriation/delivery report (as part of the Service Monitoring Report) including detailed regular statistics on data downloads from the Contractor’s data repository (who downloaded which data when) 4.2.6 Requirements 4.2.6.1 Common requirements Data repatriation/delivery will be requested: - at the end of a data elaboration project (collection, consolidation, processing); - in the course of a data elaboration project, to deliver data gradually in a “continuous cycle” approach (as opposed to strict waterfall) and reduce the time-to-destination of the data; - on a time-driven (eg quarterly) or event-driven (eg specific dataset update) basis following the follow-up activities of the various WPs; - depending on the needs of the recipients, the agreements between ESA and Third Parties, and other factors inducing the need for ESA to repatriate data. HDD-based repatriation/delivery is intended for large amounts of data (a few 100’s Gb to 20 TB) while online access is intended for small to medium amounts (up to a few 100’s Gb).In case very large (above 20TB) amounts of data need to be repatriated / delivered, the medium will normally be LTO tapes. However it may occasionally be required to repatriate/deliver small data amounts of data on HDD and large amounts of data on-line. [Req DISS-COMM-10] The Contractor shall handle requests to repatriate and deliver data (incremental amounts of data and larger portions) as a specific category of issues in the management system. [Req DISS-COMM-20] Upon request of the ESA Technical Officer, the Contractor shall repatriate/deliver data according to the specifications of the ESA Technical Officer in terms of: - identification of the data to be repatriated/delivered; - recipients of the data; - means of repatriation/delivery. Page 63/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The boundaries are indicated in the Scope section. In case the planned boundaries are under- or over-shot significantly, an adjustment of the financial conditions of this WP will be considered. [Req DISS-COMM-30] As part the data repatriation / delivery, the Contractor shall extract from its storage and deliver, unless otherwise specified by the ESA Technical Officer: - the science data as specified - the related auxiliary data - the related documentation - the related extracts of the data information system and CCM DB. [Req DISS-COMM-40] The Contractor shall not release any data, documentation or other material or information entrusted or generated as part of this contract to any party without an explicit request or agreement by the ESA Technical Officer, ESA Project Manager or DSI-CCB. The ESA Technical Officer, or their line management may take the responsibility to authorise the release of data. [Req DISS-COMM-50] The Contractor shall flag to the DSI-CCB any request from parties external to the contract for release of data. Note: in principle, the confirmation of L0 data delivery will only be given after a data confidentiality agreement was signed between the user of the data and ESA. Access to L0 primary data will normally be granted only to internal users of SPPA, and supporting organisations. [Req DISS-COMM-60] The Contractor shall, on a monthly basis and upon request of the ESA Technical Officer, produce a data repatriation/delivery report showing detailed statistics on the data repatriated/delivered, the channel, and for data delivered on-line the statistics per user and affiliation. [Req DISS-COMM-70] The repatriation / delivery report shall be embedded in the Service Monitoring Report and is not required as a self-standing deliverable. However the Contractor shall also deliver intermediate releases of the repatriation / delivery report on request independently from the Service Monitoring Report. [Req DISS-COMM-80] The Contractor shall perform Verification activities to ensure the correctness of the repatriation / delivery activities and outputs. [Req DISS-COMM-90] The Contractor shall document: - the principles and the approach for the verification of the repatriation / delivery activities and outputs in the Service Governance Plan; - the verification activities performed for each repatriation / delivery exercise as part of the repatriation/delivery report. ESA will validate the repatriation / delivery activities and outputs on the basis of spot checks. Page 64/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 4.2.6.2 Data Extraction [Req DISS-EXTR-10] Upon request of the ESA Technical Officer, the Contractor shall extract the specified data to HDD. The exact technical specification of the HDD (or possibly other media) will be agreed on a case by case basis between the Contractor and the ESA Technical Officer. [Req DISS-EXTR-20] The Contractor shall extract data to HDD with the throughput specified in the SLA. 4.2.6.3 Data delivery on HDD Data (primary data, auxiliary data, ancillary data, documentation) is repatriated/delivered on HDD primarily to ESA internal users (SPPA and DISS-UNIT) and to ESA partner organisations eg PACs, LTDP centres, other satellite mission operators, national space agencies etc. In exceptional circumstances, data may be disseminated to end users. The Contractor shall procure the needed HDD, copy the data and check the correctness of the copy, prepare a report on the copy operation and ship the HDD to the destination specified by the ESA Technical Officer. Note: in the case that the amount of data to be shipped is small (a few 10 Gb) it may be agreed between the Contractor and the ESA Technical Officer to replace the shipping of HDD with an on-line data transmission mechanism. The details of the mechanism to be used will be agreed case by case. Apart from the media to be used, the same requirements apply to the on-line transmission of data as to the HDD-based transmission. Preparation of the HDD [Req DISS-HDD-10] The Contractor shall procure HDDs according to the total size of the data to be transcribed, following technical specifications agreed with the ESA Technical Officer. [Req DISS-HDD-20] The Contractor shall copy the requested data from its storage to the HDD. [Req DISS-HDD-30] The data on the target HDDs shall be structured as a hierarchical tree according to a structure to be proposed by the Contractor. The default structure shall be where the first path level is the unique source media Configuration Item Id and the remaining path structure reflects the structure of the original media. [Req DISS-HDD-40] The Contractor shall label each target HDD with the following information: - Unique HDD id; - Summary of the contents including at least mission / instrument / time coverage; - Used capacity vs. total capacity. [Req DISS-HDD-50] The Unique HDD Id shall follow the format: DUPLICATE-<DISK GENERATION DATE>-<ORIGIN>-<n>of<N>, where: Page 65/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - <DISK GENERATION DATE> is the disk generation date in format DD MMM YYYY; <ORIGIN> identifies the origin of the media, and in the case of DSI is “DSI”; <n> is the number of the HDD in the series; <N> is the total number of HDDs in the series. [Req DISS-HDD-60] The Contractor shall ensure that the entire scope of data concerned is copied without any file being overlooked. [Req DISS-HDD-70] The Contractor shall ensure that the MD5 checksum of each file as copied to the destination HDD is identical to the corresponding MD5 checksum file in the source environment. [Req DISS-HDD-80] The Contractor shall produce and attached to the data a HDD inventory report including: - a list of the HDD produced identified by their unique ids - the contents of each HDD expressed in terms of mission / instrument / type of data / detailed contents. [Req DISS-HDD-90] Upon request of the ESA Technical Officer, the Contractor shall attach to the inventory report the transcription report from the original exercise of transcription of the data into the Contractor’s storage. o [Req DISS-HDD-100] Upon explicit request of the ESA Technical Officer, the Contractor shall generate and attach to the HDD to be shipped a full formatted extract of the Data Information System and CCM DB, including the full DSD, to convey all the knowledge regarding the data concerned. Shipment of the HDD The activities required include packaging and shipping the HDDs to the required destination and liaising with the contact point at the destination and with the courier until the successful reception is confirmed. [Req DISS-HDD-110] The Contractor shall package and ship the HDD together with the HDD inventory report to the requested destination. [Req DISS-HDD-120] The Contractor shall email a copy of the HDD inventory report to the ESA Technical Officer. [Req DISS-HDD-130] The Contractor shall ship the HDDs by means of a courier service chosen among the 3 major providers of such services in the country where the shipment originates. [Req DISS-HDD-140] The Contractor shall use the highest available level of reliability and tracking granularity among the options offered by the Courier service. [Req DISS-HDD-150] The Contractor shall track the shipping at least once a day and liaise with the Courier service until the successful delivery of the media is acknowledged by the contact point at the destination. Page 66/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req DISS-HDD-160] The Contractor shall notify the ESA Technical Officer in case of any issues with the shipping process, indicating the nature of the issue and the proposed recovery plan. [Req DISS-HDD-170] The Contractor shall notify the ESA Technical Officer upon successful reception of the package by the requested destination within 2 NWD. [Req DISS-HDD-180] Unless otherwise agreed with the ESA Technical Officer, the Contractor shall complete the process of delivery (including preparation and shipment) of the HDD within a time not exceeding a throughput of 3 TB per NWD. Recording of the Shipment details and destination [Req DISS-HDD-190] The Contractor shall record in the Data Information System / CCM DB the details of what data is shipped to what recipients on what media and at what date. 4.2.6.4 On-line data delivery/accessibility service This service enables on-line accessibility to specified data and accompanying information (metadata, QC reports and mission documentation) on an FTP/SFTP/FTPS server through the Contractor’s service infrastructure. Potentially any part of the data will have to be made available for QC or other purposes. The online data repository shall be available at short notice upon request. Datasets are accessible only to registered and authorised users, and primarily to ESA internal users (SPPA and its supporting organisations, DISS-UNIT). Datasets are made accessible from the Contractor’s infrastructure for a limited amount of time, e.g. for1 to 2 weeks in order to allow the recipient the time to download the data while minimising the risks of unauthorised attempts to access the data and the use of the resources. 4.2.6.4.1 Basic User Management Note: throughout the document and in particular in this section, the term user refers to the user population defined in Annex 4; therefore the end-user population of European Scientists is not in the scope of the user management described here. Even in the exceptional case that end-user dissemination may be required, there will be no management of individual user accounts for that population of users. [Req DISS-ONL-10] The Contractor shall collect from the ESA Technical Officer the list of users entitled to download the data, including name, affiliation and contact details. [Req DISS-ONL-20] The Contractor shall manage the required user population, as defined in the section on Service Level Management. [Req DISS-ONL-30] The Contractor shall provide to each user a unique password-protected access account for the specified protocol (e.g. FTP/SFTP/FTPS/HTTP/HTTPS) to access the requested data. Page 67/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req DISS-ONL-40] The Contractor shall communicate to each of the users concerned the access details to the data (IP address, protocol, user credentials). [Req DISS-ONL-50] The Contractor shall manage user credentials (esp. passwords) according to the standard rules of security applicable. [Req DISS-ONL-60] The Contractor shall implement a simple user management function through the Service desk to assist users with basic user management needs (lost password, change of person associated to an account, etc) [Req DISS-ONL-70] The Contractor shall ensure that only authorised users gain access to the data delivery server. [Req DISS-ONL-80] The Contractor shall ensure that every user attempted access (successful or not) to the service infrastructure data repository is logged. [Req DISS-ONL-90] In case of a security breach, e.g. unauthorised access to data, the Contractor shall inform immediately, within 1 NWD, the ESA Technical officer and the ESA Infrastructure and Security officer and take immediate actions to stop the unauthorised access, in accordance with the Security requirements of this SoW. [Req DISS-ONL-100] The Contractor shall ensure that the main data repository is not subject to any security risk even in the case that the security of the data delivery server is compromised. 4.2.6.4.2 Setup of an on-line data repository and loading of the data [Req DISS-ONL-110] The Contractor shall propose, agree with the ESA Technical Officer and implement/ configure for each dataset in scope on the data repository a well-defined directory structure, possibly following a “YYYY/MM/DD” (YYYY for year, MM for month, DD for day) or similar folder structure. [Req DISS-ONL-120] The Contractor shall make the data available via FTP/SFTP/FTPS according to the specifications of the ESA Technical Officer. Note: HTTP/HTTPS may also be required although this is not expected to be the default approach. [Req DISS-ONL-130] After approval by the DSI-CCB (see Common Requirements), the Contractor shall make the specified data available on-line to the recipient(s) within a time not exceeding 1 NWD for every 3 TB of data, and notify the recipients of the availability of the data. [Req DISS-ONL-140] The data delivery server shall allow simultaneous access of at least 10 users and a stable download performance for each of these 10 users of at least 100 GB/user/day including in case of simultaneous access to the same data. [Req DISS-ONL-150] Upon explicit request of the ESA Technical Officer, the Contractor shall generate a full formatted extract of the Data Information System and CCM DB, including the full DSD, to document all the knowledge regarding the data concerned, and make it accessible on the data server together with the data. Page 68/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 4.2.6.4.3 Recording of the on-line repatriation/delivery [Req DISS-ONL-160] The Contractor shall register in the Data Information System the contact details of the users entitled to download the data, together with the specifications of the data to be downloaded, the start and end dates and the request of the ESA Technical Officer. [Req DISS-ONL-170] Within the Data Information System, the Contractor shall record in detail any user access to the data delivery server, including the identity of the user, the user access details (user account) and information on the dataset(s) the user has been granted access to and when, and the data effectively accessed. 4.2.6.5 Optional data dissemination to end-users Although the standard channel for disseminating data to end users is through the Dissemination Facilities managed by DISS-UNIT, the Contractor may occasionally be required to put in place a simple way of disseminating data. The approach for simple dissemination of data to end-users shall be based on the following features: - Yearly duration (renewable) - Volume of data equivalent to the full output of the processing; - Users geographically distributed, mainly in Europe; - Up to 20 simultaneous users; - FTP/SFTP/FTPS server or better user friendliness; - Single user credentials to be shared by all the users accessing the same set of data; - Sufficient data throughput to allow appropriate performance (100 Mbps) in terms of simultaneous access and download time; - No management (registration, allocation of credentials etc) of individual users / no end-user support etc. - Constant monitoring of the data dissemination to prevent and correct service interruptions and degradation of the performance. This option shall be activated on request and funded separately from the rest of the repatriation/delivery WP. 4.3 Data Information and Configuration Management 4.3.1 Description This WP consists in collecting and managing configuration information on the data while it is being handled in the scope of the main WPs of this SoW. The long term archival of the data is performed i.a. in the PACs / LTDP Centres as part of separate contractual arrangements and is not in the scope of this contract. Indeed, DSI will only hold the data Page 69/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use temporarily, for as long as this will be useful for the purposes of this SoW. It is therefore particularly important to perform rigorous configuration management of the data. The information managed includes, in addition to the configuration information, all the knowledge about the data which is of use for determining and interpreting the characteristics of the data (esp. quality and completeness) affected by these WP’s. 4.3.2 Scope The scope of the WP includes: 1) Data Records (ranging from raw data to higher level products, browses, auxiliary, ancillary and calibration and validation data set) 2) Processing systems operated in the scope of this SoW and kept under CCM for later reference as explained under WP Integration of Processing Systems. 3) Mission Documentation to the extent that it is used as part of the WP’s of this SoW and including format description, naming conventions, Mission Operations Plans, etc. Data records are identified as: 1. Raw data: Raw data can be disregarded if consolidation to Level 0 data has been successfully completed without any raw data loss. In the case of ENVISAT, for some instruments, the raw data are turned into NRT Level 0 data, and NRT Level 0 data are processed into “consolidated” L0 data. In this context the consolidation consists in removing overlaps (by means of a tool called ECONS). 2. L0 data 3. L1 to higher levels mission data products when systematically generated as part of the mission requirements and/or reprocessed. 4. Meta-data 5. Browses whenever generated 6. Ancillary data (spacecraft ephemeris information, attitude, etc) 7. Auxiliary data required to process the telemetry payload data to generate the nominal mission products 8. Cal/Val data (needed to calibrate the satellite instruments and monitor data quality) 4.3.3 Activities The activities required include: Performing the Configuration and Change Management of the data Collecting, recording, organising, maintaining operational information about the data subject to the core activities of this SoW; Managing the resulting Data Information System and CCM DB; Providing a variety of reports and answering questions and enquiries on the data holding and its evolution. 4.3.4 Inputs Page 70/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Data holdings in scope of the core WP’s of this SoW: - Data Records - Mission Documentation - Baselines of the Processing Systems operated as part of this SoW Issues regarding the data holdings Data Change Requests 4.3.5 Outputs Data Configuration and Change Management Plan (Data CCM Plan) Data holdings under Configuration Management Data Change Requests (DCR’s) resulting from issues Dispositioned DCR’s Up to date Data Information System and CCM DB and corresponding outputs, exports and reports including configuration status accounting reports 4.3.6 Requirements 4.3.6.1 Data information management The data managed as part of this WP is an irreplaceable asset. Its value is in the ability to use it, which depends in large part on the knowledge of the asset and the management of this knowledge. As indicated in Annex 4, the Data Librarian is the ESA role in charge of managing this knowledge. It is however necessary to support this ESA role with activities in this contract, since significant information concerning the data originates in this contract or revolves around the activities performed in this contract. This WP includes the management by the Contractor of the Data Information System and CCM DB relating to the data in scope of the core WP’s. [Req DCCM-INF-10] The Contractor shall maintain and update an Information System concerning the data included in the scope of the core WP’s of the Service and in the Contractor’s data storage. [Req DCCM-INF-20] The Contractor shall cover as part of the data-related information system the information pertaining to the data, including at least (but not limited to): - the information represented in the data model of the ESA inventory data base (EO-INV) provided as input [RD EO-INV]; - the full set of attributes of each dataset contained in the Dataset Description (DSD) described in the section on CCM; - the major events related to the data, taking place within or outside this contract. [Req DCCM-INF-30] The Contractor shall include as part of the data-related information system the information related to the activities performed in the scope of this contract, including: - projects performed (consolidation, reprocessing, etc) and their status Page 71/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - data delivery activities including the data delivered, the date and the destination / audience activities of guarantee and follow-up from the core WP’s. [Req DCCM-INF-40] The Contractor shall deliver to the ESA Technical Officer a fully readable (ASCII) export of the Data Information System and CCM DB, once per month and additionally upon request, covering: - either the full current data holding (default) - or a specified subset per mission / instrument / instrument type and/or time range. ESA needs to have at its disposal the contents of the Data Information System and CCM DB without depending on the Contractor. The readable format of the dump of the information system will be agreed between the Data Librarian of ESA and the Data and Configuration Manager of the Contractor during the contract phase-in. [Req DCCM-INF-50] The Contractor shall provide to the ESA Technical Officer, regularly and on request, various reports on the data holding, including but not restricted to: - the detailed inventory of the data storage in relation to this Contract, including identification, size, coverage, status, completeness, quality, etc. - the detailed description of some or all of the datasets related to a mission / instrument, including the DSD information and all available mission / instrument / dataset information formatted for management presentation; - the evolution of the data between 2 given dates sorted per mission / instrument / dataset (reprocessing, completeness/quality improvements, additions, delivery, etc); - the evolution of the data due to a given project. These reports are of particular importance for the credibility of the Contractor and of the Service as they will be further distributed to ESA Management and used in the formation of strategic plans and as a basis for launching new projects. The precise contents and format of the reports will be agreed between the Data Librarians of ESA and of the Contractor during the contract phase-in. [Req DCCM-INF-60] The Contractor shall respond by email to requests for information on the data in scope of the core WP’s coming from the ESA Technical Officer and other ESA key roles specified in Annex 4 as well as the EOP-G line management. The Contractor shall CC the Data Librarian of its responses to the queries. [Req DCCM-INF-70] The Contractor shall update the Data Information System within 2 NWD of any update of the information contained or discovery of addition information. 4.3.6.2 Data Configuration Management [Req DCCM-CCM-09] The Contractor shall produce and apply a Data Configuration Management Plan to document the activities of configuration and change management applicable to the data as described in this SoW. The Data Configuration Management Plan is subject to the approval of the ESA Technical Officer. Page 72/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 4.3.6.2.1 Configuration Management This process includes managing and tracking the status and evolution of the Data Records (datasets and their components) and of the Mission Documentation. In addition, Processing Systems are covered to the extent of capturing and archiving baselines of the tools provided as CFI’s as per the requirements of WP Processing System Integration. [Req DCCM-CCM-10] The Contractor shall maintain - the data files, datasets and dataset descriptions, and - the mission documentation under configuration and change management, and apply all the normal rules of CCM, including (but not restricted to): - any data file/record shall be part of one or more identified CIs - any document shall be part of one or more identified CIs - any change to the data shall follow a controlled process including the recording and authorisation of the change - any change to the contents of a dataset shall give rise to a new version number. [Req DCCM-CCM-20] The Contractor’s CCM process should preferably classify the documents relevant to the data according to [RD PDSC]. [Req DCCM-CCM-30] The Contractor’s CCM process shall define and uniquely identify all Configuration Items (CIs) and their attributes, their constituent components and their mutual interrelations. The identification and structure of the CI are an integral part of the Data Configuration Management Plan and are subject to the approval of the ESA Technical Officer. [Req DCCM-CCM-40] The Contractor shall produce and maintain a Configuration Item Data List (CIDL), listing the Configuration Items subject to the various services in this SoW. [Req DCCM-CCM-50] The Contractor shall maintain as part of the CIDL an up-to-date map of the dependencies among CIs within and across datasets. The dependency map covers in particular the dependency of primary data to auxiliary data, ancillary data and other components required for the processing of the primary data. [Req DCCM-CCM-60] The CCM process, including the choice of the CIs and the CI structure, shall allow to control all changes to existing CIs, in particular: - Change to a Dataset description - Change to a data file name - Addition, removal, replacement of a data file in a dataset - Addition/creation of new Datasets, DSD, data files, mission documents - Change to mission documents. Page 73/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Note: depending on the Contractor’s design of the CCM system, changes in the contents of a data file may be treated as addition of a new file and removal of the old file. [Req DCCM-CCM-70] The Contractor shall adopt a versioning system compliant with the following features: - each data CI version will be of the form xx.yy - yy shall be increased at each minor intervention on the CI (ie intervention performed as part of the follow-up and guarantee component of the core WP’s) - xx shall be increased at each major intervention on the CI(ie intervention performed as part of a project WP). Note: The versioning system should reuse any existing identification / naming / versioning conventions. [Req DCCM-CCM-80]The Contractor shall make the DSD a component CI of the dataset (ie one of the CI elements in the breakdown of the overall CI which is the dataset). 4.3.6.2.2 Management of baselines [Req DCCM-CCM-90]In addition to tracking and recording individual changes, the Contractor shall manage and store, and as necessary retrieve and deploy, baselines of the data and of the mission documentation. [Req DCCM-CCM-100] The Contractor shall take a baseline of a dataset at a minimum: - upon delivering / starting the dissemination of a dataset - before and after every major change to a dataset (where a major change is identified as triggered by a project). [Req DCCM-CCM-110] The Contractor’s baseline management shall take account of the dependencies: - among datasets (eg primary / auxiliary) - between datasets and dataset-specific mission documentation. [Req DCCM-CCM-120] The Contractor shall label and store baselines in a way that guarantees unambiguous retrieval. [Req DCCM-CCM-130] In addition to the baselining requirements of WP Processing System Integration, the Contractor shall take a baseline of each Processing System operated as part of this SoW: - upon delivery by ESA of the Processing System as a CFI - upon the start of operations of the Processing System as part of a project; - upon the termination of operations of the Processing System as part of a project. 4.3.6.3 Data Change Management 4.3.6.3.1 Change Control Rules [Req DCCM-CCM-200] Within the scope and for the purpose of this SoW, the Change Authority applicable to the Data (including documentation) handled under this SoW shall be the DSI-CCB, as defined in Annex 4. Page 74/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The process for raising, collecting and managing issues, Service Requests, Project Requests and Data Change Requests is described in Section “Service Process Management”. Note: the ESA Technical Officer has the authority to pre-empt or to override the decision of the DSI-CCB and disposition the change autonomously. This will only happen in exceptional circumstances. In any case, only ESA is entitled to reject a DCR. [Req DCCM-CCM-210] If for any reason the ESA Technical Officer pre-empts or overrides the decision of the DSI-CCB and dispositions the change autonomously, the Contractor shall document the decision and the different approval process. 4.3.6.3.2 Change Management This process consists in the pro-active change management of the data, including managing and tracking the evolution of the datasets and their components and of the documents. The process ensures that standardized methods and procedures are applied for efficient and prompt handling of all issues giving raise to Data Change Requests in order to avoid change-related problems. [Req DCCM-CCM-300] The Contractor shall manage and record changes in a way that all the changes between any two successive baselines can be listed and individually undone. [Req DCCM-CCM-310] The Contractor shall manage the life cycle of Data Change Requests (DCR) based on a detailed DCR generated and managed in the Change Management system. [Req DCCM-CCM-320] The detailed DCR in the Change Management system shall contain at least the following information, essential to the change management process: Unique DCR identifier Originator of the issue or DCR and corresponding Contact information Change authority applicable in compliance with the above indications Date and time stamps of the DCR Detailed description of the change requested including details of the configuration items (CIs) affected Reason for the change and/or impact of not making the change Impact of the change including risks involved Cost of the change (which is zero depending on the type of issue giving raise the the DCR) Severity/urgency of the underlying issues / of the change requested Related DCRs if any Issue count/linking issues In case the DCR results from issues reported by users of the data, the DCR needs to be linked to the corresponding issues, and the occurrences of the issue counted. References/links to any issues with matching symptoms shall be included in the DCR record. This allows the severity of the situation to be properly assessed, and helps support staff identify workarounds and resolutions; Links to further information eg any associated documents Page 75/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Proposed implementation approach and schedule History An accurate record of the investigation or other activities undertaken shall be stored within the DCR Status The current status of the DCR shall be recorded in the DCR record. The range of status shall include: initiated, proposed, approved, rejected, work-around applied, complete. Other data useful for the management of the change by the Contractor. [Req DCCM-CCM-330] Except in exceptional cases where the ESA Technical Officer waive the resolution of a Data Change Request, the Contractor shall carry out the implementation and the verification of all the approved Data Change Requests (DCRs) as part of the guarantee and follow-up activities of the relevant WP’s, within the boundaries indicated for guarantee and follow-up activities. [Req DCCM-CCM-340] Whenever data or documents necessary to resolve a DCR (eg data files corresponding to a gap in the data holding) are identified and available, the Contractor shall endeavour to acquire that data or document from its source, regardless of the type of media on which it is available. This includes the scanning to PDF/A of paper documents. [Req DCCM-CCM-350] Throughout the various activities performed as part of the contract, the Contractor shall keep the Data Records and the corresponding Mission Documentation fully aligned and inter-related, including: - the links between the datasets and the dataset-specific documents - the links between the data and the documents describing the data (eg format). [Req DCCM-CCM-360] The Contractor shall define, maintain, execute and track the life cycle and workflow of the DCR’s. The life cycle shall reflect at least the steps defined below: the collection of DCRs through the Issue Management Process, which acts as the point of generation and primary collection of DCRs from all authorised sources; the appropriate approval process to ensure only approved changes to the data are implemented and deployed/delivered (or alternatively waived), including the recording of the disposition of the DCR by the approving authority competent (see above); the implementation and verification of the disposition the corresponding update of the Data Information System and CCM DB; the activation of the Release Management process for successful deployment/delivery. [Req DCCM-CCM-370] The Contractor shall ensure the feedback of the life cycle of the changes from the Change Management process and tool to the Issue Management process and tool, so that the latest status of the original issue is always up to date. 4.3.6.3.3 Assistance to the Secretariat of the Data Configuration Control Board Changes to the data will be debated and dispositioned by the Data Configuration Control Board (DSI-CCB), described under the Section “Roles and Responsibilities”. DSI-CCB meetings are the forum where the DSI-CCB debates and dispositions the DCRs for the scope of this SoW. Page 76/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use In order to accelerate the approval process in urgent cases, or for logistical reasons, electronic meetings (with minutes) or exchanges of email are an acceptable way of collecting the disposition. The DSI-CCB shall meet in principle once a month. Upon proposal of the Contractor and/or decision of the Chair of the DSI-CCB, the frequency of the meetings may be altered and/or ad-hoc meetings may be called, eg to disposition urgent change requests or to process a surge of Data Change Requests. [Req DCCM-CCM-400] The Contractor shall assist the Secretariat of the DSI-CCB, and in particular upon request of the co-chairs or secretary prepare, organize, help moderate, support, record and/or document the DSI-CCB meetings. 4.3.6.4 Status Accounting and Reporting [Req DCCM-CCM-500] As part of the execution of the DCR life cycle, the Contractor shall keep the DCR information, particularly the status, up to date at each step of the entire life cycle. [Req DCCM-CCM-510] The possible states and associated values for the status of a DCR shall include at least the following: Raised Under review Under approval Accepted Rejected Suspended Under implementation Validated Released [Req DCCM-CCM-520] The Contractor shall deliver, on demand, outputs, exports and reports (status accounting) and CCM DB extracts, including: - full inventory of the data currently in the custody of the Contractor, including the physical location of the data down to the individual media - full inventory of the original media provided by ESA including labelling information, physical location, physical and technical characteristics, precise contents, etc. - full contents, in readable form (ASCII) and/or in a format to be specified during the phase-in of the Data Information System and CCM DB. [Req DCCM-CCM-530] The Contractor shall provide the ESA Service Manager and Data Librarian with permanent read-only access to the Data Information System and CCM system. [Req DCCM-CCM-540] There may be cases in which a dataset which is subject to guarantee and follow up of the core WP’s is also, simultaneously, undergoing a project (eg reprocessing) as part of a separate project, possibly under a separate contract with a different supplier. To cover this type of situation, the Contractor shall upon request of the ESA Technical Officer: Page 77/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - - communicate the changes performed as part of the guarantee and follow up to the project team through the ESA Technical Officer. The Project Team will be responsible carry over the changes into the new development. capture into the data information system of the service, through the ESA Technical Officer, the information concerning the Project. 4.3.6.5 Provision of inputs to ESA CCB ESA operates change control on its Ground Segment infrastructure, which may occasionally impact the interfaces and the CFIs of DSI. Such changes are decided based on a variety of factors. The Contractor will be required to provide the DSI input for this decision making process. [Req DCCM-CCM-600] Upon request of the ESA Technical Officer, the Contractor shall provide an assessment of the impact of proposed changes to its interfaces and CFI’s on the cost / schedule / other aspects of the activities covered under this SoW, and potentially alternative proposals. The ESA Technical Officer will assess the Contractor’s impact assessment together with the Contractor and present the agreed input to the relevant ESA change control body, which will take the final decision on the basis of all the drivers involved. Page 78/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 4.4 Service and Project Management 4.4.1 Operating model The overall operating model is represented hereunder: Page 79/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use DSI – Operating Model End users EOHelp GTO Key Roles EOP-G Managers GMQ ESL, QWG Request / Originator Satisfaction Request / Originator Satisfaction Request / Originator Satisfaction Request / Originator Satisfaction DSI Service Desk (1st level) Ticket opening No Ticket in scope of DSI? Ticket Resolution Yes TO / ESA Key Roles External sources of information / Sources of additional data (beyond CFI) DSI - Core Services Destination of data repatriation / dissemination: PACs Dissemination Facilities etc - Support Services D-CCB Legenda Sources of requests / tickets Third parties / contributors Supervision bodies Service Contractor 4.4.2 Use of documented procedures Page 80/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j GMQ/ ESL/ QWG ESA UNCLASSIFIED – For Official Use [Req SERV-PROC-10] The Contractor shall ensure that the service and underlying infrastructure are managed, monitored, serviced, maintained and operated according to documented Service Operational Procedures. [Req SERV-PROC-20] The Contractor shall train the operators involved in the service to the technical functioning of the service tools and infrastructure and to the operating procedures applicable to the service. The Contractor’s Service Operational Procedures are not a deliverable subject to the approval of the ESA Technical Officer. However ESA reserves the right to examine them on request. 4.4.3 Support Processes (Availability, Performance and Capacity Management) This section lists general requirements applicable for Capacity, Availability and Performance Management. 4.4.3.1 General Requirements [Req SUPP-PERF-10] The Service Infrastructure shall fulfil the levels of capacity, availability and performance, respectively during NWH and EWH, specified in the SLA. [Req SUPP-PERF-20] The capacity at any time shall be sized to meet the performance and availability requirements of all the projects and Service activities on going at the time. Note: the ESA Technical Officer will share with the Contractor all the information available from the Demand Management process concerning the planning and scheduling of the projects. The planning and schedule of the projects to be undertaken in the scope of this SoW will be known in advance (2-3 years) with a margin of fluctuation, and the accuracy will improve gradually as the time nears (1 year – 6 months). However it may also occur that projects are decided at short notice and/or re-scheduled at short notice (~ 2-3 months). The above requirement applies in all cases. [Req SUPP-PERF-30] In addition to the required performance levels specified in the SLA, the Contractor shall take all reasonable precautions to ensure that the Service infrastructure and Service Management System remains available outside the schedule specified in the SLA. 4.4.3.2 Technical Requirements The technical requirements are designed to ensure that the infrastructure used by the Contractor for the delivery of the Service is cost-effective and to avoid vendor- and supplier- lock-in. In addition, the technical requirements aim to minimise the likelihood and impact of infrastructure failure. Page 81/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SUPP-TECH-10] The Contractor shall ensure that the infrastructure used for the purpose of the Service is: - in line with the current mainstream technology, excluding niche or immature technology; - based on standard components and building blocks (in terms of choice of HW, OS, etc); is commercially available (excluding proprietary technology) and chosen amongst the best selling in its category. [Req SUPP-TECH-20] The Contractor shall ensure that the infrastructure used for the purpose of the service is scalable, to support: - the peaks and troughs induced by the variations in time of the level of demand; - the potential continuous increase in the amount of data to be stored, extracted and repatriated/delivered, resulting from the increase of scope of the service as well as the generation of data by the service itself; - the possible activation of the option for end-user dissemination; - the fast restoration of the processing environment in case of need; - other possible business events resulting in ESA requesting an increase in service capacity. [Req SUPP-TECH-30] The Service infrastructure shall be coherent in terms of: - the same standard solution shall be applied throughout the Service infrastructure to implement the same function / feature; - the number of internal and external interfaces and data flows shall be minimal. [Req SUPP-TECH-40] The Contractor shall ensure that all the components of the infrastructure are covered by the appropriate guarantee and/or maintenance arrangements, compatible with the end-to-end performance parameters of the SLA. [Req SUPP-TECH-50] The Contractor shall ensure that no data is stored in media/equipment that has exceeded its recommended useful life time as per Constructor documentation. [Req SUPP-TECH-50] The Contractor shall submit an Infrastructure Architectural Design Document describing the architecture main building blocks and their interfaces as well as the approach to fulfilling the infrastructure requirements of this SoW. The Contractor’s Infrastructure Architectural Design Document is a deliverable subject to the approval of the ESA Technical Officer. 4.4.3.3 Process Requirements [Req SUPP-PROC-10] The Contractor shall ensure that the Service Operational Procedures include Infrastructure Operational Procedures defined and applied for the management, monitoring, servicing, maintenance and operation of the Service infrastructure. [Req SUPP-PROC-20] The Contractor shall devise, document in the Data CCM Plan and apply a standard labelling system for the media and equipment used to archive data. Page 82/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SUPP-PROC-30] The Contractor shall include in the operational procedures regular checks on the continued legibility of the labels and their physical adherence to the corresponding media/equipment. [Req SUPP-PROC-40] The Infrastructure Operational Procedures shall include in particular: - system housekeeping at OS and DB level; - automatic, semi-automatic and manual failure detection and status monitoring and failure detection; - preventive and corrective interventions, triggered either by the Monitoring and Detection activities or through a notification to the Service by a user or by an ESA Service key role. [Req SUPP-PROC-50] The Infrastructure Operational Procedures, including preventive and corrective interventions, shall be based on the use by the Contractor of the Service Management System. Note: this is required in particular for tracking and tracing purposes. [Req SUPP-PROC-60] The use of the Service Management System for the Infrastructure Operational Procedures shall be based on a classification of issues: - separate from the other issues managed in the system; - fully accessible in R/O mode to the ESA Technical Officer. [Req SUPP-PROC-70] The contractor shall produce, maintain and execute an Infrastructure Maintenance Plan composed of: - a list of all Infrastructure components (HW/ OS/ DBMS/ COTS/ Middleware/ Tools /data storage and network components) detailing the version information and any expected need to upgrade due to obsolescence (ie technical upgrade triggered by the discontinuation of technical support by the corresponding vendors) or cross incompatibility within the contract period. - the corresponding Upgrade and Migration action plan for each component. The Contractor’s Infrastructure Maintenance Plan is not a deliverable subject to the approval of the ESA Technical Officer. However ESA reserves the right to examine it on request. ESA reserves the right to audit the Contractor’s infrastructure for compliance with the requirements of this SoW related to: - Physical security and access control including adequacy of the physical environment - Existence and day-to-day utilisation of written procedures - Training of the operators - Adherence to the other requirements listed in this section. [Req SUPP-PROC-80] The Contractor shall facilitate ESA’s audit process by providing access to the infrastructure as well as the evidence (written procedures, logs, training records etc) and cooperation necessary to perform an audit by the ESA Technical Officer upon 3 NWD’s notice. The access to the archive shall include: - Physical access to the archive and media/equipment - Read-only access to the data itself. [Req SUPP-PROC-90] The Contractor shall produce, maintain and apply a Support Process Management Plan covering: - Capacity - Availability Page 83/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - Performance Including in particular: - projected capacity required in the short/mid-term based on the expected nature and level of activity (see the Section on Demand Management); - corresponding plan for the evolution of the infrastructure. 4.4.3.4 Monitoring and Detection Requirements [Req SUPP-MGT-10] The Contractor shall define infrastructural resources and corresponding parameters critical to the fulfilment of the availability, performance and capacity requirements. [Req SUPP-MGT-20] The critical parameters shall include: - the level of utilisation / occupation of the critical resources, including CPU, RAM, storage, media, and bandwidth; - the number of uses of those resources with a limited maximum number of uses recommended by the Constructor, including tapes; - the age of those resources with a limited life span recommended by the Constructor, including tapes. [Req SUPP-MGT-30] The Contractor shall define thresholds for the critical parameters, defined such that the thresholds are reached before any performance requirement is impacted. [Req SUPP-MGT-40] The Contractor shall continuously monitor: - the status of the critical resources; - the current (real-time) and projected critical resource parameters usage against the defined thresholds. [Req SUPP-MGT-50] Where necessary, the Contractor shall automate monitoring features by means of automated tools, semi-automatic (eg scripts) or manual (eg procedures) means. [Req SUPP-MGT-60] The Contractor shall detect any critical infrastructure component not in nominal status or defined threshold exceeded and notify the ESA Technical Officer. [Req SUPP-MGT-70] The Contractor shall immediately trigger the appropriate preventive and/or corrective maintenance activities upon: - critical infrastructure parameters exceeding the defined threshold values; - detection of a non-nominal situation of the infrastructure and/or notification; - notification of a non-nominal situation by a user or by an ESA Service key role. 4.4.3.5 Reporting Requirements [Req SUPP-REP-10] The Contractor shall include in the Service Monitoring Report a section on Infrastructure Status and Monitoring Report containing in particular: - the composition, allocation and status of the infrastructure; Page 84/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - the monthly values and trends on all critical parameters; a summary of the incidents reported against the infrastructure during the period and their status; a summary of the evolution of the infrastructure during the period; the current forecasts of infrastructure needs; the current plans of evolution of the infrastructure. 4.4.4 Service Continuity Management [Req SERV-DR-10] The Contractor shall apply the following policy for Service Continuity: the data is ordinarily held in the Contractor’s storage; the Contractor takes a backup of the data regularly and ships a copy of this backup to ESRIN (as a location different from its regular storage), which serves as disaster recovery copy; the Contractor is requested to have a plan for the setup and activation of a disaster recovery environment in case of a disaster, able to preserve the full data holdings and partially restore the capabilities of the projects and related services. [Req SERV-DR-20] The Contractor shall be able to ensure the continuity of the service operations in the event of any one disaster striking its facilities by producing and maintaining a Disaster Recovery plan and related emergency procedures as a section of the Service Continuity Plan. [Req SERV-DR-21] The Disaster Recovery plan shall cover: - the full recovery of the data holdings (including science data, documentation, service data, processing systems software, infrastructure settings etc), based on the backup policy requirements of this section; - the recovery of an operational capability commensurate with the nature and urgency of the DSI scope: unlike a receiving station DSI covers no real time functions and no data loss is incurred in case of loss of functionality. [Req SERV-DR-22] The Contractor shall propose as part of the Disaster Recovery plan the recovery of operational capability to an extent which is cost effective in relation to the type and architecture of infrastructure used. The Contractor shall consider the following priorities: - recovery of the complete data holdings: top priority - recovery of the data information system and CCM DB: high priority - recovery of the data dissemination capability, with a revised throughput to be agreed with the ESA technical officer: high priority - completion of on-going projects with revised schedule to be agreed with the ESA Technical Officer and ESA Project Managers concerned: high priority - execution of projects not yet started: medium priority - recovery of the remainder of the Management System: medium priority - recovery of the full SLA compliance: low priority. Note: for Disaster Recovery, only the plan is requested as part of this SoW, not any implementation or spare infrastructure. In case of disaster, the contractor will be required to provide within 20 NWD a project proposal (cost and schedule) to execute the Disaster Recovery section of the Service Continuity plan. Page 85/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SERV-DR-30] The scope of the Service Continuity Management (backup policy as well as Disaster Recovery system and plan) shall cover the data currently present in the Service infrastructure (in the broad sense, including data records, documentation and processing systems), as well as the Service Management System, Data Information System and CCM data base and the full set of service documentation, the operational tools, infrastructure settings, etc. [Req SERV-DR-30] The backup policy shall be as follows: - the FREQUENCY shall be as follows: o Incremental backup: daily o Full backup: weekly - the RETENTION shall be as follows: o Daily backup: last 7 o Full backup: last 3 - a copy of the last full backup of every month shall be shipped to ESRIN; - the backup media shipped to ESRIN shall be LTO5 tapes; - the ESRIN copy of the backup media shall be shipped to ESRIN to the attention of the ESA Data Librarian; - the backups shall be accompanied by a detailed listing of their contents. After delivery of each full backup, ESA will check the backup and return the previous backup tapes to the Contractor for re-use. [Req SERV-DR-40] The Contractor shall liaise with the ESA Technical Officer to organise the shipment of the tapes to be reused. [Req SERV-DR-50] The Contractor shall develop, manage and maintain a Service Continuity Plan including at least the following: scope; approach; roles and responsibilities; backup and restore procedures; disaster prevention, detection and mitigation (eg fire / flood / intrusion protection) processes and procedures including alert systems and decision making process for disaster assessment and recovery activation; disaster recovery as specified above; service continuity test and validation approach. When Disaster Recovery is activated the SLA and the schedule of all projects will be relaxed in agreement between the Contractor and the ESA Technical Officer and the ESA Project Managers concerned. 4.4.5 Security Management Security Management is addressed in Annex S of this SoW. Page 86/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 4.4.6 Asset Management [Req ASS-MGT-10] The Contractor shall manage as part of the Infrastructure Maintenance Plan an inventory of all the assets (software and hardware, data storage components and media, etc) used for the provision of the Service throughout their lifecycle (licenses, maintenance contract, obsolescence, etc…). The asset inventory provides a complete list of the assets and who is responsible for their control. 4.4.7 Issue Management The Issue Management process consists in the collection, analysis, processing and follow-up of issues raised in relation to the data and services in scope. There may be cases in which the issue management support officers (Service Desk) will need to resort to the specialized and deeper technical knowledge of the data and infrastructure engineers for the execution of their activities. Such situations shall be treated internally by the service as part of the organization of the service and in a way completely transparent to the execution of the service, provision of the required support, production of deliverables and corresponding schedules. A “co-location” approach where members of the Service Desk and data and infrastructure engineers are located at the same premises is encouraged. This approach is considered important in the service organization. [Req SERV-ISS-10] The Contractor shall handle Issue Management both as a pro-active and as a re-active process, anticipating or responding to situations that affect, or potentially could affect the service. [Req SERV-ISS-20] The Contractor shall analyse issue statistics to identify the occurrence of multiple issues or increasing trends for specific types of issues, and shall seek to resolve the underlying causes of these issues, so as to prevent their recurrence. [Req SERV-ISS-30] The Contractor shall ensure that the results of issue management for each issue are adequately fed back into all the service processes by providing data for performance metrics on issue response and resolution times. [Req SERV-ISS-40] The list of possible status of an issue shall include at least the following: Open waiting for treatment Solved with workaround (*) a workaround resolution has been applied and is pending the confirmation from the Originator of the issue Page 87/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Solved a resolution has been applied and is pending the confirmation from the Originator of the issue Unsolved no resolution can be applied (e.g. in case of a known defect for which fixing has been waived) Rejected no resolution can be applied (e.g. in case of a Change Request rejected by the DSI-CCB) Closed Pending Issue Originator Pending DSI-CCB pending ESA Technical Officer pending Third parties pending other analysis in progress Page 88/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j Solved or Unsolved status is accepted by the Originator of the issue waiting for an event not under the control of the Contractor but from the originator of the issue (e.g. clarification, pending input). This status is always accompanied by the reason for being pending waiting for DSI-CCB disposition waiting for an intervention by the ESA Technical Officer. This status is always accompanied by the reason for being pending (Pending notification to the ESA Technical Officer) waiting for an event not under the control of the Contractor but of third parties involved in the end to end user perspective of the service. This status is always accompanied by the reason for being pending waiting for an event not under the control of the Contractor. This status is exceptional and always accompanied by the reason for being pending indicates that the issue is in charge of the Contractor that is working to analyse and to find a resolution. ESA UNCLASSIFIED – For Official Use under implementation indicates that the resolution is being implemented. This status is always accompanied by the planned date/time for completion (*) When an issue is solved with this status, a new issue is raised for the final resolution of the original issue. The SLA applies to the original issue until it is solved, then to the subsequent issue with the relevant severity. [Req SERV-ISS-50] Issue management and Control procedures shall be supported by the Service Management System. 4.4.7.1 Issue collection The user population defined in Annex 4 will be generating issues of the types defined in Chapter Service Level Management, section Issue Classification. [Req SERV-ISS-60] The Contractor shall ensure the collection of issues through: - email (available to all entities allowed to raise issues on the Service); - as a desirable feature, direct access to the Management System (available to EOP-G Service Key Personnel and EOP-G Personnel designated by the ESA Technical Officer). Detailed requirements for the Service interface Definition (SID) and for the operating model to interface and coordinate any third parties impacting the provision of the end-to-end service are provided in the section “Interfaces to Third Party services” 4.4.7.2 Coordination vs Interfacing of third parties Coordination (as opposed to simple interfacing) is required in the relationship between the Contractor and third parties (including the EOHELP Service Desk) which contribute to providing an end to end service from the User / Customer point of view. [Req SERV-ISS-70] The Contractor Shall be the end-to-end owner of its related issues and of the Issue Management. It shall therefore be responsible to manage, coordinate, supervise any third parties (including the EOHELP Service Desk), check the progress of the resolution and whether the third party’s performance is aligned with any relevant SLA as communicated by the Technical Officer of the Service in question, and escalate if necessary. [Req SERV-ISS-80] When a ticket/issue is assigned to any third parties by the Contractor, the Issue Management process shall automatically notify the third party. Page 89/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The third party will be responsible to notify the Contractor via email of the next change of status of the issue. The closure of the ticket/issue (switch from “Solved” to “Closed” status) remains the responsibility of the Contractor Service desk, after agreement of the originator of the issue. 4.4.7.3 Raising of issues [Req SERV-ISS-90] The Contractor itself shall raise an issue whenever a risk, shortcoming or opportunity for improvement occurs and the resolution is liable to preserve or improve the service to users, mitigate the risk or facilitate subsequent service work or projects. [Req SERV-ISS-100] The Contractor shall continually look out for existing issues in the data (gaps, overlaps, non-compliances, missing documentation etc.) which may not yet be identified or documented and generate corresponding issues in the Service Management System. [Req SERV-ISS-110] The Contractor shall continually look out for existing solutions to known issues in the data (newly available document, files usable to fill a gap, etc) which may not yet be resolved or addressed and generate corresponding Change Requests (CRs) in the Service Management System. Note: the Change Requests generated in relation to the data will give raise to Data Change Requests (see Section Issue processing) In particular, initiatives that tend to improve the collection and recording of Data-related information in support of the Data Librarian are encouraged. The raising of preventative issues to avoid negative operational consequences of any slowly degrading situations is also encouraged. Issues are normally directed to the Service Desk, which opens the ticket in the Service Management System. [Req SERV-ISS-120] Whenever the issue is raised directly to a key person or representative of the Contractor, or raised by the Contractor itself, the Contractor shall record the issue in the Service Management System. [Req SERV-ISS-130] In order to speed up the service rendered to the user, the scope of related issues covered by Service Desk at the first contact shall: - be no lower than specified in the SLA; - increase gradually based on knowledge transfer from level 2 support. Issues shall be handled at the first contact by Service Desk when sufficient documentation on the subject of the issue and related procedures can be supplied by the Contractor’s Service Management. After approval of the inclusion of new issues, the Service Desk shall process such issues at the level of the Service Desk. Any further tickets related to such issue that would be not resolved at the first contact by Service Desk will be considered as a failure of the Service Desk, to the extent that the ticket was solvable at the first contact. [Req SERV-ISS-140] Whenever an issue is likely to re-occur and can be solved by applying systematically a well-defined procedure with no impact on configuration items, the Contractor shall: Page 90/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use include that issue among those to be covered at first contact document in detail the solution of the issue train the designated Service Desk personnel to solve the issue. 4.4.7.4 Classification of the issues An issue is classified as described in section “Issue classification” of chapter “Service Level Management”. An issue is classified according to its type (e.g. PR, CR, SR) and severity (High, Medium, and Low) The preliminary classification of the issue is done by the Service Desk. The Contractor has always the responsibility within the overall execution of the service to ensure that the issue is properly classified and to modify the classification as needed. Moreover the ESA Technical Officer has the prerogative to further modify the classification of the issue, and have the final word on the classification. The classification is normally based on the conformance of the data and service to written requirement specifications and/or other documentation. In cases where the available info and documentation alone does not allow the classification, a common-sense approach is to be used, based on the information provided by the generator of the issue and on generally accepted service practices, and applying the rules defined in section “Issue classification” of chapter “Service Level Management”. The severity classification of the issue shall also be a function of the business context. For example, depending on the visibility of a particular dataset in terms of usage and applications, an error reported in the data will take a higher level of severity. In addition, the severity classification of the issue shall be incremented from Low to Medium or from Medium to High if the issue is or becomes urgent or in case the processing of the issue has been delayed by an error in the initial classification made by the Contractor. 4.4.7.5 Security Issues A security issue occurs when a violation of the Security requirements or Interconnection Agreement occurs. Security issues are always to be considered with severity “High”. [Req SERV-ISS-150] In case of security issue, the Contractor shall carry out the actions recommended in the approved action plan following the activation of the requirements for Defence against Security Vulnerability (SeeAnnex S) and at no additional cost. 4.4.7.6 Issue analysis [Req SERV-ISS-200] Whenever an issue is raised, the Contractor shall, as the first step of the analysis, detect whether the issue is classified as a PR (Problem Report) with priority High (also referred to as blocking problem). Page 91/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SERV-ISS-210] In case the issue is a blocking problem, the Contractor shall immediately and in accordance with the SLA, devise a quick but working solution to allow the originator of the issue to overcome the block. The quick solution may be the final solution or a provisional one, i.e. a work-around such as a manual procedure, an alternative user scenario, etc. The Contractor shall generate a Problem Report and activate the problem management process to implement the quick solution. If an issue is identified by the Contractor as non-blocking, the ESA Technical Officer may at his/her discretion, following reasonable criteria, override this assessment and increase the severity level of the issue to blocking. The further steps apply equally to all issues (including blocking problems and other issues): [Req SERV-ISS-220] The Contractor shall identify the root cause of the issue (as opposed to covering just the immediate symptoms) so as to propose the most cost-effective and long-lasting resolution. The root cause of an issue reported may reside in the infrastructure (HW, SW tools, network, storage components, etc.), procedures, knowledge aspects, etc. The Contractor shall determine an efficient resolution i.e. the best compromise between at least the following factors: effectiveness of the resolution, cost and time to implement, preservation of the integrity of the data, conflicts, overlaps or synergies with other open issues, impact on operations, integrity of the system of tools supporting the service (including functional and technical impact cross-impact among tools), functional and technical impact on the infrastructure (HW, SW, tools and storage components) professional standards and best practices [Req SERV-ISS-230] The Contractor shall assess, flag and track any external dependencies between the resolution of this issue and events or decisions beyond the scope of this contract and beyond the decision power of the stakeholders of the issue. [Req SERV-ISS-240] The Contractor shall assess, flag and track any dependencies between the resolution of this issue and other issues, in order to ensure the coherence and alignment of all data and other service components in the resolution of the issue. [Req SERV-ISS-250] The Contractor shall assess, flag and track any similarities to other issues, open and closed, in order to ensure coherence through time in the treatment of similar issues, avoid duplication of work on identical or overlapping issues and take advantage of synergies. [Req SERV-ISS-260] The Contractor shall assess, flag and track any impacts of potential resolutions of the issue, not limited to the scope of the original issue and shall avoid detrimental side effects of the proposed resolution. Page 92/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SERV-ISS-270] The Contractor shall assess, flag and track any conflicts among issue resolutions or between issue resolutions and other drivers, and reconcile them. [Req SERV-ISS-280] The Contractor shall perform the necessary assessment to ensure the tracing: between proposed resolutions and the original issues; among issues and resolutions with similarities, impacts, conflicts or dependencies. [Req SERV-ISS-290] Whenever a resolution of an issue bears upon more than one data set, the Contractor shall: 1. breakdown the issue into autonomous but cross-referenced individual data set-level issues; 2. breakdown the resolution into autonomous but cross-referenced individual data set-level resolutions. [Req SERV-ISS-300] In case the resolution of the issue is not in the scope the baseline service, the Contractor shall evaluate the cost of implementing the resolution. [Req SERV-ISS-310] In case the issue submitted is out of the scope of the service performed by the Contractor (misrouting of an issue): - in the case of an issue submitted by a Customer or Stakeholder of the Service, including EOP-G personnel, the Contractor shall: o endeavour, on a best effort basis, to help the originator of the issue nonetheless; o kindly remind / inform the originator of the issue about the exact scope of the Service; - In the case of an issue erroneously routed to the Service by a third party (eg EOHELP), reject the issue and send it back to the third party following the agreed interface for misrouted issues; in all cases: o inform the ESA Technical Officer; o track the request in the management system in order to propose an enlargement of the Service scope if the same type of request becomes frequently recurrent. 4.4.7.7 Comprehensiveness of the analysis and resolution [Req SERV-ISS-400] The analysis of the issue and the proposed resolution shall cover the end to end chain (from the originator’s perspective) of causes and consequences involved in the occurrence of the issue and the complete removal of negative consequences of the issue. The complete chain may involve several functional and technical aspects linked through a process or system interfaces where side-effects of the issue may have propagated. Page 93/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SERV-ISS-410] Whenever the issue has an impact on the data, the analysis of the issue and the proposed resolution shall cover the correction of all the instances of affected data in the Contractor’s storage. [Req SERV-ISS-420] Whenever the issue has an impact on the data, the Contractor shall propose, on the basis of the records of data delivery and of the projects for which the data concerned in an input, a list of communications to be made about the data issue and the text of the communication, including always SPPA in the list. Upon request of the ESA Technical Officer, the Contractor shall also take care of sending the Communication. In the case of blocking problems affecting the Data Information System or CCM DB, the correction of the problem is to be considered as equally urgent as the correction of data-related problems. 4.4.7.8 ESA Approval [Req SERV-ISS-500] Any proposed issue resolution that involves a significant change to: the architecture or technology of the data storage components; the data model and main functions of the Data Information System and CCM system; shall be submitted to the explicit approval to the ESA Technical Officer in addition to the regular Configuration and Change Management process; in this case the Contractor should propose alternative solutions and make the range of possible options available for consultation by the ESA Technical Officer. ESA may reject the proposed resolution at its discretion based on justified and objective motivations. In case ESA rejects the proposed resolution, the Contractor shall propose an acceptable alternative solution in accordance with the SLA. 4.4.7.9 Issue processing [Req SERV-ISS-600] The Contractor shall prepare and enter the issue in the Service Management System, including all the information related to the issue and necessary for the processing, logging, monitoring and reporting. [Req SERV-ISS-610] Whenever the issue entails a change, the Contractor shall prepare and enter a Data Change Request (DCR) in the Data Change Management system or an Infrastructure Change Request (ICR) in the Infrastructure Change Management System. [Req SERV-ISS-620] Depending on the issue classification, the Contractor shall: (US) need for user support: provide the originator of the issue with the necessary support or information in compliance with the requirements of the corresponding Service processes (PR) problem report: activate the Problem Management process. (CR) change request : . Page 94/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use o o If the required change impacts data configuration items, activate the Data Change Management process (this is typically the case for the guarantee and follow-up portion of the core WP’s) If the required change impacts infrastructure configuration items, activate the Infrastructure Change Management process (RNP) preliminary request for new project: analyse the requirements provided by ESA, design (high level) the solution and create the Project proposal including all required information and a draft Project Plan, in compliance with the requirements of the corresponding Service processes including Project Management. Immediately escalate the matter to the ESA Technical Officer, who will evaluate the proposal and submit to the appropriate approval authorities. (SR) Service request: o If the SR requires a change to a CI (data or infrastructure), activate the corresponding Change Management process o Otherwise, implement the requested action in compliance with the requirements of the corresponding WP. In some cases, the processing of an issue may require a combination of the above actions. For example, a recurrent problem in the infrastructure and related tools should trigger not only the problem management process but also the change management process. [Req SERV-ISS-630] The service shall cover as part of the issue management process all the components, including: the data holdings (data records and mission documentation) the Service Infrastructure, including HW, storage components, middleware, tools, etc. any Data processing software (not including the CFIs) including any custom ancillary scripts the Service Documentation (such as technical documentation, operating documentation) any Configuration and parameters used in the fulfilment of the Service including the operation of the processing systems and other CFI’s. [Req SERV-ISS-640] Whenever the resolution of an issue has an impact on the Level 0 or auxiliary data, the Contractor shall evaluate the impact on the higher levels of data, document it in the Data Information System and notify the Data Librarian. Coherence of the Service Infrastructure and Tools [Req SERV-ISS-650] The Contractor shall keep all the components of the Infrastructure and Service tools and their corresponding documentation fully aligned, coherent and complete. Page 95/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use This covers the necessary documentation updates, including the requirements and specifications of the technical and service documentation, the operational procedures and any training material. 4.4.7.10 Issue follow-up and closure [Req SERV-ISS-700] Upon allocation of an issue to the Service and insertion of the issue in the Service Management System, the Contractor shall automatically acknowledge receipt of the issue to the originator and to the ESA Technical Officer within the time allowed in the SLA. [Req SERV-ISS-710] For issues resulting in changes to the Data holdings, the Contractor shall indicate as part of the acknowledgement of receipt the classification and the contents of the issue/ticket generated, and the planned implementation date. [Req SERV-ISS-720] As part of the follow up of the issues, the Contractor shall notify each change of status of the issue along its defined life cycle to the ESA Technical Officer and to the originator of the issue. [Req SERV-ISS-730] For blocking problems, the Contractor shall continuously (at least twice per NWD) provide feedback on the progress until the implementation of the solution or work-around is completed with success. [Req SERV-ISS-740] For all issues, the Contractor shall set the status of the issue to Solved or Solved with Workaround when, depending on the classification of the resolution: the necessary clarification has been provided to the issue originator; the solution/change has been delivered to Production; the operational procedure requested has been completed and the outcome validated; the Project Proposal for a new project has been delivered. [Req SERV-ISS-750] Upon resolution of an issue (status set to Solved), the Contractor shall automatically notify the originator of the issue and the ESA Technical Officer within the time allowed in the SLA and ask for confirmation that the issue may be closed (set to Closed status). [Req SERV-ISS-760] The Service Desk, as the owner of the issues, shall be responsible for changing the status of the issues to Closed. Final closure of an issue shall only take place when the issue originator has confirmed the closure or 5 NWD have elapsed without an answer. In case the originator of the issue rejects the closure of the issue, the Contractor shall set the status of the issue back to Open. [Req SERV-ISS-770] The Contractor shall perform issue categorization upstream of the processing (category assignment by the Service Desk upon opening of the ticket: “symptom”) as well as downstream (categorization by the Contractor upon resolving the ticket: actual). [Req SERV-ISS-780] The Contractor shall identify in the issue entry and categorization all the details required to perform in-depth analysis of the Service provision. Page 96/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SERV-ISS-790] The Contractor shall automatically escalate to the ESA Technical Officer all issues assessed as severity High (only in the case of blocking problem) and all issues that have exceeded resolution time or other SLA thresholds. As indicated in the Performance Management and Measurement Reporting chapter, the Contractor is required to report periodically (monthly) on key issue Management performance indicators. 4.4.7.11 Issue originator satisfaction [Req SERV-ISS-800] Every issue shall be handled to the satisfaction of the issue originator in terms of courtesy, timelines, accuracy and completeness of the information provided, and compliance with the SLA. [Req SERV-ISS-810] Upon closure of each issue, the Contractor shall enquire about the satisfaction of the issue originator based on a classification with at least 4 levels and register the originator’s level of satisfaction in the service management system for each ticket as part of the follow up of the issue management process. [Req SERV-ISS-820] The percentage (proportion) of issues (tickets) where issue originator feed-back is very satisfied or satisfied by the service shall be no lower than specified in the SLA. As indicated in the Performance Management and Measurement Reporting chapter, the Contractor is required to report periodically (monthly) on issue originator satisfaction. 4.4.7.12 Autonomy in performing the service [Req SERV-ISS-850] Aside from the mandatory interventions of the ESA Technical Officer or issue originator to oversee the service performance, provide further clarification, inputs and validate the outputs, the Contractor shall endeavour to perform the analysis, processing and closure of the issues in full autonomy. It is Contractor’s responsibility to involve the personnel and competence necessary to have the required mastering of the data-related, managerial, functional and technical aspects of the Service. Involving ESA in the analysis or resolution of the issues would denote a problem in the service performance and should be done as a last resort. Involving the issue originator is also discouraged, except in cases where more information is needed on the issue and cannot be obtained without involving the originator. 4.4.7.13 Problem Management While issue management is focused on managing all issues (requests) raised to the service and restoring normal service as quickly as possible including with work-arounds, problem management is focused on the identification and resolution of underlying problems and their root causes. Page 97/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Issue management provides problem management with much of the information it requires in order to identify the existence of underlying problems. [Req PB-MGT-10] For each Problem Report with High severity, the Contractor shall produce within the time allowed by the SLA a Root Cause Analysis (RCA) report including: An executive summary; A Service Impact section, detailing the impact on all service areas affected and, if applicable, the various deviations from the SLA caused by the incident; A Problem description section, where the problem is fully explained in terms of cause and implications at a technical level; Resolution section, where the resolution activities are fully explained at a technical level; Sequence of events, where the chronology of the incident and resolution is detailed; Improvement plan, where the preventative actions on Contractors, ESA and/or third parties are proposed. [Req PB-MGT-20] In case of a Blocking Problem (severity High) the Contractor shall work continuously until the problem is solved, without interruption due to normal working hours and without charging any costs for extra working hours. Note: this is not a requirement for 24x7 service since the raising of the issue is regulated by the NWH. This requirement is about not interrupting the work required to solve a blocking problem once the issue is raised. [Req PB-MGT-30] The Contractor shall provide as part of the monthly Service Review Report details and documentation on ‘known problems’, including known workarounds and solutions, and make them available on the Knowledge Management System. Issue management makes use of this information to provide a quick response/resolution of incidents. [Req PB-MGT-40] Whenever the resolution actions to solve a problem require changes to data configuration items, these changes are implemented under the control of the data change management process. [Req PB-MGT-50] Whenever the resolution actions to solve a problem require changes to infrastructure configuration items, these changes are implemented under the control of the infrastructure change management process. [Req PB-MGT-60] The Contractor shall ensure that the process of Problem Management interacts adequately with the Core Services and Support and Management Services to communicate in particular: - expected problem resolution times; - SLA thresholds and actual resolution times; - Identification of the CI’s impacted by a problem. Page 98/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req PB-MGT-70] The Contractor shall support and record the process of Problem Management with the Service Management System in a way to ensure the constant availability through the Management System of detailed information on the progress of problem resolution and the resulting state of the Service. [Req PB-MGT-80] Except in exceptional cases where the ESA Technical Officer waive the resolution of a Problem Report, the Contractor shall solve all the PR generated by the issue management process as part of the guarantee and follow-up activities of the various core WP’s, within the boundaries indicated for guarantee and follow-up activities, or as part of the support WP’s. [Req PB-MGT-90] Whenever a temporary solution /work-around (eg making available a dataset with a partial coverage if a major problem is detected in the main dataset) has been put in place to respond to a problem, the Contractor shall make available the final solution and perform the removal of the workaround when the final solution is deployed. 4.4.8 Service Infrastructure Configuration and Change Management Service Infrastructure Configuration Management [Req INFR-CCM-10] The Contractor shall produce and apply an Infrastructure Configuration and Change Management Plan to document the activities of configuration and change management applicable to the infrastructure as described in this SoW. The Contractor’s Infrastructure Configuration Management Plan is not a deliverable subject to the approval of the ESA Technical Officer. However ESA reserves the right to examine them on request. [Req INFR-CCM-20] The Configuration Management Process shall unequivocally define and uniquely identify all Configuration Items (CIs), their constituent components for all the components of service infrastructure under the Contractor responsibility and shall define the information to be recorded for each CI. [Req INFR-CCM-30] The Contractor shall produce and maintain a Configuration Item Data List (CIDL), which enumerates the Configuration Items on Services Infrastructure CIs, including: - the HW and SW systems and tools; - network and security components; - any components produced or procured by the Contractor in the scope of the Service and Projects; - any component developed or procured by the Contractor in order to meet a Service or Project requirement and which modifies the data in any manner (eg semi-automatic script for gap-filling, etc.). - the Service Management System and tools, Data Information System and Data Configuration Management System, etc. - all service deliverables as defined in the relevant chapters, including the service documentation and contractual documentation, operating procedures, management and other documentation (eg SID) shared between ESA and the Contractor; - the reports and other deliverables of the Service and Projects as from the initial delivery to ESA. Page 99/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req INFR-CCM-40]The contractor shall include on CIDL an up-to-date map of the dependencies among CIs within and across Infrastructure products. The dependency map shall be updated and delivered at each delivery of a new product/infrastructure component. [Req INFR-CCM-50] The Contractor shall apply formal, state-of-the-art Configuration and Change Management to the Service Infrastructure. Service Infrastructure Change Management [Req INFR-CCM-60] The Service Infrastructure Change Management shall apply to all changes to the infrastructure regardless of the origin: - continued compliance to service availability , performance, capacity requirements; - continued compliance to security requirements; - project requirements; - service improvements; - etc. [Req INFR-CCM-70] The Contractor shall apply formal CCM to the Service Infrastructure as from the KO of the Contract, taking as initial baseline of the infrastructure the reference architecture proposed as part of the bid. [Req INFR-CCM-80] The Contractor shall manage and record changes in a way that all the changes between any two successive baselines can be listed and individually undone. [Req INFR-CCM-90] The Contractor shall manage the life cycle of Infrastructure Change Requests (ICR) based on a detailed ICR generated and managed in the Infrastructure Change Management system. [Req INFR-CCM-100] The detailed ICR in the Change Management system shall contain the necessary information for a formal and rigorous change management of the infrastructure. [Req INFR-CCM-110] The Contractor shall define, maintain, execute and track the life cycle and workflow of the ICR’s. [Req INFR-CCM-120] The Contractor shall take a baseline of the infrastructure used to execute each processing activity in a way that: it is possible at all times to determine what infrastructure was used to produce the output data. it is possible to restore the status of a particular infrastructure product/CIs at the time of the baseline was taken. Infrastructure baselines shall include the full set of infrastructure CIs including the documentation (covering requirements, design, technical and user documentation, procedures, etc) [Req INFR-CCM-130] The Contractor shall determine the impact of the proposed changes. The Change Authority applicable to ICRs shall be an Infrastructure CCB setup and managed by the Contractor. Page 100/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req INFR-CCM-140] The ESA Infrastructure and Security Manager shall be invited as an observer to the ICCB. Note: the ESA Infrastructure and Security Manager may not participate to all the I-CCB meetings but will expect to be fully informed of the agenda and minutes of the meetings. [Req INFR-CCM-150] In order to provide visibility on the infrastructure CCM process, the Contractor shall : - provide to the ESA Infrastructure and Security Manager ESA for information including on-request reports (status accounting) and CMDB extracts; - provide the ESA Infrastructure and Security Manager with permanent on-line read-only access to the configuration data repository. The Contractor’s Infrastructure status accounting reports and CMDB extracts is not a deliverable subject to the approval of the ESA Technical Officer. However ESA reserves the right to examine it on request. ESA reserves the right to perform Configuration or Change verification and audits. [Req INFR-CCM-160] The Contractor shall ensure that the process of Infrastructure CCM interacts adequately with the Core Services and Support and Management Services. [Req INFR-CCM-170] Infrastructure configuration management and control shall be supported by the Service Management System. [Req INFR-CCM-180] The Contractor shall ensure the feedback of the life cycle of the changes from the Change Management to the Ticket Management tool within the Service Management System, so that the latest status of the original issue is always up to date and available on line for users and the other ESA stakeholders. [Req INFR-CCM-190] For each possible change of the Service Infrastructure managed by the service which is required to be compliant with SLA, the assessment, analysis, procurement, build and test necessary to guarantee the delivery of the change shall be part of the service at no extra cost to ESA. 4.4.9 Release management 4.4.9.1 Release Management for Data This section applies to the outputs of the WP’s which elaborate data. The WP of integration of processing systems is a preparatory WP to the elaboration of data by that processing system. As such this section also applies to this WP. The verification activities are described as part of the respective WPs. The process of releasing data starts after the successful completion of the Verification and consists in 3 further steps: data Validation; data Readiness Review data roll-out. Page 101/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 4.4.9.1.1 Data Validation The validation of the data consists of a combination of 3 options : scrutiny of the Verification Test Report; re-run of part of the verification; run of additional tests and validation activities (free tests). The validation exercise will be conducted by an ESA data validation team led by SPPA and including the ESA Project Manager in charge of the data elaboration activities being validated. The ESA validation team may use the services of external contractual entities to perform the validation activities. [Req VAL-DATA-10] The Contractor shall support the validation as described in this chapter. [Req VAL-DATA-20] The Contractor shall support the data validation process by: preparing the Validation portion of the Verification and Validation plan and recording the validation activities and outcomes in the validation portion of the Verification and Validatin report providing access to all the material involved in the exercise, including all the outputs of the activity and of the verification process plus any additional material required by the ESA validation team in relation to the validation; providing explanations / clarifications on the verification report and material in general; re-executing and/or facilitating the re-execution of V&V tests and actions on request of the ESA validation team; executing / facilitating the execution of additional V&V tests and actions on request of the ESA validation team. Note: for brevity, the Validation portion of the Verification and Validation plan is also referred to as the Validation plan. It is however not a separate deliverable but an integral part of the V&V Plan deliverable. Similarly, the Validation portion of the Verification and Validation report is also referred to as the Validation repotr. It is however not a separate deliverable but an integral part of the V&V Report deliverable. The Validation plan and report will be subject to the approval of the ESA Technical Officer. As part of the data validation exercise, SPPA may decide to assign or re-assign the Master flag to the newly validated dataset. 4.4.9.1.2 Data Readiness Review The data shall undergo a RR before its roll-out (deployment) to operations / repatriation to ESA and/or LTDP centres. The RR shall consist in the checking of a checklist based on the following 2 sets of criteria: Common criteria including: Data Information System and CCM DB up to date Successful outcome of the Validation process; Page 102/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Availability and validation of all the deliverables Readiness of the infrastructure including performance and capacity aspects Communication of the new data ready in terms of contents and list of recipients. Data specific criteria. [Req RR-DATA-10] The Contractor shall propose a Data RR checklist (list of readiness criteria) including at least the criteria listed above. The Data RR checklist shall be subject to the approval of the ESA Technical Officer. [Req RR-DATA-20] The Contractor shall ensure that the common and project-specific criteria including those of the above list are fulfilled, and provide the corresponding evidence. [Req RR-DATA-30] The Contractor shall support the ESA validation team in checking / confirming the status of all the criteria in the Data RR list. [Req RR-DATA-40] The Contractor shall record the outcome of the Data RR in the Service Management System and shall record a summary of the outcome in the Data Information System. 4.4.9.1.3 Data roll-out [Req ROL-DATA-10] Upon successful completion of the RR, the Contractor shall roll-out the data to operations by flagging the dataset correspondingly. [Req ROL-DATA-20] The Contractor shall prepare a Data Delivery Note containing at least the associated issue identifiers, the name and contact details of person responsible for the release, date/time of release, the data affected, the reference to the Readiness Review and its outcome and, if applicable, the fall back plan. As from the roll-out, the data becomes usable for operational purposes (eg processing and repatriation). [Req ROL-DATA-30] The Contractor shall submit to the ESA Technical Officer a list of candidate recipients of a communication of the availability of the newly validated data. [Req ROL-DATA-40] The Contractor shall submit to the ESA Technical Officer the draft of a communication to inform the candidate audience of the new availability of data. The text of the communication shall include a summary of the activities which led to the data being validated and rolled-out, as recorded in the Data Information System. [Req ROL-DATA-50] The Contractor shall ensure the feedback of the roll-out process to Data Change Management process and tool. 4.4.9.1.4 Planning of the data deliveries Page 103/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req ROL-DATA-50] The Contractor shall prepare, maintain and implement the Data Delivery Plan, planning the contents and issue date of data to be rolled-out on the basis of: - the planning of the completion and verification activities of all projects - the number and severity of the DCR solved in the to-be-released data - any dependencies among the various datasets to be released - any dependencies between the various activities impacting the various datasets to be released - the indications of the Data Librarian on the importance and urgency of the availability of the to-bereleased data. [Req ROL-DATA-60] The Data Delivery Plan shall be embedded in the Service Monitoring Report and is not required as a self-standing deliverable. [Req ROL-DATA-70] In the case of urgent solutions in response to blocking issues (severity High), the Contractor shall perform the immediate implementation, verification, validation and delivery of the solution independently from the nominal schedule of planned deliveries and in time to comply with the SLA. 4.4.9.2 Release Management for the Service Infrastructure The Service Infrastructure is subject to a standard engineering process for release to operations. [Req INFR-REL-10] The Service Infrastructure evolution, verification and validation shall not interfere with the operational activities, in particular with respect to availability and performance. 4.4.9.2.1 Service Infrastructure Verification [Req INFR-VERIF-10] The Contractor shall prepare and execute an Infrastructure Verification and Validation plan including test scenarios, test cases and test data, covering the issues resolved by the to-be-released update of the infrastructure. The Contractor’s Infrastructure Verification and Validation plan is not a deliverable subject to the approval of the ESA Technical Officer. However ESA reserves the right to examine it on request. [Req INFR-VERIF-20] The Contractor’s verification plan shall cover in particular: - non regression aspects for the components affected by changes; - functionality, capacity and performance of the components affected. [Req INFR-VERIF-30] The Contractor shall generate a Verification and Validation Report listing the tests performed according to the plan, and the test results. The Contractor’s Infrastructure Verification and Validation Report is not a deliverable subject to the approval of the ESA Technical Officer. However ESA reserves the right to examine it on request. [Req INFR-VERIF-40] The Contractor shall appoint for the verification and validation tasks personnel who is not directly involved in the performance of the activities. Page 104/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 4.4.9.2.2 Service Infrastructure Validation The validation of the infrastructure consists of a combination of 3 options: scrutiny of the verification report; re-run of part of the verification; run of additional tests and validation activities. [Req INFR-VAL-10] The validation exercise will be conducted by the Contractor Infrastructure and Security Manager. [Req INFR-VAL-20] The Contractor shall perform the validation process by: re-executing of a selection of verification tests and actions as documented in the V&V plan; executing selected additional tests as documented in the validation plan Documenting the activities, tests performed and outcomes in the Infrastructure Verification and Validation Report. [Req INFR-VAL-30] The Contractor shall provide ESA access to all the material involved in the Infrastructure V&V, including all the outputs of the verification and validation process. 4.4.9.2.3 Service Infrastructure Readiness Review The infrastructure shall undergo an SRR before its roll-out (deployment) to operations. The SRR shall consist in the checking of a checklist based on the following 2 sets of criteria: Common criteria including: Service Infrastructure CCM DB up to date Successful outcome of the Validation process; Availability and verification of all the deliverables Readiness of the infrastructure with respect to performance and capacity aspects Communication of the new data ready in terms of contents and list of recipients. Project specific criteria. [Req SRR-DATA-30] Before the SRR, the Contractor shall produce an instantiated list of SRR criteria based on the criteria listed above. The instantiated SRR checklist is not a deliverable subject to the approval of the ESA Technical Officer. However ESA reserves the right to examine it on request. [Req SRR-DATA-40] The Contractor shall ensure that the common and project-specific criteria including those of the above list are fulfilled, and provide the corresponding evidence to the ESA Technical Officer. Page 105/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SRR-DATA-60] The Contractor shall record the outcome of the SRR in the Service Management System. 4.4.9.2.4 Service Infrastructure roll-out [Req ROL-INFR-10] Upon successful completion of the SRR, the Contractor shall roll-out the Service Infrastructure to operations. [Req ROL-INFR-20] The Contractor shall prepare an Infrastructure Delivery Note containing at least the associated issue identifiers, the name and contact details of person responsible for the release, date/time of release, the components affected and, if applicable, the fall back plan. The Infrastructure Delivery Note is not a deliverable subject to the approval of the ESA Technical Officer. However ESA reserves the right to examine it on request. [Req ROL-INFR-30] The Contractor shall ensure the feedback of the roll-out process to Service Infrastructure Change Management process and tool. As from the roll-out, the updated Service Infrastructure data becomes operational. 4.4.9.2.5 Planning of the infrastructure releases [Req REL-INFR-10] The Contractor shall prepare, maintain and implement the Service Infrastructure Delivery Plan (roll-out to operations), planning the contents and issue date on the basis of: - the planning of the completion and verification activities of projects - the number and severity of the ICR solved in the release - any dependencies among the various components to be released - any dependencies between the various activities impacting the various components to be released - the indications of ESA on the importance and urgency of the availability of the new delivery in production. The Contractor’s Infrastructure Verification and Validation plan is not a deliverable subject to the approval of the ESA Technical Officer. However ESA reserves the right to examine it on request. [Req REL-INFR-20] In the case of urgent solutions in response to blocking issues (severity High), the Contractor shall perform the immediate implementation, verification, validation and delivery of the solution independently from the nominal schedule of planned deliveries and in time to comply with the SLA. 4.4.10 Service Level Management This section contains the information that specifies the level of service required and constitute the SLA between the Contractor and ESA. [Req SERV-SLA-10] The Contractor shall keep the SLA under formal configuration and change control. Page 106/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The defined Service Level Management process aligns ESA needs with the delivery of the services during the Operational Phase. It provides the interface with ESA that allows the other processes to deliver solutions that are in line with the contractual agreements. [Req SERV-SLA-20] The Contractor shall apply a clear and rigorous process to: - define detailed metrics on the basis of the Quality Matrix of this SoW - measure performance against KPIs and performance requirements; - measure contractual penalties incurred for SLA violations; - accurately monitor the adherence to the SLA; - report performance and related information through the Service Management System, ensuring the delivery of the services in line with the contractual conditions. [Req SERV-SLA-30] The Contractor shall be subject to penalties in cases when the agreed Service Levels are not met, as detailed in the Quality Matrix. 4.4.10.1 User Population and Location The user population (see Annex 4) is mostly located in ESRIN. There is no expected impact of the location of the users on the provision of the Service. 4.4.10.2 Data Separation Data under elaboration needs to be distinguished and separated from the data delivered / available for delivery. [Req SLA-ENV-10] Any updates to the data shall ensure a clear differentiation between: - Data undergoing changes (continual improvement, processing etc) - The data which constitutes the current baseline for the given dataset, to be used eg for any day-today request for data repatriation. [Req SLA-ENV-20] During the modification process of the data, the current baseline of the data shall remain available for all the functions of the Service. [Req SLA-ENV-30] After validation of a dataset produced by the Service, the roll-out process (see section on Service Process Management) shall consist in making the newly validated data a baseline available for further tasks including delivery. 4.4.10.3 Networking The SLA and related minimum acceptable values (as laid out in the Quality Matrix, attached as an Annex 3 to this SoW) for each KPI’s include all services related to the network connections procured by the Contractor, security services and working environments. Page 107/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use It is not considered necessary or required for the Contractor’s infrastructure to be connected with a dedicated network link to the ESA network, unless the Contractor considers this necessary for the fulfilment of the requirements. In case the Contractor requires to be connected with a dedicated network link to ESA (ESA network is extended to the Contractor’s site); the network link will be implemented, through ESA network service provider and will be supervised by ESA EOP Security Officer; a direct access by ESA will be ensured for monitoring / retrieving data (e.g ESA Security Officer can decide to disconnect the link in case of emergency). 4.4.10.4 Service Coverage The following definitions apply: Normal Working Day (NWD): Monday to Friday during 52 weeks of a year, with the exception of ESRIN bank holidays (there are 12 bank holidays a year); Normal Working Hours (NWH): From 08:00 to 18:00 (CET) on normal working days; Hours (h): From 00:00 to 24:00 (CET) on 365 days of a year; Extra Working Hours (EWH): defined and agreed as necessary to extend the applicability of the SLA. The NWD and NWH is the standard time during which the service metrics of the SLA apply (in terms of time to close an issue, availability, performance and response time, etc..). However the service may in addition be required for certain Datasets or Tasks for periods of EWH, ie outside the NWD-NWH, either during the night, week-ends or ESRIN bank holidays. If there are ESA requests of EWH, such requests will be managed as extra requirements (not considered part of the baseline service). Such requests will be communicated to the Contractor, normally 4 weeks (and in any case at least 10 working days) prior to the requested period of coverage. Any work outside NWH which may be necessary for compliance with the SLA is included in the baseline service. 4.4.10.5 Service Interruptions [Req SLA-INTR-10] Planned downtime for routine maintenance, upgrades or problem resolution: shall not be performed during NWH/EWH, except for release of authorised emergency change requests. shall be scheduled, as far as possible, on a predefined yearly calendar. Page 108/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SLA-INTR-20] Planned downtime for routine maintenance, upgrades or problem resolution shall be agreed with ESA Technical Officer and in consultation with the User/Customer representatives. [Req SLA-INTR-30] In case of planned or unplanned downtime, the Contractor shall inform the affected users in advance in compliance with the SLA. [Req SLA-INTR-40] The downtime notification shall indicate as a minimum the data, tools, processing or any tasks involved, the affected users, the nature and the duration of the service interruption. [Req SLA-INTR-50] The Contractor shall issue a confirmation note of service restoration after every downtime: to the ESA Technical Officer only in case the interruption was planned and the schedule of the interruption was respected; - to the ESA Technical Officer and the user population impacted in all other cases. 4.4.10.6 Issue Classification 4.4.10.6.1 Typology The following table lists the type classification of the issues to be handled in the Service. Note: the word “incremental” in this table has the meaning defined in the description of the Guarantee and follow-up activities of the various projects (3% /1 year threshold). As a consequence, a request for improvement to a dataset shall be classified as a CR or RNP ( threshold exceeded), otherwise as a SR. The choice of classification between CR and RPN for an improvement above the threshold will be done on the basis of the extent of the change and other situational factors on a case by case basis. Abbreviation PR Term Problem Report Definition and/or Meaning General definition for the classification of an issue as PR: an Issue is categorized as Problem Report (PR) type when an event occurs which is not part of the standard operation of the projects/service and which causes or may cause disruption to or a reduction in the quality of projects and services. In particular, the following events/requests shall be classified as PR: - a problem occurs during the execution of a project (eg a breakdown in the project infrastructure) - a user has trouble obtaining the on line data (e.g. ftp server not available, slow performance) - a user has trouble obtaining the data from a Media (e.g. data not available/readable, etc) - a user has trouble obtaining the on-line data info from a tool under the Contractor scope (e.g. Web Portal, Service Management System, etc..) - the Contractor finds a defect in a CFI provided by ESA – Page 109/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use CR Change request Note: in this case the Contractor manages the PR however the problem is not for the Contractor to resolve - a user finds a defect (e.g. corrupted data, data scientifically incorrect, wrong file name, missing data) in the data managed/delivered by the Contractor - a security attack or breach of ESA Security directives occurs. General definition for the classification of an issue as CR: an Issue is categorized as Change Request (CR) type when the following conditions are all matched: • its resolution requires a modification to any Configuration Item (CI) • a change to the approved requirements is needed for its implementation • the change is not covered by a Service Request (as is the case in the 3% threshold of Guarantee / followup). Note: on the other hand, when the implementation can be performed without any change to the approved requirements, the issue is categorized as of PR (Problem Report) type. The approved requirements are defined by o this Statement of Work, o the individual project requirements, approved project proposals and relevant annexes. In particular, the following requests shall be classified as CR: - a CR is necessary following change in the requirements of a project (eg improved infrastructure of the project for faster processing) - user asks for incremental collection, consolidation, processing of data not covered by the guarantee and follow-up activities of a project (ie above 3% threshold defined in the corresponding WPs a user or the Contractor itself identifies an opportunity for service improvement - the Technical Officer requests delivery/repatriation of data on HDD significantly beyond the threshold indicated in section 4.2.2 US Need for support Page 110/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j user General definition for the classification of an issue as US: an Issue is categorized as User Support (US) type when the resolution involves: - better information to the user; - collection / capture of further information on the data which does not involve modification to the data itself, eg capture of an external mission-related event; ESA UNCLASSIFIED – For Official Use - - - any User management operation (Access Management such as User account creation, change, deletion) elaboration of service data and preparation of ad-hoc reports and statistics for service data analysis and specific objectives any correction of anomalies which resulted from system malfunction and/or user error and not requiring a change in any data configuration Item In particular, the following requests shall be classified as US: - a user asks for information related to a data set - a user finds media or mission documentation or data of unknown usefulness for the Service scope and which needs to be evaluated - a mission-related event is communicated by the mission owner/operator (e.g. satellite stopping the provision of data for a certain period) - support is requested by the ESA Data Librarian and users of the Service - a report of a variety of contents and formats is requested by an ESA Key Role to answer questions and enquiries on the data holding and its evolution RNP Request for new A request expressing a need for new project or request for Project Engineering Services shall be classified as RNP: e.g.: - a user asks for Data collection for a new data set - a user asks for a new project of type data collection - a user asks for a new project of type data consolidation - a user asks for a new project of type data processing - a user asks for a new project of type processing system integration - a user asks to restore a processing capability/environment after a long period (as defined in the project requirements) since the original installation/integration (e.g. from a back-up), including patching and running of a readiness review of the processing system SR A request for the Contractor to perform activities relevant to the Service itself Actions relevant to the service which contribute to the overall management and delivery of the service and are not classified in any of the above categories. This includes actions requested during service meetings. In particular, the following requests shall be classified as SR: Page 111/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - - - - - - - - a user asks for incremental collection, consolidation, processing of data as part of the guarantee and follow-up activities of a project (ie under 3% threshold defined in the corresponding WPs) the ESA Technical Officer requests the sending of HDD to a destination outside the Contract, or any other form of data repatriation / delivery the ESA Technical Officer decides to perform Audit (e.g. Security, service audit) a user mistakenly asks for the handling of an issue that is out of the Contractor scope (misrouted ticket) the DSI-CCB, after consultation with the ESA Data Curator or other competent authority, asks for the destruction of an Original media a user asks for shipping back of Original media previously delivered to the Contractor a user requests the provision of Service reporting and statistics not explicitly required in the SoW a user requests the provision of data reporting and statistics not directly required in the SoW (ad hoc Data Reporting) a user makes any request related to Service process management and support processes managed by the service a user expresses a complaint or a commendation an ESA key role raises an escalation OPS-UNIT/SPPA elevates the status of a dataset to a master dataset a user asks to restore a processing capability/environment within a short period (as defined in the project requirements) since the original installation/integration (e.g. from a back-up), including patching and running of a readiness review of the processing system SPPA (and/or supporting organisations) asks for a processor s/w patch to be installed on an operational processor The management and resolution of PR, US and SR issues is a task which is part of the baseline service (each new occurrence has no extra cost for ESA). Moreover, CR-related activities such as capture, analysis, high level design and estimation necessary to provide estimation/ Impact assessment for the change requests are part of the baseline service (and do not give raise to additional costs). Page 112/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Activities to deploy changes (including the support to validation) shall be part of the Release Management process (baseline service) and, therefore and do not give raise to additional costs. Following an RNP, the analysis and design activities for creating and submitting to ESA a corresponding Project Proposal (including all required information and draft Project Plan) is a task which is part of the baseline service (each new occurrence has no extra cost for ESA). The implementation of the issues related to Guarantee and follow-up activities of projects is not required as part of the baseline service after the expiry of the period of Guarantee and follow-up. 4.4.10.6.2 Severity Type The following table shows the rules for the allocation of a severity level to the issues. TERM Definition and/or Meaning HIGH Total or partial loss of the following critical functions: - access / utilisation of the Data Information System and CCM DB - extraction or delivery of data Negative impact on the successful outcome (quality and schedule) of a project while its root cause is not related to the project. Issues escalated by ESA Technical Officer from Medium to High Security-related issue. MEDIUM Significant degradation of the critical functions listed above, or loss of non-critical function Issues managed with workaround (root cause not yet resolved) after more than a month Issues escalated by ESA Technical Officer from Low to Medium Issues traced to: - processing operational failures not detected during the processing project [Req PRO-MONIT-10] - corrupted / duplicated files in consolidated data [Req CONS-STG0-210] [Req CONS-STG0-510] - gaps in a consolidated dataset not justified in the Technical Note [Req CONS-STG0-120] Page 113/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use LOW All issues not listed in the previous points. [Req ISS-CLASS-10] In case of regression (same issue reoccurring after being solved or new issue created by the resolution of another), the severity shall be increased. 4.4.10.7 Key Performance Indicators (KPI’s) The KPI definitions shall be the basis for the SLA. The KPI and acceptable limit values apply to the end to end service regardless of the Contractor’s chosen approach and organisation of the infrastructure and other components. In particular any interventions of maintenance on hardware, COTS and other components are included in the end to end SLA. The following table describes the classification of KPI in two macro categories: CATEGORY 1 DESCRIPTION The service is observed and measured and the results are reported to ESA. The results concerning the major prominent and critical aspects for ESA are evaluated by ESA together with the Contractor in order to identify areas requiring specific improvements. The values of the KPI are checked for gaps against the minimum allowed value (target values) in order to assess the potential application of penalties. The service is observed and measured, and the results are reported to ESA. 2 The results are evaluated by the Contractor in order to identify areas requiring specific improvements. The KPI’s are measured taking into consideration: the service scope of coverage. The measurements shall always be calculated excluding those factors that are not under the Contractor responsibility (eg service interruptions requested by ESA, third parties components unavailability, ESA internal network unavailability, etc). [Req SLA-KPI-10] The Contractor should ensure that the performance measurement regarding the KPI will be performed on data collected in automated manner and from the Service Management System deployed for the scope of this service. Any exceptions should be agreed with the ESA Technical Officer. [Req SLA-KPI-20] The Contractor shall compute and make available on-line to ESA the actual values of the KPI’s. Page 114/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SLA-KPI-30] The Minimum/Maximum allowed value for KPI’s related to issues shall be based on: Issue Type Issue Severity as defined in the quality matrix. [Req SLA-KPI-40] The Contractor shall implement and operate a comprehensive, state-of-the-art integrated Service Management System to measure, perform, report and assess the KPI’s requested in this SoW. 4.4.10.8 Quality Matrix The Quality Matrix is described in Annex 3. It defines the target value and the minimum acceptable value assigned for all KPIs of the SLA, based on the classification of the issues (type and severity) managed by the Contractor. 4.4.10.9 Penalties Penalties are based on the violation of the Service Level Agreement (allowed values of KPI’s subject to penalty are violated). The penalties, including the financial aspect, will be calculated as indicated in Annex 3. 4.4.10.10 Cancellation Criteria The cancellation criteria are based on the calculation of the KPI and associated penalties, and correspond to a defined scale of penalty-level failure with failure to restore promptly and in a durable manner. The cancellation criteria are indicated in the Annex 3. 4.4.11 Performance Management Measurement and Reporting Performance and customer satisfaction management are critical factors in the success of this Service Initiative. [Req PERF-MGT-10] The Contractor shall measure and record service and project status and performance levels by means of tools and systems integrated with the overall Service Management System. [Req PERF-MGT-20] The Contractor shall measure and record service and project data in real-time. This means that performance shall be monitored as the service / projects unfold and any necessary corrective actions shall be taken while it is time to impact positively on the outcome and not only at the end of the reporting period. [Req PERF-MGT-30] The Contractor shall produce and keep up to date a Performance Measurement and Reporting plan to define and document a list of performance measurements, including at least all the KPI’s, Page 115/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use and for each the full end-to-end process covering the measurement rules, methods and frequency, the data collection and validation and the publication of the measurement results. The Performance Measurement and Reporting plan shall be submitted for approval to the ESA Technical Officer. [Req PERF-MGT-40] The performance measurement processes shall be objective and unambiguous. [Req PERF-MGT-50] The Performance Management Measurement and Reporting process shall apply to the SLA including the actual performance of the projects against the project plans and the approved Project Proposals. [Req PERF-MGT-60] The service level measurements shall be calculated excluding those factors that are not under the Contractor’s responsibility (e.g. service interruptions requested or caused by ESA). In addition to the reports defined in this SoW, further specific reports on particular issues may be requested occasionally by the ESA Technical Officer. The Contractor is also welcome to pro-actively provide additional reports providing a complementary view of the Service and Project performance. 4.4.11.1Satisfaction surveys [Req PERF-SAT-10] Tickets resulting from issues shall be declared Closed only by the issue originator (or in case the originator does not respond either way, by the Contractor Service Manager after 5 NWD, informing the ESA Technical Officer). [Req PERF-SAT-20] Upon closure of a ticket, the system shall require the issue originator to express a level of satisfaction on the treatment of the issue related to the ticket. 4.4.11.2 Service Monitoring and reporting 4.4.11.2.1 Service Progress Meetings and Reports A Service Progress Meeting is held in principle once a week, with the purpose to review and discuss current issues and the action list. [Req PERF-PRO-10] The Contractor shall propose the agenda of the Service Progress Meetings with input from the ESA Technical Officer. One permanent point in the agenda is the action list, managed by the Contractor Service Manager. [Req PERF-PRO-20] The Service Progress Meeting shall be supported by a Service Progress Report delivered by the Contractor Service Manager at least 1 NWD before the meeting and containing: - the current issues; Page 116/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - the action list; any salient point in the Service or Project performance. 4.4.11.2.2 Service Review Meetings and Monitoring Reports [Req PERF-REV-10] A Service Review Meeting is held in principle once a month, with the purpose to: review the main service activities and schedules; review the service indicators and statistics and take appropriate preventive and corrective action against deviations; review the risk situation and risk management actions; review any significant question or salient service point. [Req PERF-REV-20] The Service Review Meeting is supported by a Service Monitoring Report delivered by the Service Manager containing the following reports: - Summary of the activities performed during the period; - Data Status Report, classified by mission / instrument and including: - list of dataset baselines per mission / instrument with for each the list of DCR that it implements - list of DCR per dataset in each of the states of the life cycle defined in the Change Management section - list of dataset baselines with for each the list of CR that it implements and its status; - Data Delivery Plan, planning the contents and issue date of data to be rolled-out - Data repatriation / delivery report - Project Status Report, including: - list of on-going projects - list of salient open points per project, taken from the monthly report of each project - list of closed projects - Demand Status Report, including: - list of projects / activities at each stage of the process - list of forthcoming projects and pre-requisites to kick-off including any blocking points - Performance Status Report, including: - a statistical summary of the issues and tickets handled during the period, showing their status and classification, and the trends over time; the statistics must include a section on the life cycle of the issues and the time elapsed in the various phases (eg time for the user to obtain a CR implementation or PR correction etc.) - performance against service level targets including all KPI’s of the SLA; - list of violations of the SLA, together with an analysis of their root causes, corrective and preventive actions, and recommendation for improvement; - penalties incurred (calculated and reported quarterly); - the list of waivers submitted during the period and their disposition; - the list any Quality alerts occurred during the period and their status; - any early warnings of potential future problems; - any third party interventions; - Infrastructure Status and Monitoring Report - Service Activity Report, including: Page 117/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use handling and outcome of major events, eg inclusion of a new mission, major event in a mission, major problems and changes including traceability through the change life cycle; - workload characteristics and volumes information o issues, problems, changes and all service requests; o volume of data stored and number of datasets and files; o resource utilisation; - analysis of the major trends in the service parameters and KPI’s referred to the SLA; - activities scheduled; - specific actions undertaken to improve service quality and to solve the reported issues; - a summary of the knowledge management activities including a statistical summary of the queries from EOP-G personnel answered and not answered; Risk Management Report, including: - risk register for each project of the risk items identified and assessment of each in terms of likelihood and impact (ESA will provide information on risks identified in previous projects, if available); - the list of preventive actions and risk mitigations; Management Report, including: - an executive summary with the salient points of the other report components; - all indicators used to compute the economic value corresponding to the continuous activities; - economic value of all past, on-going and planned projects; - risk assessment and mitigation actions; - security issues if any (this part of the report shall be omitted if there are no security issues during the period, and shall be delivered in a separate document otherwise); - any other significant service issue. - - - [Req PERF-REV-30] For all SLA violations identified in the Service Monitoring Report, the Contractor shall, prior to the next monthly reporting deadline, propose and initiate corrective actions and preventive actions to avoid reoccurrence. [Req PERF-REV-40] In annex to the Service Monitoring Report, the Contractor shall deliver a TicketProcessing Report (as an Excel file) showing for each ticket at least the following information: - The tracking number assigned to the ticket; - The initial and final categorisation of the ticket; - The priority and severity classification; - The start date/time (the time when the ticket was logged in the Issue Management System); - The end date/time (the time when the ticket was logged as closed in the Issue Management System); - Duration (the absolute time of how long the end-to-end treatment of the ticket lasted according to the Issue Management System); - Description (a brief description of the ticket and solution); - Originator of the issue, stakeholders and Users affected; - Configuration items involved (dataset, tool, processing system, or function involved); - Service affected; - Conformance or violation of the KPI concerned. [Req PERF-REV-50] The Contractor shall deliver the Service Monitoring Report including the TicketProcessing Report and all the accompanying material for a given month by the 5th of the following month. Page 118/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req PERF-REV-60] One month before the yearly Service Review (see below), the Contractor shall deliver an executive summary of the Service Monitoring Report. [Req PERF-REV-70] In addition to the regular reporting, the Contractor shall report specifically to the ESA Technical Officer any blocking problem and shall communicate the evolution of the situation and the progress in the resolution at least twice a day. [Req PERF-REV-80] Notwithstanding the need to perform the service in full autonomy, the Contractor shall report and escalate to the ESA Technical Officer urgent and major issues immediately and independently of the formal reporting instruments. [Req PERF-REV-100] The Contractor shall log all major Service events in an Operational log. Entries to the log shall include, at least: a description of the event; the impact on the service; any proposed changes to the Service. The Contractor shall make the Operational log an annex to the Service Monitoring Report and shall make it available to the ESA Technical Officer on request. [Req PERF-REV-110] The Contractor shall perform all performance management, measurement, report and control on the basis of the Service Management System and ensure that all the related information is constantly updated and kept available. [Req PERF-REV-120] The Service Management System shall include a “service datawarehouse” with a user friendly interface available to the ESA Technical Officer allowing to perform real time monitoring and reporting, interrogation of individual tickets, production of statistics, navigation, etc in order to generate, supervise and validate the Service performance and the reporting described in this section. [Req PERF-REV-130] The Contractor shall ensure that the ESA Technical Officer and other personnel designated by the ESA Technical Officer have permanent read-only on-line access to all areas of the Service Management System including the Service Data Warehouse. 4.4.12 Communication Management This WP consists in communicating noteworthy information to the stakeholders of the Service by means of a web site operated, maintained and populated by the Contractor. In addition, the WP includes the proposal by the Contractor of inputs for the EO weekly newsletter. [Req SERV-COMM-010] The Contractor shall setup and maintain a Service web site to communicate the main aspects of the activities performed in the Contract. The web site shall be considered as the institutional portal to the Service. [Req SERV-COMM-020] The Contractor shall propose a structure, design and contents for the Service web site, to be approved by the ESA Technical Officer. Page 119/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SERV-COMM-030] The structure of the Service web site as proposed by the Contractor shall include a public area including: - a portion dedicated to the description of the services as seen by the Customer; - a portion dedicated to the list of projects including completed, on-going and planned projects with their main objectives, the initial planned completion date (as per project proposal), the actual completion date (for completed projects), the currently planned completion date (for ongoing projects); - a portion dedicated to information and statistics on the repatriation/delivery of data; - a portion dedicated to an extract of the Data Information System and CCM DB; - a portion for service-related events (eg presentations, infrastructure upgrades, useful tips, etc). [Req SERV-COMM-040] The structure of the Service web site as proposed by the Contractor shall include a restricted area including: - operational information on the service (contact points, email address for submitting tickets) - access to the System Management System (for those users allowed to access it) - any other restricted-access function or information. [Req SERV-COMM-050] The restricted area of the web site shall be available to a small (20 to 30) population of EOP-G managers identified by the ESA Technical Officer. [Req SERV-COMM-060] The Contractor shall perform basic user management (attribution of credentials, reset of password, provision of user list, etc) on the users of the restricted area of the web site. [ReqSERV-COMM-070] The Contractor shall keep the information presented in the Service web site up to date with respect to the evolution of the projects and activities of the service, requests from the ESA Technical Officer and external events, with no more than 1 week latency. [ReqSERV-COMM-080] The Contractor shall include in the Service Review Report statistics on the number of contacts to the website, and on the website users’ activities. [ReqSERV-COMM-090] The Contractor shall propose to the ESA Technical Officer the messages to be used for official announcement to the users of main events in the service, and shall take care of the communication after approval of the messages. [Req SERV-COMM-0100] The Service web site (public and restricted areas) shall carry no financial, contractual, security-related, classified or confidential information, and shall not be used for disseminating EO data. [Req SERV-COMM-0110] The Contractor shall propose inputs to the EO weekly newsletter in the form of short articles reporting an achievement or other noteworthy event or situation in relation to the work performed under this SoW. Web based reporting: The following is suggested (but not required) in addition to the above requirements. [Req SERV-COMM-0200] The Contractor should propose a web-based reporting, including contents, format and related processes, with the following features: Page 120/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - cover the performance management reporting (Service Progress Report and Service Review Report) cover the project reporting including progress and issues freeze the status of the reporting at the date that the past reports have fallen due show at all times the latest status of all the items reported be accessible only to a selected list of readers as agreed with the ESA Technical Officer give notification (eg via email) of updates. The web-based reporting, if agreed, will replace all or part of the traditional reporting based on documents and email. 4.4.13 Interfaces to Third Parties The term “third party” refers in this SoW to any organisation which interacts with the service and/or contributes to the fulfilment of the objectives or participates to the operations but is not part of the ESA- or Contractor- component of the service itself. The Third Parties concerned in this SoW are indicated in Annex 4 (see “Roles interfacing the Contractor”). The interfaces between the Contractor and the various Third Parties involve: - ESA personnel working in the various organisational units mentioned in Annex 4. In the same category are also classified the key personnel of the relevant supporting organisations, who act on behalf of ESA personnel. - Organisations supporting ESA via Contractual arrangements, and the services of these organisations. An interface to User support services (EOHELP) is expected to be necessary: while the regular services shall interface a small population of specialised users who will interact directly with the Service’s 2nd line support, the end user population will require support by the Contractor for data generated by the Service and disseminated to end users. This support will be provided not directly to the users but through EOHELP, which is the single interface to the user. Also, in case the option for end user dissemination is activated, support or interaction with individual users will be requested and will take place via EOHELP. The following requirements apply to the Service’s interfaces with third party Contractors’ Services: 4.4.13.1 Interfaces to ESA Personnel The interface of the Contractor with ESA personnel will be formally through the ESA Technical Officer, while operationally direct contacts may be established through email, telephone, meetings etc. [Req INT-KEYP-10] Upon request of the ESA Technical Officer. the Contractor shall establish operational interfaces, based on email, telephone and face-to-face interactions, with key personnel either of ESA or of organisations supporting ESA as necessary for the fulfilment of the Service. Page 121/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req INT-KEYP-20] The Contractor shall establish and maintain a list of the interfacing points, including person, role, organisation, contact details. The list of interfacing points shall be recorded as part of the Service Management System. [Req INT-KEYP-30] The Contractor shall ensure that all the interactions with the operational interfaces are recorded, including at least any actions agreed, decisions made, conclusions reached. The record of the interactions shall be recorded as part of the Service Management System. [Req INT-KEYP-40] The Contractor shall ensure that the issues / tickets / actions resulting from the interfaces are properly recorded and managed in the Service Management System. 4.4.13.2 Interface to Third Party services ESA will endeavour to setup and formalise reciprocal contractual arrangements for service-to-service interfaces with the third party contractors involved. In that case, the following requirements will apply. Coordination of third parties The Contractor shall interface and coordinate any third parties impacting the provision of the end-to-end service from the user/Customer point of view. [Req INT-3RDP-10] The Contractor shall be the end-to-end owner for the resolution of any issues impacting the Services covered by this SoW. Therefore the Contractor shall be responsible to: - manage, coordinate, and supervise any third parties involved in the end-to-end delivery of these services; - monitor the progress of the resolution; - whenever the 3rd party’s contractual SLA is known, check the performance of the 3rd party against its SLA and activate actions of reminder and escalation to ESA in case of violation. [Req INT-3RDP-20] Whenever the resolution of a Service issue becomes pending on the action of a third party, the Contractor shall ensure that: - the 3rd party is notified automatically (eg by email) by the Management System of the Service; - the 3rd party is able to update the status of the issue to “solved” in the Management System of the Service after completing its intervention, through a defined (eg email) interface. The closure of the ticket/issue (switch from “Solved” to “Closed” status) remains the responsibility of the Contractor, after agreement of the issue originator. [Req INT-3RDP-30] The Contractor shall dedicate a section of the monthly Service Monitoring Report to 3rd party interventions including for each 3rd party involved: - the issues / tickets raised to the third party; - the outcome; - the actual performance; - the actual performance with respect to the SLA of the 3rd party, if known. Formalisation of the interfaces Page 122/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The ESA Technical Officer, together with the technical officer of the Contracts of the 3rd parties, will facilitate the establishment and formalisation of the 3rd party interfaces. It is expected that the interface between the Contractor and the supplier of SPPA in charge of delivering and maintaining the processors and QC tools and of performing the quality control on the data will be the subject of a documented interface. [Req INT-3RDP-40] Upon request of the Technical Officer, the Contractor shall establish, jointly with each 3rd party concerned, an interface covering the common processes of the respective services and their points of interaction. [Req INT-3RDP-50] The interface shall be documented in a Service Interface Document (SID) jointly prepared by the two Contractors and approved by the two Technical Officers. The interface with the 3rd parties shall include the communications necessary for the resolution of the issues concerned. The communication channels shall be established in the SID and shall include in principle email, telephone, and access to service management systems. [Req INT-3RDP-60] The Contractor shall keep the SID under Configuration Management (notwithstanding the same operation performed by the other Contractor). [Req INT-3RDP-70] The SID for each service area shall contain: the list of joint processes and operational points of interaction between the 2 services; the list of Roles and Responsibilities of the 2 parties in relation to the interfacing processes, including the respective operational escalation points; the flagging of which interactions impact the end to end fulfilment of the Services specified in this SoW, for which the Contractor will be the owner and will not only interface but coordinate / monitor / remind / urge / escalate the 3rd party; the operational, technical and organisational flows of data and control supporting the interface. 4.4.14 Escalation procedure [Req ESC-PRO-10] The Contractor shall escalate to ESA any problem which has or risks to have a significant impact on the service and projects, in particular: - ESA or third party failing to take action or provide information as and when necessary for the service and projects - failure or expected failure by the Contractor to fulfil the SLA. Contact details of the levels of escalation in ESA will be communicated at Kick-off. [Req ESC-PRO-20] The Contractor shall provide as part of the Service Governance Plan an escalation path to be used by ESA whenever it considers that a problem which has a significant impact on its business activity is not being treated adequately by the Contractor or that the visibility it receives on the treatment of the problem is not sufficient, or when the SLA is violated. Page 123/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req ESC-PRO-30]The Contractor shall make available at least the following roles and escalation levels: Escalation Function Equivalent function in the Contractor’s context Name Address / Location Cell phone nb Email Address Service Delivery Manager Service Quality Manager Account Manager Contract Manager BU Head & Country Delivery Officer Country Quality Manager Chief Financial Officer Chief Executive Officer [Req ESC-PRO-40] The Contractor shall communicate to the ESA Technical Officer an updated version of the escalation procedure table at most 3 NWD after any change in the information contained. 4.4.15 Tools The following is the full set of tools which should support the projects and services described in this SoW. The Contractor is expected to deploy at least those required as mandatory in the rest of the document. Service Management System: Issue Management System / Ticket Management tool; SLA Management tool; Service statistics and reporting management tool; Infrastructure Configuration and Change Management tool; Availability, Performance and Capacity Monitoring tool; Page 124/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Intrusion detection tool; Service web site; Data Information System, comprised of: o Data Information DB/repository; o Data Configuration and Change Management tool including CCM DB. 4.4.16 Project Management The Contractor will be managing projects in relation to: - the project-oriented WP’s, namely: o Data collection o Data consolidation o Integration of Processing Systems o Bulk processing / reprocessing/reformatting operations - other activities which may be required by the ESA Technical Officer to be executed as projects. This section lists the requirements which apply to the management of these projects. Project Management methodology [Req PRO-PM-10] The Contractor shall apply state-of the art project management methodology to its project activities. The preferred methodology is Prince2, however alternatives that may be proposed will be considered. Note: the application of the methodology is expected to be commensurate with the size, level of risk and complexity of each specific project. Project Manager [Req PRO-PM-20] The Contractor shall assign a Project Manager to each project to be conducted within this Contract. The ESA Technical Officer/Project Manager will approve or reject the choice of Project Manager and in case of rejection justify their decision. [Req PRO-PM-30] Each Project Manager proposed by the Contractor shall have following profile: o 5 to 10 years of experience in project management and in managing projects and teams on similar activities; o Skills and experience in data quality assessment and data processing of ESA EO data; o Specific knowledge of ESA EO data (degree or equivalent experience); o Ability to take full responsibility for the project objectives and to deliver to the satisfaction of the ESA Technical Officer and Customer; o Effectiveness in understanding requirements and communicating. Page 125/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req PRO-PM-40] For the execution of all activities specified within a project, the interface for the Contractor shall be the ESA Project Manager. For operational activities the Contractor may be requested to establish working interfaces with other Personnel, affiliated to ESA or to other parties / companies. [Req PRO-PM-50] The Contractor shall not replace the appointed Project Manager during project execution unless authorised by the ESA Technical Officer. [Req PRO-PM-60] The Contractor shall not communicate on project activities to external entities (outside this Contract) without authorisation of the ESA Technical Officer. [Req PRO-PM-70] The ESA Technical Officer reserves the right to request the replacement of a Project Manager. The replacement shall be in place within 3 weeks of the request. Project Proposal / project plan [Req PRO-PM-80] Upon request from the ESA Technical Officer of a Project Proposal, the Contractor shall provide an estimate in terms of schedule and parameters to be applied to the cost model. For any projects were the cost model does not apply, the Contractor shall also provide an estimate in terms of cost. The time allowed for the provision of these estimates is defined in the SLA. [Req PRO-PM-90] The estimate of the Project Proposal in terms of schedule and costs shall be accurate with respect to the actual Project Proposal, with a margin of 20%. [Req PRO-PM-100] Upon request from the ESA Technical Officer , the Contractor shall deliver a full project proposal including a project plan within the time agreed with the ESA Technical Officer. [Req PRO-PM-110] The proposed project plan shall be compliant with the WP inputs and description of this SoW relevant to the project. In particular for processing projects, a major input to the project plan will be the Processing Strategy Document (or equivalent document showing the processing strategy required). The projects performed in the scope of the Contract (eg reprocessing operations) will often cover some of the activities of a more general project (eg reprocessing campaign) carried out in ESA and involving several players and activities, some of which not related to the scope of this SoW. In such cases ESA will normally produce a high level project plan and deliver it as reference document to the Contractor. [Req PRO-PM-120] The proposed project plan shall be compliant with and traceable to the higher level Project Plan which may be produced by ESA to cover the overall project. [Req PRO-PM-130] The project plan shall be commensurate with the nature, dimension and complexity of the project and contain at least the following: - the proposed key personnel including Project Manager, and their time allocation; - the proposed WBS; - any proposed deviations from the requirements of the WP descriptions, and associated justification; - a Financial Section including the calculation of the cost according to the Financial Management requirements (cost model) of this SoW. Page 126/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Project execution [Req PRO-PM-140] The Contractor shall execute a project in accordance with the project plan and keep the plan up to date. [Req PRO-PM-150] The Contractor shall use the Service Management System to track the actions and issues of the project, by means of a project-specific classification of the issues. The project issues shall be fully visible to ESA. Meetings and reporting [Req PRO-PM-160] The Contractor shall report on the progress, issues and other project-related matters in the course of regular meetings on the basis of a meeting schedule agreed in the Project Proposal. ESA reserves the right to call a meeting at any time during the course of the project. Meetings shall nominally be held at ESRIN or through teleconference. [Req PRO-PM-170] The contractor shall deliver a Monthly Project Report at least 5 NWD before the end of each month, including at least: - the progress, issues, risks and achievements per WP; - project inputs to the Service Monitoring Report. Project closure [Req PRO-PM-180] A project shall be considered concluded when all the project activities are validated by ESA, including any activities necessary in order to start performing: - subsequent activities on the outputs of the project (eg. to start processing after the integration of a processing system, etc); - the guarantee and follow-up activities of the project. [Req PRO-PM-190] The Contractor shall prepare and conduct a final presentation for each project conducted within this Contract. [Req PRO-PM-200] The Contractor shall, after the completion of a project, deliver a Project Final Report including Lessons Learnt with suggestions for improvement (see also the section on Continuous Improvement). [Req PRO-PM-210] The Contractor shall facilitate the collection by the ESA Technical Officer of Customer satisfaction information and record the resulting information in the Service Management System. 4.5 Continual Improvement This section addresses the continual improvement of the Service itself. The Contractor team together with the ESA team shall be committed to a programme of continual improvement in recognition of: Page 127/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - the importance of the satisfaction of the Customers of the Service; the innovative nature of the approach, which lends itself naturally to optimisation and improvement. [Req STRA-IMPR-10] The Contractor shall continually collect information targeted to the continual improvement of the Service, from a variety of sources including: - the Customers’ reported level of satisfaction as part of the various projects; - the lessons learnt document generated as part of each project; - the issue originators’ feedback on the closure of the tickets; - the project compliance records in particular on schedule compliance; - the service performance management records in particular on SLA compliance, root cause analysis, corrective action and preventive action; - the Contractor’s internal service records and management reports; - the operational log of major service events; - the regular Service Progress Meetings and Service Review Meetings; - the day-to-day experience of the Service; - any other sources and opportunities. [Req STRA-IMPR-20] The Contractor shall document in the Service Management System the improvement information collected. [Req STRA-IMPR-30] The Contractor shall derive from the improvement information collected concrete improvement actions including: - improvement to the Service operational procedures; - improvement to the Service tools and systems, in particular the Service Management System, Data Information System and CM DB; - improvement to the Service interfaces and sources of information and data; - improvement to the Service technical infrastructure and security; - improvement to the staffing and training processes; - improvement to the Performance targets (KPI’s) and service performance management, measurement and reporting processes, procedures and tools; - improvement to the Service (including Project) requirements and related documentation and deliverables; - improvement to the Service CFI’s and the CFI delivery process; - any other area of the Service amenable to improvement. [Req STRA-IMPR-40] The Contractor shall propose Change Requests to the Service and related infrastructure including: - a coherent set of improvements – eg improvements to the operating procedures and corresponding adjustment of the Service Management System; - an assessment of the expected positive effect of the proposed improvements to the Satisfaction of the Customer and/or the cost of the Service. [Req STRA-IMPR-50] The Contractor shall collect and record Change Requests from the ESA Technical Officer and other key roles for further improvements to the Service. The change requests proposed for the improvement of the Service shall be dispositioned: Page 128/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - by the Service infrastructure CCB for the changes with no contractual impact; by the yearly Service Review for the changes with a contractual impact. [Req STRA-IMPR-60] At the end of each year of Service, the Contractor shall present to the Yearly Service Review : - the list of changes to the services approved and implemented in the course of the year and since the start of the service; - the actual improvements obtained from the changes. Any demonstrated improvement to the cost of the Service, the quality being equal or better, shall be shared evenly between ESA and the Contractor. 4.6 Risk Management The risk management approach to the service provision is part of the service management and is based on an evaluation of the risk factors and for each risk factor an assessment of its likelihood of occurrence and the impact of its occurrence. Performing risk management in compliance with ISO 27001 requirements for Information Security is not required but would be considered an asset. [Req RISK-MGT-10] The Contractor shall systematically and continuously run a risk management program to identify, assess, manage, document, prevent, mitigate and monitor all risks associated with the processes, services and projects contained in or related to the scope of this SoW. [Req RISK-MGT-20] In case either the likelihood or the impact of a risk item is considered medium to high by the Contractor or by ESA, the Contractor shall take prevention and/or mitigation action. [Req RISK-MGT-30] The Contractor shall document: - the list of risk factors, - the assessment of each (in terms of likelihood and impact) - the corresponding prevention and mitigation actions - the resulting effect on the risk level in the Risk Management Section of the Service Monitoring Report, which will be reviewed at the monthly Service Review Meeting. [Req RISK-MGT-40] The Risk Management Process shall cover all the risks, threats, likelihood, potential impact, preventive and mitigation actions related to the service. This includes, but is not limited to, risks belonging to the following categories: Transition risks (e.g. failure to meet the phase-in schedule, disruption of the service when taking over, staffing issues etc.) Service delivery risks (e.g. failure by the Contractor to meet service targets for: quality, user and customer satisfaction, functionality, availability, reliability, flexibility, change management etc.) Project risks Technical risks Page 129/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Security and infrastructure vulnerability risks Financial risks (e.g. supplier financial viability, danger of bankruptcy, changes in ownership structure as a result of mergers and acquisitions) Organizational risks (e.g. supplier major organizational changes, migration of data and support centres etc.) Sub-contractor risks (monopoly situations, lack of required skills and knowledge, common processes and integration) Exit risks (e.g. lock-in with suppliers, human resources issues, assets transfer, knowledge transfer etc.) Disaster risks. The Risk Management Section of the Service Monitoring Report and the associated action list are subject to the approval of ESA. 4.7 Quality management The Quality management requirements are reported in Annex Q. 4.8 Additional Services 4.8.1 Non-standard activities The core WPs described in Chapter 3 follow a standard process, which has many advantages including the predictability of the outputs, the management of the expectations, the gain of economies of scale, the application of a cost model, etc. Activities related to the overall objectives of this SoW may however occasionally depart from the standard WP descriptions of Chapter 3. This section applies to such activities. Additional, non-standard activities may be required (RNP) by the ESA Technical Officer. Other Service stakeholders, including the Contractor, may also propose additional activities. In case of additional activities proposed by the Contractor, the proposal in order to be considered must be justified by an appropriate business case showing convincingly the benefits over the costs. In all cases, any additional activity shall be approved formally by the ESA Technical Officer before the Contractor undertakes it. Due to the non-standard nature of such activities, the process, inputs and outputs applicable will be agreed by the ESA Technical Officer and the Contractor on a case-by-case basis. For the same reason, the cost will not be governed by the cost model as described in Annex M: the Contractor will be requested to provide a specific proposal for the activities in question, based on hourly rates agreed before contract signature. [Req ENG-ACT-10] Depending on the size and complexity of the engineering activity required, as determined by the ESA Technical Officer, the Contractor shall produce an Engineering Activity Proposal reflecting the contents required, which will normally include: Page 130/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Description of the Work Package(s) and activities to be performed, Planning of the activities Required resources (manpower, technical platforms, etc.), Key personnel, Roles and Responsibilities, WP deliverables, Cost. The engineering and managerial approach for the activities will be defined case-by-case in cooperation with and under the authority of the ESA Technical Officer. In most cases the following requirements will apply. [Req ENG-ACT-20] The engineering activity shall be executed under the responsibility of a single project /activity manager appointed by the Contractor, who will be the single point of contact of the ESA stakeholders for the activity [Req ENG-ACT-30] The Contractor shall apply a Project Management approach to carry out the activity in line with the accepted Engineering Activity Proposal and commensurate with the size and complexity of the activity. [Req ENG-ACT-40] Under no circumstances shall the execution of Engineering Activities impact adversely the provision of the service and the execution of the standard projects. [Req ENG-ACT-50] Engineering Activities will normally be performed on a FFP approach based on hourly fees agreed as part of the overall contract. [Req ENG-ACT-60] In the case that activities somehow related to the Service in scope of this SoW are mandated by ESA to an external Contractor in a separate contractual arrangement, the Contractor’s own Engineering Activities shall consist in participating and contributing to the external activity as required by the ESA Technical Officer. The Engineering Activities that will be required vary in content, scope and purpose. The following is a nonexhaustive list of possible examples of such activities: 5 implementation of a given redundancy-reduction strategy following Stage 1 of data consolidation; inputs/contributions to project activities carried out outside the scope of this SoW, including user requirements definition, specifications, architecture and design, interfaces, verification and validation, acceptance, readiness, deployment; small pilot projects to prototype, test, and validate technical aspects in relation to the scope of the Service and Projects; any other activities related to the overall context of the Service. GOVERNANCE Page 131/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use This chapter covers the main guiding principles which drive the management and execution of the projects and of the service. [Req GOV-10] The Contractor shall produce and maintain an overall Service Governance Plan to document the major guiding principles of the management and execution of the projects and of the service. The Service Governance Plan is a deliverable subject to the approval of the ESA Technical Officer. [Req GOV-20] The governance plan shall document in particular, but not be limited to: - The major processes and corresponding flows of information; - The corresponding Roles and Responsibilities, organisation chart and allocation of key personnel; - The interfacing and communication processes within the Service Organisation and between the Service Organisation and other parties; - Escalation procedure. 5.1 Customer and Demand Management The main objective of this Initiative is to improve the satisfaction of OPS-UNIT’s Customers, of which a typology is given in Annex 4 Roles & Responsibilities. OPS-UNIT Personnel will manage directly the relationship with the Customers. However the Contractor has a major role to support OPS-UNIT in the Customer relationship to help increase the satisfaction of the existing Customers and thereby the likelihood of additional business. [Req STRAT-DM-10] The Contractor shall publish and maintain up to date on the restricted part of the online knowledge management system the Service Catalogue of OPS-UNIT. Note: the catalogue of services of OPS-UNIT covers the services in scope of this SoW, with the addition of end-user dissemination which is not the responsibility of OPS-UNIT but is an integral part of the end-to-end value chain of the data. The Contractor Service Manager may occasionally be invited by the ESA Technical Officer to take part in information sessions / presentations given to existing or prospective Customers. [Req STRAT-DM-20] Any contacts between the Contractor and the Customers shall take place at the request or under the supervision of the ESA Technical Officer. In particular, the Contractor shall refer to ESA Technical Officer any enquiry by a Customer / potential Customer. The Demand Management process put in place by OPS-UNIT with its Customers has 3 phases for a project: Phase 1: Intention, where the project is initially broadly defined at an early stage (6+ months before the start) and is still not confirmed Phase 2: Confirmation, where the Customer has made the decision to go ahead with the project Phase 3: Implementation, where the project is funded and is proceeding. Page 132/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req STRAT-DM-30] Upon request of the ESA Technical Officer, the Contractor shall provide the information necessary for the Customer to transition from Phase 1 to Phase 2 and from Phase 2 to Phase 3, for information under its direct control, including: - cost of the project as per cost model; - duration of the project in compliance with requirements; - status / amount / other attributes of the data, for data held as part of the Data Information and Configuration Management WP. [Req STRAT-DM-40] Upon request of the ESA Technical Officer, the Contractor shall endeavour to provide the best estimate of the information necessary for the Customer to transition from Phase 1 to Phase 2 and from Phase 2 to Phase 3, for information not under its direct control, including: - status / amount / other attributes of the data, for data not held as part of the Data Information and Configuration Management WP. [Req STRAT-DM-50] Upon request of the ESA Technical Officer, the Contractor Service Manager shall attend (part of) the relevant Demand Management or related meetings with the Customers and provide additional clarifications. [Req STRAT-DM-60] The Contractor shall pro-actively advise the ESA Technical Officer on matters of demand management, including: - suggested additions / removal / rescheduling of projects; - suggested changes to the scope and contents of projects. [Req STRAT-DM-70] The Contractor shall document the business case of its recommendations on matters of demand management with a clear emphasis on the costs and benefits, centred on the best interests of the Customers and OPS-UNIT. [Req STRAT-DM-80]The Contractor shall maintain and keep up to date a repository of all the past, current and planned projects, including a master schedule of all the projects. The schedule of planned projects shall be based on the indications of the ESA Technical Officer. [Req STRAT-DM-90] Upon request of the ESA Technical Officer, the Contractor shall produce various reports and statistics on the projects at the various stages, in particular per Customer (organisation unit / contact point), budget, type of project, mission, instrument, type of instrument, cost, etc. [Req STRAT-DM-100] The Contractor shall feed demand information into its infrastructure management processes since the early stage (intention) and ensure that any implementation process can start within 1 week of the final go-ahead of the Customer. 5.2 Roles and Responsibilities The ESA and Contractor R&R are defined in Annex 4. In particular, attention is drawn to the need for the Contractor Service Manager to be frequently present in ESRIN for formal and informal meetings with ESA key roles. Page 133/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 5.3 Meetings and Deliverables 5.3.1 Meetings The formal meetings are listed in the various corresponding WPs. [Req MTG-10] The Contractor shall provide: - Proposed agenda points at least 5 NWD before the meeting; - Minutes of all meetings within 5 NWD of the meeting. The MoM are subject to approval by ESA. [Req MTG-20] The Contractor shall ensure the participation of its team to each meeting is adequate to the subjects in agenda and the technical or managerial content of the discussion. [Req MTG-30] Meetings shall be held in principle by video-conference. Upon request of the ESA Technical Officer, the meeting shall be held either in ESRIN or in the premises of the Contractor. Note: physical meetings in ESRIN will normally take place for the KO and the final presentation of projects. 5.3.2 List of deliverables The list of deliverables is summarised in Annex 5. The delivery schedule indicated in the table is a recapitulation of the deliverables of the various WP’s. The delivery schedule should be understood as a default. Deviations may be introduced by both parties following a mutual agreement. Similarly, the table indicates templates for some of the deliverables. Any alternative template proposed by the Contractor will be taken into consideration and a final decision will be taken by ESA. The alternative template proposed has to reflect at least the contents required in the original template, and more if deemed appropriate. The review / approval / acceptance approaches mentioned in the table are described in the next section. [Req DELIV-10] The Contractor shall write all the deliverables in English. [Req DELIV-20] The Contractor shall document all the aspects of the service performance, including the proceedings, meetings, reviews, decision making, schedules and plans and make these records available electronically. [Req DELIV-30] The deliverable documents are delivered in electronic form in MS-Office format without a password. Page 134/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req DELIV-40] If the use of a password or other form of protection mechanism is deemed necessary, the Contractor will agree its usage beforehand with the ESA Technical Officer and the password (or other protection removal mechanism) will be communicated to the relevant ESA interfaces. [Req DELIV-50] The Contractor shall deliver any software developed (eg data handling tools for consolidation etc) in the scope of the contract, including documentation and data, in source form together with the build instructions to produce the executable form. If the Contractor plans to make use of any non-commercial tool (eg. its own proprietary tool), this will in principle be accepted only if a copy of the tool is made available to ESA on a permanent basis and/or the tool produces or exports data in a format that can be imported into a tool that is permanently available to ESA. 5.3.3 Use of deliverables and CFIs [Req OWN-10] The data and the processed data, as well as support service data (including user details, issue information, etc), shall remain the property of the Agency. The Contractor shall provide to the Agency the source code of all of the tools used, of the parameter settings and of any customisation of COTS used, and shall provide the build instructions (including making files), the functional and technical documentation, appropriate training material, and the contents of all the components of the management system (incl. the knowledge management component). [Req OWN-20] Upon request of ESA, the Contractor shall provide in a timely manner and in the agreed format all material and data that is the property of ESA. [Req OWN-30] The Contractor shall retain, backup and take all precautionary measures to preserve data and material under its custody that is the property of ESA. [Req OWN-40] The Contractor shall ensure full compliance with any licensing constraints linked to any of the CFI’s (eg restrictions to the use/ modification/ disclosure of parts of the CFIs, etc). [Req OWN-50] ESA shall be entitled to freely use and reuse the deliverables produced under the Contract and to pass on the deliverables to Third Parties for use or reuse for the Agency’s Own Requirements other than for the Project specified in the Contract, during the Contract and afterwards. The Contractor shall ensure that no restrictions will be imposed on the right of the Agency to provide to Third Parties (e.g., other contractors) the deliverables produced under the Contract, whether for modification, maintenance, updating or any other purpose. 5.3.4 Review and acceptance procedures of the deliverables In all cases the final authority for approval and acceptance of deliverables rests with ESA. The acceptance processes vary according to the nature and criticality of the deliverables. The various processes are described hereunder. Page 135/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The Contractor shall provide the deliverables to the reviewing participants at least 10 working days before the requested feedback date (review or issue of RIDs or comments). This section indicates the process by which the various types of deliverables will be reviewed and accepted. 5.3.4.1 Explicit and implicit approval In all cases, the review and approval may be either explicit or implicit. The implicit approval is granted by default without further communication by ESA if no comments have been received by the Contractor within 10 working days. ESA reserves the right to request changes to a document after its approval (implicit or explicit). Document approval is always implicit unless ESA notifies the Contractor that the relevant approval will be explicit. 5.3.4.2 Review/acceptance of deliverable documents by Formal RID The major deliverable documents will be subject to formal RID-based review. ESA will use RID (Review Item Discrepancy) forms to identify shortcomings in the documents and report them to the Contractor. A formal review meeting will be held to review and disposition each RID. The disposition is discussed jointly, and the final decision will be made by ESA. Whenever a new document or a major change to an existing document is produced, the formal review will be preceded by informal reviews of intermediate but coherent versions eg complete versions of parts of the documents. The informal deliveries are based on written comments and suggestions and email exchange. 5.3.4.3 Review/acceptance of deliverable documents by Formal Comments Formal comments will take the form of bulleted lists of points to be corrected in the deliverable, or in the case of documents of annotations included in the document itself and marked. The response expected from formal comments is either implementation in the deliverable or a discussion in a meeting. Following the discussion, ESA reserves the right to confirm or waive the formal comment. On occasions, ESA may decide to use formal comments to review deliverables for which the formal RID review is the normal review approach. 5.3.4.4 Review/acceptance of deliveries Page 136/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The acceptance of the deliverables is based on the validation approach which is described in the above sections “Data validation” and “Infrastructure validation”. In case the decision on the acceptance is negative, the product will not be accepted and rework by the Contractor shall be required followed by a new verification and validation process. If ESA considers that the PR generated at validation level are of minor importance, the delivery may be provisionally accepted and put in production, subject to the resolution of the remaining PR within a period of time to be set by ESA. 5.4 Yearly Service Review [Req YEAR-REV-10] An annual contractual review shall be conducted with ESA to address and manage the following aspects: financial review, to ensure the Price adjustment of the Service in accordance with the Cost Model, taking into account the additions and removals of Projects from service scope (cost variation linked to volume of activity) and the yearly variation in unit costs (cost variation linked to time / experience) and the deliverables/activities not delivered/executed according to plan, with the effect of: o retroactively adjusting the price of the Service for the past calendar year to reflect the variations to the Projects Portfolio and service scope o setting the unit cost of the Service for the next calendar year based on the composition of the Project Portfolio at the time of the review; this will be subject to adjustment at the next yearly review; SLA and service improvement review to analyse the performance of the previous year (including trends in the KPI’s) and improvements achieved, and to evaluate possible further improvements or cost savings; Contractual review to update any Contractual Documents if needed to reflect the previous points and review/update the payment plan. This may in particular be driven by the Continual Improvement exercise. Note: in the event that any penalties should need to be applied, the corresponding impact will be taken into account in the financial part of the yearly review. 6 FINANCIAL MANAGEMENT The costs charged to ESA are: - the transition (phase-in / phase-out) costs, covering the costs of the initial setup of the service as described in the next chapter; - the costs of the Project activities, as defined in this SoW, which are charged to ESA according to the model described in the next section. Page 137/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Note: Change Requests are conceptually and financially small projects, their implementation follows however a lighter process. The service and support activities are not charged separately and are funded through the Project activities as overheads. Once a project is launched, all the related activities are funded. While the main project activity (eg collection of data) is a one-off task, the other service and support activities give raise to many occurrences of service issues related to guarantee and follow-up, data repatriation / delivery, Configuration and Change Management, Project and Service Management. These repeated occurrences constitute the baseline service. No extra payment is due for each new occurrence of these activities: they are performed under the initial project funding until the end of the guarantee and follow-up period. The guarantee and follow-up period lasts for 1 year after the end of a project and is renewable. In terms of types of issues (see Service Level Management) the baseline service covers Problem Reports, User Support, Service Requests. The provision of project proposals (see [Req STRAT-FIN-20]) is also part of the service baseline. [Req STRAT-FIN-10] The Contractor shall comply with the Financial Management processes and requirements defined in Annex M. In particular, the Contractor shall base the proposed costs of all projects on these required Financial Management processes. [Req STRAT-FIN-20] The Contractor shall respond to any request of the ESA Technical Officer for dedicated projects or engineering activities not corresponding to the standard WP’s of this SoW with project proposals including committing financial proposals based on FFP price types. Note: for any project or engineering activity, related or not to the current scope of activities of the Contract, ESA reserves the right to: - request a proposal to the Contractor and/or alternative suppliers; proceed or not with the project; in case it proceeds with the project, grant the contract to the Contractor or to an alternative supplier. [Req STRAT-FIN-30] The Contractor shall keep a complete and fully transparent record of all the financial aspects of the contract, including the computation of the yearly expenditure per mission / instrument and per project and support /service activity. [Req STRAT-FIN-40] The Contractor shall perform the computation of the payment plans in compliance with the contractual provisions. Page 138/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req STRAT-FIN-50] The Contractor shall prepare all the necessary inputs for the yearly adjustment of costs including the fully transparent calculations. [Req STRAT-FIN-60] The Contractor shall include in the computations of the yearly adjustment of costs any financial penalties resulting from failure to perform according to the SLA. [Req STRAT-FIN-70] The various cost elements of the contract shall be charged as follows: the Transition Phase shall be quoted at : FFP conditions the projects as described in Ch 3 shall be quoted at FUP conditions based on the cost model of Annex M; all the costs of the project and baseline service shall be covered by the cost model and no further cost shall be charged to ESA Any project activity not falling under the types of projects covered by the cost model shall be quoted at FFP conditions based on the contractual daily fees. 7 TRANSITION MANAGEMENT 7.1 Phase-in The Transition Phase is the period used for the service to become operational. The Transition Phase is broken down into: - A phase-in period; - An initial service period. The phase-in is the period of a maximum duration of 4 months in which the Contractor prepares and executes the following tasks: o Acquisition of knowledge o Service rehearsal o Service Management System setup (including all supporting tools and processes) o Agreement with the ESA Technical Officer on the format of all reports o Finalisation of Transition deliverables o Service Readiness Review (SRR) The initial service period (also known as period of grace)of a maximum duration of 4 months is characterised as follows: o The period starts when the Contractor is ready to start the operational phase in accordance with the outcome of the “Service Readiness Review” (SRR) executed at the end of the phase-in o The Contractor is fully responsible to execute the services, without any exception in terms of data/instruments and/or services and processes and/or requirements, unless otherwise agreed; o The SLAs are applicable but the penalty and the cancellation scheme are not applied (i.e. the values of the KPI will be measured and reported but the penalties will not be applied); o The cost model of the operational phase is applied. Page 139/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use At the end of the initial service period, the transition is complete and the SLA and associated penalties are fully applicable. Service Readiness Review (SRR) The objective of the SRR is to demonstrate that the services and associated infrastructure are ready for the subsequent operations. The criteria to be considered in the SRR shall cover at least the following areas: Service overall o All the service components are set up and ready o All the service stakeholders (Contractor and ESA/Third Parties) are trained to the necessary tools / processes etc. o A Service rehearsal has been performed on the service processes and has produced successful results o The Service Management System tools have been setup, tested, validated and are operationally available; in particular this applies to the CCM tools; o The Security criteria are satisfied: Test/validation of the compliance matrix for the security requirements defined in [ADS 1] Security plan list and contents approval by ESA Security authority Vulnerability scanning aimed to ensure that all main vulnerabilities have been patched o Interfaces with Third Parties have been identified and defined (and documented in a Service Interface Definition document where applicable) and are operational Technical o Service infrastructure setup and validated o Availability, Performance and stress tests of the service infrastructure successfully passed Organisation, Deliverables and Documents o Contractor resource allocation defined, agreed and documented in the Transition Plan (name, company, role, location, equipment, roll in-off date, etc) and available for the initial projects; in particular, the allocation of Personnel and other resources from the Contractor and sub-contractors shall be addressed under this point; o Processes and Procedures successfully documented in the corresponding deliverables, executed and tested, including in particular CCM process and operational procedures; o Infrastructure (HW/SW/COTS) support maintenance in place including the corresponding contracts o Transition deliverables (as per Annex 5) accepted and placed under CCM control; o Template and content definitions of all reports and deliverables (as per Annex 5) agreed; o Transition Plan for Phase-in delivered and accepted by ESA and activities / actions complete. [Req PHO-10] The Contractor shall document and submit to ESA for its approval a comprehensive and detailed Transition Plan for Phase-in, which shall lead to the successful fulfilment of the SSR. Page 140/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req PHO-20] The Contractor shall allocate during the Transition phase the key personnel (from the Contractor and any sub-contractors) destined to be allocated during the normal operations. Note: in other words it is not acceptable for the Contractor to appoint a “ transition team” which will be disbanded and replaced by an “ operations team” at the end of the Transition phase –in. [Req PHIO-30] The Contractor shall keep the Transition Plan up-to-date until the Transition is finalized and completed. In case the SRR fails at the end of the 4-month phase-in period, and the Contractor needs an additional amount of time to pass the SRR successfully, the duration of the period of grace (initial service period) will be reduced by the same amount of time. In addition, ESA will then consider the systematic re-routing of projects falling into the potential scope of the contract to other contractual arrangements, until the SRR is passed successfully. On the other hand, early evidence of the successful preparation of the SRR and demonstrated advanced readiness of the fundamental components (infrastructure, CM system) of the service may lead to awarding projects before the final formal completion of the SRR. In particular, processing projects with a processor made available early in 2013 will have a strong pressure in ESA to be executed as soon as the processor is made available. 7.2 Phase-out The contract phase-out is the 4-month period preceding the end of the contract, in the case that the contract is not renewed with the Contractor. Constructive cooperation during the phase-out is a major sign of professionalism. The purpose is to complete any on-going projects and to transfer to the new Contractor the material and knowledge necessary for a successful phase-in. Phase-out tasks and requirements [Req PHO-10] No later than 1 month into the phase-out period, the Contractor shall submit a Transition plan for phase-out to the Technical Officer, covering the phase-out activities. [Req PHO-20] The phase-out activities shall include the transfer to the new Contractor of: - All the open issues (extracted from the Management System) at some agreed cut-off date to be defined in the plan - All the rest of the contents of the Management system and all its components (including the Data Information System and CCM DB) - The full data holding (in the broad sense, including science records, documentation, etc.) Page 141/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - The cooperation with the new Contractor for the resolution of all issues during the phase-out period, including: o A) full visibility on the resolution of issues under the Contractor’s responsibility for at least one month o B) the formal transfer of the responsibility to the new Contractor o C) support to the resolution of issues under the new Contractor’s responsibility for at least one month [Req PHO-21] Upon completion of the transfer of the required material to the new Contractor, the Contractor shall, after confirmation of the Technical Officer, erase its own copy of the data holding in scope. [Req PHO-30] The phase-out activities shall include the delivery by the Contractor to the new Contractor of Knowledge Transfer / Training sessions covering at least the following: - Issue management process (at least 12 hours) - Data management and data CCM process (at least 24 hours) - Roles, responsibilities and interfaces (at least 6 hours) Each Knowledge Transfer / Training session shall last no longer than 4 hours. [Req PHO-40] During the phase-out period, the Contractor should allow the new Contractor read-only access to (selected portions of) the Management System. Phase-out approach and constraints The timing of the phase-out will be arranged in a way that no active projects need to be transferred, ie any on-going projects will be completed (up to the ESA validation) before the end of the contract. No work-shadowing of the Contractor personnel / activities by the new Contractor will be requested (neither at incumbent premises nor ESA premises), in order not to affect the nominal activities. The Contractor is free to propose work-shadowing sessions and a corresponding, commensurate relaxation of the SLA. [Req PHO-50] Except where explicitly specified and agreed by the ESA Technical Officer, the SLA shall remain applicable during Phase-out. [Req PHO-60] Direct communication between the Contractor and the New Contractor is necessary and encouraged, however the Contractor shall document (eg MoM) and CC to the ESA Technical Officer: all communications contributing to the Knowledge Transfer of the Contractor to the New Contractor (eg any presentations, training sessions, Q&A sessions, etc) - all deliveries of material (eg of data, documents) to the New Contractor. [Req PHO-70] Those phase-out activities which involve ESA and/or the new Contractor shall take place at ESRIN and/or by teleconference or videoconference. [Req PHO-80] The Contractor shall facilitate / contribute to the provision by ESA of the facilities (training / meeting rooms, video-conferences, PCs…) needed for the execution of the plan. Page 142/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req PHO-90] The Contractor shall maintain the service organisation and at least 75% of the key personnel allocation stable during the 6 months preceding the end of the contract. 8 ESA UNDERTAKINGS AND CFI’S The CFIs of this contract are the processing systems and the input data, and the related documentation, on a per-project basis. Any other software deemed necessary by the Contractor for the purpose of the service including transcribing, ingesting, inventorying data into the service infrastructure, removing overlap within datasets or extracting data from the service infrastructure e.g. by geographical area is to be provided by the Contractor under its responsibility. The CFI related to the initial basket (Annex 2) will be delivered either at Contract KO or at the start of the corresponding projects. Page 143/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use 9 ANNEXES Annex 1 Definitions and Abbreviations Definitions Baseline service: the baseline service refers to those activities which do not give raise to additional funding or financial management at each occurrence, and are financed through the project activities. The baseline service includes: - the guarantee and follow-up component of each project - all the Support Services, which are: o Data Information and CCM o Data Repatriation / delivery o Service and Project Management o the other Support Services In terms of issue classification, the baseline service includes: - the management and resolution of PR, US and SR (including in particular the activities linked to the Guarantee and follow-up of the projects); - CR-related activities such as capture, analysis, high level design and estimation necessary to provide estimation/ Impact assessment for the change requests; - activities to deploy changes (including the support to validation); - following an RNP, the analysis and design activities for creating and submitting to ESA a corresponding Project Proposal (including all required information and draft Project Plan). On the other hand, the baseline service does not include: - the execution of projects (RNP implementation); - the execution of CR above the incremental threshold of project Guarantee and follow-up (3%) - the implementation of the issues related to Guarantee and follow-up activities of projects after the expiry of the period of Guarantee and follow-up. Bulk-processing: processing of a dataset which has not been systematically processed so far. The need originates from user demand for complete datasets (as opposed to on-demand processing). Customer: The word Customer refers to the internal entity in EOP-G (or other ESA organisation) who requests and funds an activity and to whom OPS-UNIT has to report for the activity. A Customer may delegate his / her authority to a Customer representative. Data completeness: defined as the ratio of files effectively available to the data actually acquired. Data Delivery: in this SoW, data delivery means the act of making data available to a third party, by whatever means including availability in an on-line repository, shipping of HDD, etc. The word delivery is used when the meaning is generic. When the method of delivery is relevant it is specified. Dataset: set of data which: Page 144/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - - have in common at least the following elements of Data Set Description: o mission o instrument and if applicable instrument mode o data level o archiving format were generated by the same acquisition / processing / transcription system. Dataset Description (DSD): set of attributes of a dataset necessary for the proper management of the activities defined in the core WP of this SoW and including: o dataset id o version information o dataset description in terms of composition o dataset data level o archiving format o time span (start and end dates) o completeness and quality attributes (presence of corrupted data, duplicates, overlaps, homogeneity of file naming and file formats etc) o provenance and lineage attributes, including transcriptions, conversions, processing, consolidation etc o mission id o mission and payload description o instrument id o instrument description o references / dependencies / links to other datasets (eg linking aux data and L0 data to the corresponding L1 data, linking a consolidated dataset to the component datasets, etc.) o references / dependencies / links to specific mission documents (eg linking a particular dataset to its format description document) o references / dependencies / links to processors and other systems (eg linking aux data, L0 data and L1 data to the corresponding processing system) o physical information including volume, number and list of data files, type of media, age of the media, identification of each of the media, physical location of each of the copies, procedure to apply and equipment needed to read / transcribe the data, etc. o service information including date of entry into the scope of the service, original media from where the data was transcribed, source, outcome of the original completeness / quality checks, record of operations done as part of the service o master flag (as defined in Annex 1) o any other attribute useful to the full identification and description of the dataset. Note: a dataset (and therefore its Data Set Description) is unique for all the identical physical copies of the data. In particular, in case that a copy of a dataset is made eg for delivery to a third party, the dataset under configuration management is still the instance which is in the Contractor’s storage. A trace is kept of the event that a copy was made and delivered but there is still only one dataset under configuration management. Page 145/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Data quality: fitness of the data for their intended use. The QA4EO (Quality Assurance for Earth Observation) framework defines guiding principles in this context (see http://qa4eo.org). Criteria for the assessment of data quality, discussed in the context of this SoW are mainly the following: - presence of corrupted files in a dataset; - presence of duplications and overlaps; - presence of gaps within the files; - scientific quality resulting from the acquisition eg limits of the visibility of the station / azimuth / incidence angle etc. - quality factors resulting from previous transcription / processing of the data. Exact duplicates: in terms of files from different sources being found redundant during a consolidation exercise, exact duplicates are systematically removed, meaning files with the same title and the same contents bit-by-bit. Any further removal operation (eg files with same contents but different titles) is performed only after approval by SPPA. Job Order file: is the main interface between the Management Layer and a processor. The Job Order file contains the list of input/output files each task of a processor is supposed to use/generate. The Management layer passes the job order as parameter to each task of the processor. Management Layer: Processor tasks are scheduled, orchestrated and controlled by a Management Layer. The Management layer also allows to monitor task execution. Master dataset: the dataset which has been designated by the DSI-CCB as the best available dataset for a given mission / instrument / time period / geographical coverage. The master is the dataset to be used as input for reprocessing exercises. The master dataset label is awarded to a lower level (normally L0 but also auxiliary data) dataset and propagated through processing towards the upper levels, eg a L1 master dataset is the result of processing a L0 master dataset. Typically, a master dataset is the result of a consolidation exercise and is accompanied by a detailed completeness/consistency report which highlights any remaining gaps and inconsistencies. Product: the outcome of a processing. A product may be physically a file or a combination of several files, possibly aggregated into one single resulting archive (tar / zip) file. Lineage: the lineage of the data is the genesis of how a particular dataset has been created initially and evolved through several operations into its current version, including acquisition, processing, consolidation, transcription, etc. Media: any physical support of the data including tapes, HDD, optical discs etc. Orchestration model: set of rules to be applied to the selection of inputs (primary and auxiliary files), sequencing, and correspondence between inputs and outputs for a processor. Implementing the required orchestration model in the operational infrastructure is one most important task of the Integration of a processor. Processing: application to a dataset of a processing system. Page 146/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Processing System: generic name for processor (prototype/IPF), format converter, QC tool or any other tool which is applied to a dataset to generate another dataset. A processing system consists of a collection of executables and tasks. Each task is specified by its input and output. The Contractor will be provided with processing systems as CFI according to this definition. Therefore it is for the Contractor, as part of the Integration of Processing Systems WP, to setup of a full end-to-end processing chain as defined in [Req INTSYST-50] including as applicable the implementation of the processor’s orchestration model (correct feeding of the processor with the best available aux data and chaining of processing steps as per processor documentation). Processor: in this SoW, this term refers to a processing system which produces higher level scientific data (L1, L2…) from lower data (L0, L1…). The development of a processor starts with analysis and assessment of a number of individual algorithms (typically QWG task) that are coded into a processor prototype (typically ESL task) and is further coded into an IPF Instrument Processing Facility by Industry. In many cases it is attempted to leave the implementation to IPF away, to reduce the implementation cycle for a processor and corresponding costs. Reformatting: processing of data, not changing the scientific content but the archiving and/or output format of the data (e.g. L0 conversion from Wilma into SAFE format). Reprocessing: processing of data systematically to Level 1 or Level 2 (e.g. global ocean and land colour data, atmospheric composition data, sea surface temperature data). The need originates from the improvement in the processing baseline of those datasets, as a result of a better understanding of the instrument behaviour, of the geophysical validation activities and of the geophysical parameter retrieval methods. Service Infrastructure = all the means required for the provision of the service and the execution of the projects, including computer power, storage components, software, network connectivity, media, media Read/Write devices, etc. Where applicable (eg for capacity requirements) workforce follows the same requirements as the infrastructure. Service Management System: integrated set of tools (including eg ticket management tool, configuration and change management tool, documentation management system, capacity and performance monitoring tool, etc.) used in the management of the service. The Service Management System is considered part of the Service Infrastructure. Transcription = in this SoW, the word Transcription refers to the copy of data from any source media (tape, HDD, other) to another; this include the copy of data from any input media into the Contractor’s data storage. Transformation = any action on the data including transcription, bulk-processing, reprocessing, format conversion, QC tool, etc. Unit of cost = observable, quantifiable feature of the project which is used as the basis to determine the cost of the project, through the application of a simple formula. Page 147/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Verification & Validation: Verification is the process of reviewing, inspecting, testing, checking, auditing or otherwise establishing that the outputs of the various WP’s conform to the specified requirements and expectations. Verification is performed by the Contractor before delivery of the output to ESA for validation. Validation is the process of establishing that the output of the various WP’s and associated verification conform to the specified requirements and expectations. Validation is performed by ESA after successful completion of the verification by the Contractor and delivery of the output to ESA. Abbreviations CCM CFI CI CIDL CMDB CR DCR DL DSD DSI EO ESL EWH FBR FFP FUP HDD ICD ICR IPF IPR KO LTDP NAS NDA NRT NWD NWH PAC PR Configuration and Change Management Customer Furnished Item Configuration Item Configuration Item Data List Configuration Management Data Base Change Request Data Change Request Data Librarian Data Set Descriptor Data Service Initiative Earth Observation Expert Support Laboratory Extra Working Hours Full Bit Rate Firm Fixed Price Fixed Unit Price Hard Disk Drive Interface Control Document Infrastructure Change Request Instrument Processor Facility Intellectual Property Rights Kick Off Long Term Data Preservation Network Attached Server Non Disclosure Agreement Near Real Time Normal Working Day Normal Working Hour Processing and Archiving Facility, also expected to act as LTDP Thematic Centres Problem Report Page 148/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use QWG RCA RNP SECOPS SAFE SID SLA SoW SPPA SR SRR US V&V WP Quality Working Group Root Cause Analysis Request for New Project Security Operational Procedures Standard Archive Format for Europe Service Interface Document Service Level Agreement Statement of Work Sensor Performance Product and Algorithm Service Request Service Readiness Review User Support Verification and Validation Work Package Annex 2 Initial basket of Projects The attached table shows the initial basket of projects of the Contract. The initial basket of projects is ESA’s current knowledge at the time of issuing the tender of the projects planned for the scope of the Contract during the time span 2013-1015. It specifies the data and processing system concerned and the expected start date. Specific project completion dates are specified together with the individual detailed requirements of each project. Although the initial basket is considered representative of the variety and volume of activities which will fall in the scope of the contract, there is no guarantee that any particular project will be in the scope, as explained in section “Scope of the service”. It is on the other hand expected that additional projects will be added to the scope. The attached tables show: - the initial basket of projects, including the project start and end dates. Note that the dates indicated for the processing projects do not take into account any scientific validation of the processors (to be done outside the scope of this contract). In case the scientific validation of the processor is needed, an additional ~2 months is to be planned for the end to end duration of the project. DSI initial basket project specifications rev10.xls Page 149/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Note: ALOS PRISM and AVNIR-2 bulk processing had originally been considered for inclusion in the initial basket. This has for the moment been removed from the Initial Basket for programmatic reasons. Similarly, Orbview-2 Seawifs has been removed from the initial basket although it may later be added again. Note: for the Landsat bulk processing project, a QC tool will be run as a further processing system after the data processor itself. Note: In some cases, the same processor will be used for several processing projects, therefore only one integration project is necessary, even though of course some tuning of the input data will be necessary between the various projects. Examples of cases of sharing the same processor include: - MERIS FR L0->L1, MERIS FR L1->L2, MERIS RR L0->L1, MERSI RR L1->L2 - GOMOS L0->L1, GOMOS L1->L2 - ENVISAT AATSR L0->L1, ENVISAT AATSR L1->L2 - LANDSAT TM L0->L1, MSS L0->L1, ETM L0->L1 - MOS 1-1b VTIR L0->L2, MESSR L0->L2 Note : the processors indicated for ENVISAT ASAR Wave mode and ERS-1/2 SAR wave mode (marked as TBC) might both be replaced with a new processor (S1-IPF) to be used for all Wave reprocessing activities. This new processor is expected to have similar technical characteristics and similar performance. - the detailed description of the input datasets which will be delivered as CFI DSI Initial basket input datasets rev15.xls - the detailed description of the processors which will be delivered as CFI DSI Initial basket Processor information rev15.xls Note: Future ESA data processors will in principle all be virtualised, i.e. able to run in a virtualised environment while the processor interfaces remain unaltered. Note1: the authoritative information concerning the processors is contained in the accompanying documentation (see attached list). For the convenience of the reader, part of this information has been extracted to the processor information table. However, the reader is referred to the documentation, specifically ICDs, Software Release Notes and System Technical Descriptions as the reliable source of information. Note2: A test data set will be made available at the KO of each processor integration project. It will include on case by case a sample working directory of a specified processing which contains: input sample data, output sample data, sample aux data, sample job order files, sample logs and production reports and sample QC reports. Note3: the source code and makefiles of the processing systems will in principle not be provided, only the binary installation kit for the target operating system will be provided. Page 150/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Note4: some of the processors are undergoing upgrades and the upgraded version will be used for the processing. At this stage the current characteristics of the processor is the best known approximation of the characteristics of the upgraded processor. The attached table lists technical documentation of the processors. These documents are reference documents [RD] to this SoW. DSI Initial Basket Processor and Data Documentation rev09.doc Note: this table is a list of the documents (data-related and processor-related) which will be available for the execution of the projects. Amongst these documents, those which are the most helpful for the bid preparation are published together with the tender documents. Note: further material related to the processors (eg task tables, test data, processing software installation kit) will be made available at the KO of the corresponding projects. Annex 3 Quality Matrix Together with Chapter 4.4.10 Service Level Management, the quality matrix constitutes the SLA of the DSI service. Annex-Quality Matrix rev 05a.xls Annex 4 Roles and Responsibilities This appendix describes the roles and responsibilities of ESA and Contractor players in the key processes related to this SoW. Additional information about roles and responsibilities on the ESA side can be found on-line at www.esa.int and in the EOP-G QMS. The ESA Technical Officer will have the overall responsibility for the activities performed under this SoW. However for the sake of efficiency specific tasks will be delegated to other ESA players, and the same will take place on the Contractor’s side. Operational interfaces will be setup between key players of ESA and of the Contractor. The following roles and responsibilities are involved in the work to be performed. Note: the roles are defined as functions, it is expected that on both sides more than one role will be assigned to the same physical person. Page 151/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use ESA and related players Note: in this appendix and throughout the SoW, code names(MISSION-MANAGEMENT, OPS-UNIT, DISSUNIT) are used to refer to the organisational units which carry out specific functions as defined in this annex. The mapping of the functions to the organisation is subject to change but the functions themselves will remain. The mapping to organisation and points of contact will be communicated to the Contractor at KO. Technical management / supervision of the activities Within the Directorate of Earth Observation Programmes of ESA (D/EOP), the Ground Segment and Mission Operations Department (EOP-G) is responsible, inter alia, for the operations of the Payload Data Segment (data acquisition and recording, standard product generation and distribution, instrument and data quality assurance, archiving and user services, including the day-to-day interface to the users) and the long-term preservation of ESA’s EO data. In this context, the work under this SoW is managed in the organisational unit (OPS-UNIT) which is in charge, within the Ground Segment Infrastructure and Operations Management Division, of the operations management of the end-to-end EO data flow from user requests to delivery of data and products. The roles described in this section contribute to the technical management / supervision of the activities (the contractual management / supervision is performed by the Contracts Officer). ESA Technical Officer The ESA Technical Officer is the ESA representative who has the overall technical authority over the DSI contract. The technical officer is the formal interface for all technical and managerial issues related to the contract, in cooperation with the ESA Contract Officer, who is in charge of the legal/contractual aspects. The ESA Technical Officer has the overall authority for financial matters and for the acceptance of the deliverables and the approval of the corresponding payments. Along with the ESA Technical Officer, other roles will interact with the Contractor for the supervision / management of the work covered in this SoW: the ESA Service Supervisor the ESA Demand Manager the ESA Project Manager the ESA Infrastructure and Security Manager the ESA Data Librarian the DSI-CCB. ESA Service Supervisor Page 152/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The ESA Service Supervisor is the role in charge of: - supervising specifically the service processes, in particular the issue / ticket management, workflow and reporting; - assessing the Contractor’s performance reports and in particular the KPI calculations; - handling from the ESA side any complaints from the users or from Management; - supervising the Contractor’s handling of complaints and non-conformances including the preventive and corrective actions; - assessing the Contractor’s proposals for continual improvement of the service processes and performance. ESA Demand Manager The ESA Demand Manager is the role in charge of: - liaising with the Customers of OPS-UNIT for the scope of services falling into the remit of OPS-UNIT; - recommending which projects will fall into the demand of the Contract, with regard to the nature of the demand, the constraints imposed (eg data or processor IPR’s), the performance of the Contractor, and other factors; - following up the intended Customer projects from the original idea to the confirmation of funding, and making sure that the necessary inputs (eg data, processing system etc) are available at the time of starting the project; - following up with the Customer the Departmental governance process leading to the formal approval of the requested activities; - managing and applying the internal cost model of OPS-UNIT with the Customers in order to determine the overall cost of the activities required; - managing and applying the Contractual cost model between OPS-UNIT and the Contractor in order to determine the cost of the activities required for the part falling under the Contract; - providing the corresponding inputs to the internal Contract plan and liaising with the ESA Budget management process; - providing the corresponding inputs to the ESA Contracts office and liaising with the ESA contractual process; - reporting to the Customers and upper internal Management in terms of activities, progress, performance and funding. ESA Project Manager The core services described in Chapter 3 will be handled as projects. They will be managed from the side of ESA by an ESA Project Manager. There will be one ESA Project Manager for each project; the same ESA Project Manager will manage several projects. The ESA Project Manager is the role supervising end-to-end on the side the delivery, through the Contractor and specifically the Contractor Project Manager, of the project outcomes in compliance with the corresponding WP requirements and Project Management requirements of this SoW, within the time agreed and the cost envelope resulting from the application of the cost model and/or corresponding negotiation. Page 153/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The ESA Project Manager is in charge of supervising the Project activities through the Contractor Project Manager and in particular of accepting the deliverables and approving the payment milestones. ESA Infrastructure and Security Manager The ESA Infrastructure and Security Manager is the role in charge of: - Defining the technical requirements; - Defining, in cooperation with the EOP-G Security Office, the security requirements; - Assessing the Contractor’s proposals for infrastructure for compliance to the technical and security requirements; - Auditing the Contractor’s facilities, possibly assisted by the EOP-G Security Officer or his deputy, for compliance to the technical and security requirements. ESA Data Librarian The ESA Data Librarian is the role collecting and managing the knowledge and information on the data holdings of ESA, in recognition of the fact that the data is the major asset of OPS-UNIT as an organisation, and beyond on EO data in general. This includes in particular: - the definition of procedures and tools for the collection of data-related knowledge and information; - the definition of the data model and requirements of the ESA Data Information System and the supervision of the implementation; - the supervision of the Contractor’s activities in the area of data-related knowledge and information; - the definition of the requirements and the supervision of the Data Information and Configuration Management WP, including acceptance of the deliverables; - the audit of the Contractor’s data storage for compliance to the Data Information System and CCM Data Base, and to the requirements for appropriate storage of the data; - the management of the media furnished to the Contractor as inputs / CFIs and of the media collected from the Contractor as backup data; - the definition and acceptance of regular or on-request reports on the status of the data. In all of the above, data is intended in the broad sense as defined in the section of definitions and including all levels of primary data, auxiliary and ancillary data, mission documentation and processing system information. Data Service Initiative Configuration Control Board (DSI-CCB) The DSI-CCB is the body governing the data under the custody of DSI. It takes any decision concerning this data, in particular the assessment of quality of the data. This includes in particular: approving any action (including destruction) with an irreversible impact on media provided by ESA as an input or CFI to the Contract; approving the release of dataset to any party (inside or outside ESA) and in particular approving a dataset for delivery; Page 154/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use approving the erasure or in any manner the destruction of any data; approving or rejecting the Data Change Requests (DCR’s) raised against the data in the course of the Service; approving the readiness checklist before the start of a processing activity; validating the outcomes of the data consolidation and data reprocessing activities; designating among the various existing datasets for a mission / instrument / coverage which is the master dataset; approving the proposed choices and strategies for data consolidation; in general, providing the ESA Technical Officer with expert opinions on matters concerning the data and guidance on the Service processes. Note: critical decisions with an impact beyond DSI (eg the destruction of input media) will be referred by the DSI-CCB to the Data Curator, the relevant Mission Manager and/or Mission Steering Group who will have the final decision rights. The DSI-CCB will remain the Contractor’s interface and will convey those decisions. The DSI-CCB membership includes: the Head of SPPA or his deputy, who co-chairs the Board; the ESA Technical Officer or his deputy, who co-chairs the Board; the Data Librarian, who acts as the Board secretary and in particular prepares the agenda and records the minutes; other participants on a case by case basis, as required by the co-chairs or as proposed by the Contractor Overall Service Manager and endorsed by the co-chairs. The DSI-CCB shall be the final authority for the matters under its responsibility. Roles interfacing the Contractor The roles described in this section interface the Contractor operationally, are expected to generate issues on the Service and are users (or potential users depending on the nature and operational approach of the interfaces that will be defined) of the Service Management System. ESA unit in charge of Sensor Performances, Products, and Algorithms: SPPA SPPA is the organisational unit within EOP-G (Ground Segment Department) with the role of supplying the Processor and QC tool for the project and validating the outputs from a Scientific point of view. SPPA provides the processors and QC tools which will be integrated and run as processing systems. There is therefore an interface between the Contractor and SPPA (and its supporting organisations) to receive the processor system (including corresponding documentation) and deliver CR/PR as needed. SPPA in addition provides an authoritative contribution to: - the tuning of requirements eg for the consolidation of specific datasets; - the Validation activities and decisions; - the decisions concerning the data (disposition of changes, attribution of the master dataset flag). Page 155/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use There is therefore an interface to deliver output data and information to SPPA. In addition, SPPA also has a role as a Customer (see below). SPPA is supported through contractual arrangements with various entities which will : - Expert Support Laboratories (ESL) develop and maintain the processing baseline for data reprocessing and bulk processing) - Quality Working Groups (QWG) represent the community of scientific end-users of the data - a Service (currently operated by the IDEAS consortium) is in charge of the processors configuration management and other tasks related to data quality. There is one ESL and/or QWG per instrument or type of instrument. For a given project in this SoW there will be typically one interlocutor of the Contractor designated by SPPA. The population of SPPA and supporting entities who will be using the service in general amounts to 15-20 persons. Note: in the case of processing systems other than processors (see Annex 1) and QC tools, for example for data format conversion, the processing system provided as CFI may originate from another unit of EOP-G. ESA unit in charge of Data Dissemination: DISS-UNIT For the purpose of this SoW, a specific organisation unit (DISS-UNIT) is in charge of disseminating the data to the end user community. The nominal approach to disseminating data to the end users is through DISS-UNIT, which manages the user registration process, runs the user helpdesk and is equipped with end-user tools (Single Sign On, user interface for catalogue interrogation and data download, statistics). DISS-UNIT will ask for data to be delivered to the centre designated for the dissemination, typically a PAC. DISS-UNIT relies for interfacing and supporting the end user community on a Helpdesk service (EOHELP). The Contractor will not interface with the end users but may have to interface with EOHELP in order to provide support in cases eg of problems reported by end users in data which was processed by the Contractor. Also, in the case that the option for dissemination of data to end users of this SoW is activated (see Ch. 4.2), the Contractor will be requested to support or interact with end users via EOHELP. A few (2-3) DISS-UNIT officers will be interfacing with the Service. PACs and LTDP Centres The data handled in the scope of this SoW is stored permanently in the Processing and Archiving Centres (PACs) and in the future by Long Term Data Preservation (LTDP) Centres. The physical dissemination to end users is also normally performed by these centres, acting as dissemination facilities. Page 156/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The Contractor will be interfacing the PACs and LTDP Centres for the delivery of the data. The Contractor may also have to interface the PACs and LTDP Centres (and/or the data receiving stations) to collect data and related material. There are currently 8 PACs, and there will be in future a comparable number of LTDP Centres. Other sources and recipients of data In addition to the PACs, LTDP Centres (and occasionally receiving stations), a few (5 to 10) organisations (eg NASA) under various agreements with ESA are providers and/or recipients of data. The Contractor may have to interface such organisations and their personnel in order to receive and/or deliver data. Influencing roles The roles described in this section are among the major influencers of the work performed in the scope of this SoW, although they do not have a direct interface with the Contractor and do not directly generate issues to the Service. End users of the data The end users of the data are the scientific community spread across Europe and beyond. End users are the ultimate beneficiaries of these services: the value chain is complete once the data reaches the users timely and with the proper level of quality; however the data is disseminated to the end users through DISS-UNIT; only in exceptional cases is the Service required to disseminate data to end users (see the WP on Data Repatriation/Delivery). ESA Internal Customer The ESA Customer is the role who commissions, and normally funds, the core services described in this SoW. The major Customers are: - MISSION-MANAGEMENT: Mission Managers who liaise with the user community of the data and manage the best trade-off between the mission objectives and the technical and cost constraints of the mission; - SPPA: officers of the Sensor Performances, Products, and Algorithm Management unit may commission activities (particularly reprocessing); Page 157/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - LTDP Programme: the Long Term Data Preservation programme needs data to be collected, consolidated, reprocessed and curated. Customers may also include other units of the Ground Segment and Mission Operations Department (EOPG), from the Science, Applications and Future Technologies Department (EOP-S) or from other Departments / Directorates of ESA (eg the Directorate of Science). The Customer role interacts principally with the ESA Demand Manager role (for aspects of planning funding and overall performance) and with the ESA Technical Officer role (for aspects linked to a specific project). The interactions with the Customer are handled by OPS-UNIT with the support of the Contractor as requested. LTDP Data Curator One important role in EOP-G is the Data Curator, who is in charge to manage the ESA datasets and associated knowledge. In contrast to the Data Curator, the Data Librarian is the role within OPS-UNIT which interacts with the Contractor for data-related issues, under the authority of the Technical Officer. Contractor Players The roles listed in this section are all considered key personnel roles in the contractual sense, meaning that the identification of the personnel appointed to the roles shall be part of the contract, in line with the SoW, Contractor proposal and subsequent negotiation. Subsequent changes in the appointment of personnel shall constitute a change in the contract and as such shall be subject to a formal CCN. Unless otherwise specified, all the key personnel covering the roles listed in this section are requested to be appointed at least 50% to this contract. It is however understood that the same person may cover more than one role. Each key person in the Contractor’s team is required to appoint a deputy for periods of leave or other absence. Contractor Service Manager The Contractor Service Manager is the person appointed by the Contractor to take the full overall responsibility for the overall provision of the service and is the single formal interface to ESA for all projects, processes and issues. The Contractor Service Manager is responsible for the fulfilment of the objectives of the Service, chiefly the satisfaction of the Customers of OPS-UNIT. For this purpose, the Contractor Service Manager is responsible in particular for: - ensuring the delivery of all the services in scope in compliance to the requirements of this SoW; - representing the Service from the Contractor point of view in all circumstances and acting as the Contractor’s primary interface to ESA for the purpose of the Contract; Page 158/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use - - appointing, deploying, leading, organising, managing and monitoring the resources (human and otherwise) and the means necessary for the provision of the Service in compliance with the requirements and to the satisfaction of the Customers; managing any sub-contractors in a way that the Contractor organisation behaves and appears as a single entity focused on the objectives of the Service, and representing the full entity; interfacing with the sources of the CFI’s (chiefly data and processing systems) to facilitate the successful and timely delivery; interfacing with the users to facilitate the successful and timely delivery of the Service outputs; establishing and operating any other interfaces (eg with third party suppliers) in case of need for the purpose of the Service; preventing, detecting and reporting factors prone to hamper the successful provision of the Service, proposing and undertaking preventive and corrective action, promoting continual improvement; ensuring the reporting and playing a prominent role in service-related meetings. In recognition of the strong Customer orientation of this Contract and in particular of the role Contractor Service Manager, the Service Manager is required to be present in ESRIN on frequent occasions for formal and informal meetings. The Contractor Service Manager interfaces all the ESA roles and is the direct counterpart of the ESA Technical Officer. Contractor Issue Manager The Contractor Issue Manager is the role in charge of managing the service processes, in particular the issue / ticket management, workflow and reporting. The Contractor Issue Manager is responsible in particular for: - capturing, logging, tracing, updating and in general managing the issues raised by the users and by the ESA managers of the Service, taking ownership of all open issues, ensuring that appropriate actions are taken, documented, completed and recorded to resolve the issues; - ensuring compliance with the service requirements of this SoW and the corresponding SLA; - ensuring the availability and usability of the issue management system; - pro-actively identifying issues that are particularly at risk of SLA violation and issues of particular relevance or impact on the Customer satisfaction, taking appropriate preventive action and taking personally the lead on the management of these issues; - providing statistical information and trends upon request to support the SLA management, identifying and proposing improvements to the service processes and opportunities for continual improvement; - making sure that the SLA report is produced and delivered in compliance to the requirements of the SoW; - in case of SLA violation, taking the appropriate corrective and preventive actions, performing the Root Cause Analysis, scheduling and co-chairing the Contractor internal reviews, documenting the end-toend violation management process and submitting the dossier to the ESA Service Supervisor; - in case of user complaint or commendation, performing the root cause analysis, and taking the corresponding corrective, preventive or reinforcing actions; - reporting overall on the Service performance. The Contractor Issue Manager is the direct counterpart of the ESA Service Supervisor. Page 159/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Contractor Data and Configuration Manager The Contractor Data and Configuration Manager is the role is the counterpart of the ESA Data Librarian. The responsibilities of the holder of the role include in particular: ensures that no data is ever lost, destroyed, damaged or otherwise degraded in the life time of the Service. The data is an irreplaceable asset and the most valuable asset of the Organisation. This is therefore by far the most important responsibility of this role and of the Contractor in general; implements and maintains the Data Information System and Data CCM system, processes, procedures and tools, including the Configuration and Change Management plan; ensures the continual population and update of the Data Information System and Data CCM system in a way that continually reflects the actual status of the data, the decisions of the DSI-CCB (together with their rationale and any related trade-off), and any external events relevant to the missions, instruments, datasets and other related entities in the scope of the Service; captures and records knowledge and information about missions, instruments, datasets and other related entities which are currently not in the scope of the Service, as relevant to the context and in anticipation / facilitation of a potential expansion of the service scope; takes direct operational responsibility for the Data Information and Configuration Management WP in compliance to the requirements of this SoW; delivers to ESA as requested knowledge, information and reports regarding the data and corresponding statistics; also provides opinions, advice and recommendations while differentiating clearly between information and recommendations; ensures the publication of controlled levels of knowledge and information on the restricted and open parts of the knowledge management system (data information web site); provides guidance and proposals with corresponding cost / benefit analysis concerning or new projects to be undertaken on the data, in support to the Demand Management; manages in CM the media and documents provided by ESA as CFI’s; ensures, in liaison with the Contractor Infrastructure and Security Manager, that the backup copies and ESA copies of the data are properly generated and delivered to their destinations; liaises with the Contractor Infrastructure and Security Manager to make sure that the data is safely and securely stored; facilitates the audit of the Contractor’s data storage by the ESA Data Librarian. In all of the above, data is intended in the broad sense as defined in the section of definitions and including all levels of primary data, auxiliary and ancillary data, mission documentation and processing system information. The Contractor Data and Configuration Manager is the direct counterpart of the ESA Data Librarian. Contractor Infrastructure and Security Manager The Contractor Infrastructure and Security Manager is the in charge in particular of: deploying and maintaining the infrastructure necessary to fulfil the service and project requirements; Page 160/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use making sure in particular that all the necessary security and environmental precautions are fulfilled to guarantee the reliable conservation of the data and of the media provided as CFI’s; pro-actively facilitating the integration of processing systems particularly in cases of less-than-fully available or compliant documentation of the processing systems; deploying and maintaining the service tools, and ensuring the availability and usability of the tools; ensuring that the technical and security requirements are fulfilled at all times, including the requirements linked to project and service performance; in cooperation with the Contractor Demand Manager, planning the evolutions in the infrastructure necessary to perform the various projects and services with the requested performance, and implementing the necessary variations in the capacity of the infrastructure; facilitating the audit of the facilities by the ESA Infrastructure and Security Manager, possibly assisted by the EOP-G Security Officer. The Contractor Infrastructure and Security Manager is the direct counterpart of the ESA Infrastructure and Security Manager. Contractor Project Manager The Contractor Project Manager is the role taking the end-to-end responsibility for the delivery, of the project outcomes in compliance with the corresponding WP requirements and Project Management requirements of this SoW, within the time agreed and the cost envelope resulting from the application of the cost model and/or corresponding negotiation. The Contractor Project Manager role is in charge in particular of: - planning, leading, organising and verifying the project activities in compliance with the requirements of this SoW and any potential specificities of the project, and producing the Project Plan; - performing the activities and reporting on the progress and completion: - performing the verification and facilitating the performance by the ESA Technical Officer of the validation of the project outputs; - pro-actively managing the project risks. The Contractor Project Manager is the direct counterpart of the ESA Technical Officer. Overview of the principal interfaces The diagram hereunder shows the main interfacing lines, notwithstanding additional lines of interface which will be established operationally. Page 161/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Annex 5 Deliverable Item List (DIL) The attached is the list of deliverables from the SoW. The deliverables are subject to the approval of the ESA Technical Officer. DSI DIL rev23.xlsx Annex M – Cost Model Page 162/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The cost model serves to define the cost of a Project activity on the basis of the cost of the initial basket of projects (see Annex 2) to ensure that the costs are: - predictable, - transparent, ie linked to objective parameters of the project - linked to the level of activity - aligned with the costs of the initial basket. The cost model is implemented in 3 phases: Phase 1: bid phase During the bid phase, the Contractor proposes a FFP quotation for each project in the initial basket (in addition to the FFP for the transition phase). Importantly, each individual project is quoted separately. The Contractor takes into account in the quotation the volume and schedule of the initial basket. For cost analysis purposes, the FFP quote of each project is broken down into the component of cost corresponding to: - the main project itself (from KO to validation); - the guarantee and follow-up period; - the Data info and CCM activities; - the Data repatriation / delivery activities; - the other service and support activities. Note: the Contractor is also requested to provide FUP prices corresponding to daily fees of the following professional profiles, defined for each year of the 3+2 duration of the contract: - Project Manager - Data and processing engineer - Infrastructure / network engineer - Analyst / Programmer - Data processing operator Phase 2: negotiation phase Before the KO, the FFP proposal of the bid is first object of negotiation then is used as the basis for the joint detailed definition of the cost model, through the following steps for each type of project separately: 1. 2. 3. 4. 5. Definition of a Unit of Cost Definition of a method to count the units of cost of a project Application of the method of step 2 to the initial basket Count of the total number of units contained for the particular type of project in the initial basket Calculation of the average cost per unit in the initial basket (expressed as sum of the FFP costs of the projects of this type in the initial basket / total number of units as counted in step 4) 6. Definition of qualifiers if needed, to reflect the fact that certain characteristics of a type of project have a direct bearing on the cost of the project 7. Definition of coefficients to the qualifiers, to reflect numerically the bearing of the qualifier on the cost 8. Finalisation of the cost calculation formula: Cost of a project = f (type of project, number of units of cost, qualifier(s)) Page 163/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use In addition, multiplying factors are agreed in order to reflect: - any variations in the volume of the activities, in comparison to the initial basket - the experience effect whereby the value of the unit of cost is each year lower or equal to the previous year. Note: the formula is calibrated in a way that the FFP cost agreed for a project of the initial basket is also the result of the formula for this project. Phase 3: contractual phase After the KO, the cost of any the projects undertaken (including the projects in the initial basket) is calculated according to the formula defined for the corresponding type of project. At the end of each 12-month period (annual review) of the operational Service, the multiplying factors for cost adjustment linked to volume and time are applied to the previous year’s projects. Volume is counted as the number of units in projects started during the specific 12-month period considered. In the case that a project does not fall into one of the 4 project types for which the model apply (or presents such peculiarities that the model is not considered applicable by both the Contractor and ESA), the project proposal shall be based on the agreed daily fees rather than the cost model. The attached presentation (presented at the DSI Industry Day of 22 June 2012) includes an overview of the working of the cost model. DSI cost model illustration.ppt Annex Q Quality and documentation management The Contractor should run a comprehensive and state-of-the-art quality management system. The Contractor should ensure that the quality management system is certified ISO 9001:2008. A copy of the certificate should be provided at signature of the contract and at each renewal date. The Contractor should provide, on request, evidence of their internal quality plan which should describe the relevant organization, policies, processes & procedures, industry best practices and standards used, supporting tools such as information systems. [Req QUA-10] The Contractor shall provide ESA with the results of any internal audits conducted against the contracted service delivery and any internal or external quality management system audits conducted against their operations, including ISO 9001:2008. Page 164/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req QUA-20] The Contractor shall inform ESA of any change concerning the status of the quality management certification. Quality Audit ESA reserves the right to conduct audits at any point in time, with 3 NWD’s notice, on the Contractor to review any of the processes relevant to the service. [Req QUA-30] The Contractor shall support ESA audits and provide all the requested data to ESA. Configuration management audits shall be regularly performed by the Contractor and shall include the recording of deficiencies, the instigation of corrective actions and the reporting of the outcome. The Contractor shall support the following configuration management audit activities by ESA: Functional configuration audit which ensures there is conformity between the documented baselines and the actual business environment to which they refer; Configuration control audit which ensures that no CI is added, modified, replaced and removed without an approved change request to guarantee the reliability of the CMDB; Configuration status accounting which enables tracking of the status as a CI changes from one state to another in order to control the coherence of the CMDB; Physical configuration audit which ensures the accuracy of the CMDB through a series of reviews and audits to verify the physical existence of the CI and the recording process of CIs. ESA reserves the right to access all Contractor documentation, even if the information is considered proprietary, relevant to the service along with all areas and operations within the Contractor’s facilities in which work is performed. ESA undertakes not to disclose such information to a third party. The Contractor shall maintain an audit plan of internal processes and of the processes of third party providers and on request provide the necessary information to ESA. ESA reserves the right to conduct (either directly or through nominated third parties) technical performance audits at any point in time, with 3 NWD’s notice, on the services provided by the Contractor. Page 165/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use The Contractor shall facilitate and support technical performance and security audits with the aim to assess the performance of each operational service. The audit activities may entail but not be limited to: Perform performance tests; Perform ad-hoc tests for security and network services according to applicable requirements; Compare results of audit tests with baseline performance. Network Security scans Implement corrective actions if required to improve service performance. The Contractor shall allow ESA to perform security audits in order to ensure requirements compliance. ESA security audits will be in line with the ISO 27001:2005 standard. The Contractor shall commit to take appropriate corrective actions (if any) following the outcomes of an ESA audit, and shall generate and implement dedicated corrective action plans. Annex S Security Management Whatever solution is proposed, the Contractor is required to fulfil the requirements in this section. Depending upon the model, approach, architecture, there may be differences in the instantiation of the requirements on the particular solution. For example, a private or a public cloud solution for the service infrastructure is not excluded, as long as compliance with the security requirements is demonstrated. Security requirements [Req SEC-GEN-10] The Contractor shall ensure and demonstrate compliance with [ADS-01] and [ADS-02]. [Req SEC-GEN-20] The Contractor shall implement the requirements and provide the security services of [ADS 1], tailored to the applicable situation as described in section 3.3, and potentially amended according to the Contractor’s proposal and subsequent negotiation. Note 1: It is expected that according to the classification of section 3.3 the Contractor’s network will not be an ESA network extension (non ESA Site). Note 2: Unless otherwise agreed between the Contractor and ESA before the start of the contract, all the requirements of [ADS 1] will be applicable. In case the Contractor considers that any of the requirement of [ADS 1] is not applicable to the particular situation (eg requirements applicable for ESA EO systems or ESA sites), this has to be agreed before the contract signature. Page 166/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SEC-GEN-60] The Contractor shall ensure that the NDA [ADS 2] is signed and respected by its legal representative and by each of the key Personnel of the Contract. [Req SEC-GEN-70] Notwithstanding the above compliance requirements, the Contractor shall not divulge or share any EO or Service data with any third party beyond the required activities of this SoW without the explicit authorisation of the ESA Technical Officer. [Req SEC-GEN-80] The Contractor shall consider [ADS 3] as a requirement document for aspects of data flow, network, allowed protocols to support the design of a network architecture and related implementation, taking into account the provided example of network design compatible to [ADS 1]. [Req SEC-GEN-100] The Contractor should obtain and/or maintain the certification to security Standard ISO 27001. Note: Certification to Standard ISO 27001 will be considered an asset for the Contractor, but it is not required. [Req SEC-GEN-110] The Contractor shall communicate to the ESA Technical Officer any change or confirmation in the status of its certification to ISO 27001. [Req SEC-GEN-140] The Contractor shall appoint an Infrastructure and Security manager who shall be responsible for all aspects of the service related to IT security. [Req SEC-GEN-150] The Contractor shall produce security operational procedures (SECOPS) documented as part of the Service Operations Manual. [Req SEC-GEN-160] The Contractor shall ensure that their service personnel and subcontractors have the appropriate level of awareness, training, physical access, account rights and clearance(s) to perform the required services. Network Security Although the infrastructure requested in this Contract is not requested to be an extension of the ESA network, the Security Requirements are applicable. In particular, the attention is drawn to the following network requirements: [Req SEC-NET-10] With reference to [ADS 1] and [ADS 3], the network implemented for the Service shall consist of different zones, each one characterized by common requirements and functionalities. As minimal implementation, the Contractor’s infrastructure shall be based on a network consisting of at least 3 different zones, mapped to different security levels: Page 167/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use An Internal Zone for storing and processing L0 data A demilitarized zone (DMZ) for delivery and access A zone containing the sensitive data (eg private data of users eg name and address). No physical machine shall be connected to more than one security zone. All Zones can be instantiated as more networks, tailored to the configuration proposed by the Contractor for the infrastructure. [Req SEC-NET-20] The firewall shall detect and prevent any access to Internal Zone from any other network, unless communication is initiated by servers on the Internal Zone. All incoming and outgoing traffic shall be screened by the security elements, which are in charge of blocking unauthorized connectivity, filtering dangerous access or unsupported protocols for outgoing traffic. [ReqSEC-NET-30] DMZ traffic shall be subject to filtering by security elements, limiting protocols and target machines in both directions. DMZ servers shall be prevented to connect to trusted network by firewall rules. [Req SEC-NET-40] All the equipment used for the performance of the Service shall be hosted in the network internal zone, except the equipment which is dedicated to disseminating / publishing data through the Internet. In particular, the storage systems storing the reference copy of data shall be connected to internal network only and publishing of data shall be achieved by copy of data from the storage systems sitting on the internal network to the servers in the DMZ. [Req SEC-NET-50] The DMZ network shall be dedicated to external access to the necessary data and services from Internet, based on standard data access protocols (e.g. http/https, ftp, ftps, ftps, sftp). Allowed protocols are listed in [ADS 3]. [Req SEC-NET-60] External access via terminal protocols (e.g. ssh, telnet) shall be blocked, i.e. no external user shall be allowed to open a terminal session from Internet to any server of both DMZ and trusted networks. Note: notwithstanding the above requirement, the equipment hosted in the DMZ and internal networks may be accessed by remote secure channels eg a VPN and token implementation, to fulfil operational needs and/or allow access by ESA key personnel. Page 168/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SEC-NET-70] A connectivity matrix shall be provided to ESA by the Contractor, allowing to evaluate proper implementation of security rules. The connectivity matrix shall be based on the template contained in [ADS 3]. The connectivity matrix shall include - Connections from / to DMZ machines - Connections from Internet to DMZ networks - Connections from/to internal Networks to DMZ networks The connectivity matrix shall be kept updated and redelivered as part of the Contractor’s quarterly Service Security Report. [Req SEC-NET-90] A secure privileged remote access to the trusted network shall be implemented, eg through usage of a VPN network, based on portable tokens. This access will be used by ESA for monitoring and supervision purposes, and it shall be restricted to a fixed list of machines and protocols. Legal requirements in relation to security [Req SEC-GEN-10] The Contractor shall ensure that all the Science and Service data handled as part of the service shall either be physically stored or kept on the territory of one or more ESA Member States or stored or kept on the territory of a non-ESA Member State whose legislation, however, facilitates easy recovery by ESA of its data. [Req SEC-GEN-20] The Contractor shall not make any use of the data, documentation or any other material entrusted to it as part of this contract for any other purpose than the fulfilment of the Contract. Physical security [Req SEC-PHYS-10] The Contractor shall ensure that only explicitly authorised Personnel has access to the infrastructure and physical facilities used for the purpose of the Contract. [Req SEC-PHYS-20] The Contractor shall ensure that the infrastructure and physical facilities used for the purpose of the Contract are fitted with an alarm and associated procedure able to detect and react to: - unauthorised access to the facilities; - adverse temperature and other environmental conditions. [Req SEC-PHYS-30] The Contractor shall ensure that the infrastructure and physical facilities used for the purpose of the Contract are fitted with uninterruptible power supply. Page 169/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use Defence against security vulnerability [Req SEC-DEF-10] The Contractor shall monitor and update the service infrastructure to detect security vulnerabilities. [Req SEC-DEF-20] The Contractor shall perform security threat analysis of all the Service components and proactive monitoring of all security related services, implement effective reaction/mitigation procedures, and report as part of the Project Progress Meetings. [Req SEC-DEF-30] The Contractor shall monitor the vulnerabilities of the infrastructure announced by the related vendors and suppliers of security services. [Req SEC-DEF-40] In case of a security threat, the Contractor shall: - liaise with the ESA office in charge of security assessments (ESACERT) and/or with the ESA Technical Officer to determine the level of urgency of the security patch; ESACERT and the ESA Technical Officer shall have the final authority in deciding the level of urgency of the patch; - plan and schedule the application of the patch in accordance with the level of urgency; - perform an impact analysis of the patch on the infrastructure and on the Service; - perform regression testing and the necessary changes to eliminate any adverse impact of the security patch on the Service; - report to the ESA Technical Officer within the time allowed in the SLA. [Req SEC-DEF-50] The Contractor shall monitor the accesses and attempted accesses to the Service infrastructure. [Req SEC-DEF-60] In case of a security incident, the Contractor shall liaise with the ESA Technical Officer, the EOP-G Security Officer and ESACERT and/or with the ESA Technical Officer to assign a degree of severity to the attack and propose an action plan. Security incidents shall be handled according to [RDS 4] [Req SEC-DEF-80] The Contractor shall record in the Service Management System: - the security alerts and incidents and their subsequent resolution; - the threats detected and patches installed; - the proactive measures undertaken by the Contractor to improve the security of the Service infrastructure; [Req SEC-DEF-90] The Contractor shall report on security service related issues within a dedicated chapter in the Service Progress Report. Note: Security related reporting has to be considered sensitive information and thus shall be provided only to the ESA TO. Authentication and user management Authentication services are essential for the security of the access of ESA users and Customers to the data and services managed in the scope of this SoW. Page 170/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SEC-AUTH-10] The Contractor shall provide authentication services and control and monitor the access to all the data and services managed in the scope of this SoW. [Req SEC-AUTH-20] The Contractor shall perform the common activities of user management as per relevant operation manual and in accordance with the necessary authorization, including: Account creation, change, deletion, blocking, archiving and recovery; Registration and de-registration of users/groups; Administration of user access rights, privileges and restrictions through defined application and process profiles and security roles; Administration of end user distribution lists used for notification of downtimes, upgrades; Password administration and reset in compliance with the corresponding ESA security policy. [Req SEC-AUTH-30] The Contractor shall ensure that user management section in the operation procedure manual: provides complete information and procedures for Contractor Personnel; is in line with the user functions and roles/privileges within ESA as defined in [RDS 2]. [Req SEC-AUTH-40] The Contractor shall create and allocate personal accounts (ie accounts associated to a single, identified person) as opposed to functional or group accounts, unless explicitly requested by ESA. [Req SEC-AUTH-50] The following account and password policy shall be applied: All accounts have a password; Privileged passwords are not granted except to the Contractor Personnel, ESA Technical Officer; Privileged passwords are changed every 6 months; Privileged passwords are difficult to guess (including by brute-force attack); Privileged passwords are changed when a staff member leaves the Contractor team; Passwords are not physically visible in any way. Antivirus [Req SER-VIR-10] For all the systems used as part of the service, including personal computers used by the Contractor Personnel, the Contractor shall: - Equip the system with anti-virus software; - Ensure that the virus definitions are up to date at all times; - Ensure that the systems are scanned for viruses at least once a week; - Ensure that any virus infection is immediately detected and resolved; - Report on virus-defence activities in the quarterly Service Security Report. Data Security, Integrity and Confidentiality Page 171/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j ESA UNCLASSIFIED – For Official Use [Req SEC-CONF-30] Unless otherwise specified, the Contractor shall consider all data, documentation and information produced and provided by ESA as proprietary information which may only be used within the scope of the contracted activities. [Req SEC-CONF-50] The Contractor shall retain the logs of user accesses and activities for a retention period of no less than 12 months. Security audits ESA reserves the right to perform Security Audits. Page 172/172 SoW - DSI Date 19/11/2012 Issue 1 Rev 12-j
