Copernicus Space Component ESA Data Access Overview
J. Martin (ESA), R. Knowelden (Airbus D&S)
• Introduction and Context
• Operational Scenarios
– Translation into interfaces
– Translation into services
• Current Data Access Landscape
– Introduction to Evolutions
• Feedback Process
The CSCDA System – Introduction
•
•
The Copernicus Space Component Data Access (CSCDA) System provides comprehensive and coordinated access to EO data products from multiple satellites to Copernicus space data users across Europe.
It allows the capacity:
– To link transparently the different EO Data Providers and the various Copernicus Service Projects (CSPs) using coordinating functions.
– To create synergy and sustainability across the various Copernicus Contributing Missions (CCMs).
– To facilitate the data access for the services and aim at long‐term data preservation and access.
•
•
The CSCDA manages the coordinated production of Data Sets: multi‐
mission coherent data collections pre‐defined according to specific user needs in terms of data type and mode of operation. They are produced under pre‐agreed schedules or as an ad hoc response to user requests.
A key component of the CSCDA is the Coordinated Data Access System (CDS), ensuring Data Set construction and dissemination by coordinating the CCM’s production and dissemination flows.
Data Flows from CCMs to Copernicus Users
End‐to‐End Operational Scenarios
ARCHIVE RETRIEVAL
REPROCESSING
SCE‐02
Regional Monitoring (direct flow)
SCE‐03
Regional Monitoring (via CDS)
SCE‐04
Geographic Area Coverage
SCE‐05
Background Acquisition
SCE‐06
Rush Satellite Tasking
SCE‐07
Crisis Monitoring
SCE‐08
Rush Retrieval from CCM Archives
SCE‐09
Normal Retrieval from CCM Archives
SCE‐10
Direct Retrieval from Archives of Global Systematic Missions
SCE‐11
Consolidated Data Streams from Global Systematic Missions
SCE‐12
Reprocessing of Historical Series for Global Systematic Missions
SCE‐13
Direct Retrieval from CDS Coordinated Archive
SCE‐14
Off‐Line Mapping in Non‐Rush Mode
Emergency
Data Streams from Global Systematic Missions
On‐Demand
SCE‐01
Predefined
TASKING
CSCDA Scenario
Scenarios Demonstration
• Demonstrations are provided of two scenarios. These provide a good overview of CSCDA functionality and interfaces, and the tools used, showing both future tasking and archive retrieval in an emergency context.
1)
2)
•
•
•
Scenario 6: Rush Satellite Tasking
Scenario 8: Rush Retrieval from CCM Archives
Both scenarios are triggered by requests from an authorized user for archived and tasked data, which are received and processed by a CDS/REACT (Rapid Emergency Activation for Copernicus Tasking) Operator.
The scenario uses as its basis a hypothetical flooding event in central Europe.
NB: only the principal steps are shown.
Scenario Initiation
Emergency
Service
REACT Operator
Flooding Occurs
User requests satellite data by completing and submitting online SPERF forms
Reception of SPERF forms. Creation of sub‐data sets.
Possible missions selected.
Propagation to CDS subsystems.
CCM
SCE‐06: Rush Satellite Tasking
Emergency
Service
REACT Operator
CCM
GEST tool used to create emergency dossier and identify candidate CCM(s) Emergency Order forms sent to selected CCMs Optional check with user
Feasibility check and final confirmation
Order Status monitoring
Retrieval of Data Product
Order receipt and acknowledgement. Feasibility analysis.
Mission planning, acquisition, processing
Data dissemination on ftp pick‐up point
Catalogue, archiving. Placement of data in Emergency Dossier
Circulation to CDS
SCE‐08: Rush Retrieval from CCM Archives
Emergency
Service
REACT Operator
CCM
GEST tool used to create emergency dossier and search for candidate archived products via HMA Catalogue Interfaces
HMA Emergency Order sent via HMA Ordering Interfaces
Order Status monitoring
Retrieval of Data Product
Order receipt. Product retrieval & processing (if required)
Data dissemination on ftp pick‐up point
Catalogue & Archiving. Placement of data in Emergency Dossier
Circulation to CDS
From Scenarios to CDS Interfaces
• Definition of Interface Standards towards CCMs
• Favouring re‐
use of HMA / OGC and other standards
From Scenarios to a Service Charter
The CSC Ground Segment provides a set of main services to all users,
independently from the access infrastructure
+ Research Support Service and Hosted Processing Service: Support prototyping of new algorithms on a dedicated platform ‐ new service under definition
19/03/2013
11
Current Copernicus Space Data Access Landscape
Open Access – The Scientific Data Hub
• Initial Operations:
– Self Registration and Sample Products open since S‐1A launch 04‐2014
– Routine Data flow opened following IOCR 10‐2014
– Rolling Archive • Initial 12 months data population to support Satellite Ramp‐Up phase
• no timeliness guarantee
– Quota restriction of 2 concurrent downloads to ensure bandwidth availability for all users
– Go to http://scihub.esa.int
Hub Access – A common solution for – Open / Collaborative/ International ‐
• Simple UI
• Powerful API
– Cross platform and compatible with many programming styles
• Script, Java, App …
–
–
–
–
RESTFul / HTTP
Simple Authorisation
OpenSearch
OpenData
Sentinel‐1 User and Data Statistics
(open access data hub)
By 26 March 2015:
 5979 registered users
 77536 products available for download
 543211 products downloaded by users,
representing about 680 TB of data
Roadmap for Data Hub
• New user interface under development to better support multi‐Sentinel context
• S‐2 support in line with satellite commissioning • S‐3 support in line with satellite commissioning • S‐5P support in line with satellite commissioning • Open Source – software release via public repository in June 2015
Current Copernicus Space Data Access Landscape
Dedicated Data Access for Copernicus • Dedicated services for eligible users of Data Warehouse Datasets
• Point of entry via https://copernicusdata.esa.int
• Subject of major evolutions in 2015
How to access the data today?
FTP for large volumes (with group account) Copernicus Client (Eoli‐SA) for single products (with user account)
Introduction to Evolutions
DataWareHouse 2.0 Requirements
New CCMs/CCMEs
Parallel/Related Technology Developments for CDS
Continuity with current baseline
CDS v2.1
INSPIRE
CDS v3.x
Feedback from Routine Operations & End Users
HMA Interfaces Extension
Additional requirements captured during the engineering reviews
20
Data Access Interface Evolutions
• Core data access services offered through a uniform web‐
based interface, supported on major browsers
• Comprehensive DISCOVERY, VIEW and DOWNLOAD services provided towards full archive
• Friendly to Web and OGC standards for Earth Observation
• Detailed product image as map layers (in selected cases)
• Sharable shopcarts, dataset search, browse and product URIs
• Systematic user‐defined downloads using performant
download manager
• Progressive deployment for S‐1, 2, 3 and CDS from Q2 2015
Data Access Interface Evolutions
Feedback processes
• To helpdesk: eosupport@copernicus.
esa.int
• To Account Managers, copernicus.account.ma
nager@esa.int via feedback in questionnaire
• Via workshops
Thanks for your attention!
User REGISTRATION
User ACCESS MANAGEMENT
HELP DESK Service
OFFLINE MAPPING Service
EMERGENCY Service
BULK DISSEMINATION Service
Coordinated
Access ‐
Continual Improvement
DISCOVERY Service
VIEW Service DOWNLOAD Services