The GEOSS Data Sharing Working Group: Report to 27th Executive Committee Meeting Alan Edwards GEOSS Data Sharing Working Group Co-Chair Geneva, Switzerland 19-20 March 2013 DSWG Report to EXCOM-27, 19/03/2013 GEOSS Data Sharing Principles The GEOSS 10-Year Implementation Plan states: “The societal benefits of Earth observations cannot be achieved without data sharing.” And then sets out the GEOSS Data Sharing Principles: There will be full and open exchange of data, metadata and products shared within GEOSS, recognizing relevant international instruments and national policies and legislation. All shared data, metadata and products will be made available with minimum time delay and at minimum cost. All shared data, metadata and products being free of charge or no more than cost of reproduction will be encouraged for research and education. DSWG Report to EXCOM-27, 19/03/2013 Please note: There is NO GEOSS Data Policy The GEOSS Data Sharing Implementation Guidelines confirm: “GEO welcomes all data contributions into the GEOSS. When registering data in GEOSS, the contributor should present any restrictions arising from relevant international instruments and national policies and legislation, and the duration of each restriction, that is applicable to the exchange of the data, metadata, and products submitted”. DSWG Report to EXCOM-27, 19/03/2013 Implementation Guidelines for the GEOSS Data Sharing Principles (DSP) Accepted by the GEO-VI Plenary, Washington D.C., Nov. 2009 • Promote the implementation of the principle of full and open exchange of data according to GEOSS Data Sharing Principles (DSP). • Enable GEOSS users to reuse and re-disseminate shared data, metadata and products. • Ensure consistency in the implementation of the GEOSS DSP with relevant international instruments and national policies and legislation. • Implement pricing policies consistent with GEOSS DSP. • Reduce time delays for making data available through GEOSS. • Promote research & education uses of GEOSS data, metadata and products. DSWG Report to EXCOM-27, 19/03/2013 GEOSS Data Sharing Action Plan for the Implementation of the GEOSS DSP • Builds upon the crucial concept of the full and open exchange and on the Implementation Guidelines accepted by Plenary • Notes that the achievement of full and open exchange of, and access to, key datasets requires actions by: – GEO collectively – Members and Participating Organizations individually – GEOSS tasks and cross-cutting activities DSWG Report to EXCOM-27, 19/03/2013 GEOSS Data Sharing Action Plan for the Implementation of the GEOSS DSP Specifies 8 actions to be pursued by: GEO Collectively Actions 1, 2, 3 and 4 GEO Members Actions 5 and 6 GEO Members and PO Actions 7 and 8 DSWG Report to EXCOM-27, 19/03/2013 Proposed Data Sharing Action 1 • Create the GEOSS Data Collection of Open Resources for Everyone (GEOSS Data-CORE) to address GEO Societal Benefit Areas • The GEOSS Data-CORE is a distributed pool of documented datasets, contributed by the GEO community on the basis of full, open and unrestricted access and at no more than the cost of reproduction and distribution * It should be noted that data accessible in the GEOSS Data-CORE is a subset of all data registered in the GEOSS DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE • It should be noted that the following conditions may be placed on data registered in the GEOSS Data-CORE by data providers, but are not required: (i) Attribution; (ii) User registration; (iii) Marginal cost recovery is permitted, (i.e., not greater than the cost of reproduction and distribution). • These are not considered as representing restrictions on the access to, or use and redistribution of, the data. DSWG Report to EXCOM-27, 19/03/2013 Proposed Data Sharing Action 2, 3 & 4 2. Establish a Task Force (Working Group) to: – Monitor the use and impact of resources made available with full and open access; – Promote the efficacy of the Data Sharing Principles in delivering societal benefits; – Evaluate the outcomes of this Action Plan. 3. Maintain the GEOSS Common Infrastructure as the architectural framework essential to implementing the Data Sharing Principles 4. Integrate implementation of the Data Sharing Principles, as appropriate, into the activities of GEO Work Plan Tasks and GEO Committees DSWG Report to EXCOM-27, 19/03/2013 Proposed Data Sharing Actions 5 and 6 5. Take leadership to establish national coordinating mechanisms to promote and monitor engagement with the implementation of the GEOSS Data Sharing Principles and provide feedback to GEO 6. Develop flexible policy frameworks to ensure that a more open data environment is implemented DSWG Report to EXCOM-27, 19/03/2013 Proposed Data Sharing Action 7 & 8 7. Maximize the number of documented datasets made available on the basis of full and open access 8. Promote with data providers within their territories the benefits of full and open access to data DSWG Report to EXCOM-27, 19/03/2013 The 2010 GEO Ministerial Declaration (i) (ii) (iii) Commit(ed GEO Members) to: maximize the number of documented datasets made available on the basis of full & open access; create the GEOSS Data Collection of Open Resources for Everyone (GEOSS Data CORE), a distributed pool of documented datasets with full, open and unrestricted access at no more than the cost of reproduction and distribution; and develop flexible national and international policy frameworks to ensure that a more open data environment is implemented, thus putting into practice actions for the implementation of the GEOSS Data Sharing Principles; DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data Sharing Working Group The DSWG supports GEO in its goals to put into practice the GEOSS Data Sharing Principles, the Implementation Guidelines for the GEOSS Data Sharing Principles and the GEOSS Data Sharing Action Plan. The DSWG has established 4 Subgroups: 1. GEOSS Data-CORE and GCI Subgroup; 2. Legal Interoperability Subgroup; 3. Documentation and Data Quality Subgroup; 4. Capacity Building Subgroup DSWG Report to EXCOM-27, 19/03/2013 The Capacity Building Subgroup • Provides support to GEO Members to enable them to establish national coordinating mechanisms and develop flexible policy frameworks; • Promotes with data providers the benefits of full and open access to data, which will also help to maximise the number of documented datasets made available on the basis of full and open access. DSWG Report to EXCOM-27, 19/03/2013 The Capacity Building Subgroup • The overall strategy is to identify lessons learned and best practices based on the experiences of GEO Members who have already established national coordinating mechanisms or developed flexible policy frameworks. • This will form the basis for documentation that can help GEO Members that would like to establish their own national coordinating mechanism or work to develop flexible policy frameworks. DSWG Report to EXCOM-27, 19/03/2013 The Capacity Building Subgroup • The Subgroup will also address the need for a strategy to inform nations regarding the benefits of data sharing and ways to improve national capacity through data sharing. • In support of this objective, the DSWG hosted a side-event at the GEO-IX Plenary in Brazil that focused on positive data sharing experiences. DSWG Report to EXCOM-27, 19/03/2013 The Documentation and Data Quality Subgroup • Promotes comprehensive documentation on data quality, including accuracy, stability, precision, spatial and temporal sampling characteristics, error characteristics, timeliness, completeness, reproducibility, calibration, accessibility, stability traceable to international standards, and reprocessing. • The work of this Subgroup complements that being carried out within the Infrastructure Tasks. DSWG Report to EXCOM-27, 19/03/2013 The Documentation and Data Quality Subgroup • Work on data documentation and quality is gathering momentum, with a draft recommendation having been formulated on guidelines for the documentation of data quality. • Comments on these draft guidelines are most welcome. • The SG is also exploring recommendations on user feedback, including determining what is needed and reviewing existing user feedback for improved data documentation on accuracy, stability, precision, and spatial and temporal sampling characteristics. DSWG Report to EXCOM-27, 19/03/2013 The Legal Interoperability* Subgroup • Reviews and updates the legal interoperability summary white paper and background papers; • Compiles a list of open access licenses and waivers, as well as restricted licenses, used within GEOSS and their key characteristics. [This will be submitted to the GEO-X Plenary, and updated before each subsequent GEO Plenary.] * Legal interoperability for data means that the legal rights, terms, and conditions of databases from two or more sources are compatible and the data may be combined by any user without compromising the legal rights of any of the data sources used. DSWG Report to EXCOM-27, 19/03/2013 The Legal Interoperability* Subgroup • Plans to develop educational and training materials on open access licenses and waivers for data providers and users; • Is reviewing user feedback received related to the implementation of open access licenses and waivers and associated digital rights management technologies; • Is leading DSWG efforts to develop recommendations and feedback relevant to science and technology activities. DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE and GCI Subgroup Looks to: • Expand the availability of detailed metadata to facilitate data and service discovery, assessment, access, and integration for decision support; • Promote the continued growth in the number of datasets made available on the basis of full and open access; • Assist in the implementation of the GEOSS Data-CORE. DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE and GCI Subgroup Call for GEOSS Data-CORE resources • On behalf of the DSWG, in October 2012 the Commission started contacting GEO Members and Participant Organisations who had pledged resources to the GEOSS Data-CORE to make their open resources visible and available as such. • We have been also asking potential contributors to the GEOSS Data-CORE (ECDS, PANGAEA, SERVIR, etc.). • After a first call for data with a moderate response, in the wake of the GEO Plenary (December 2012) we sent reminders to those who had not responded. • The second call was more successful, with actions being taken by contributors like UKGEO or PANGAEA and commitments made by SERVIR, ITC or INTERACT Project. DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE and GCI Subgroup GEOSS-DataCORE original pledges (I) Institutions & projects INPE ICARE ETHER CNES CLS IFREMER IRD LEGOS ACRI GEO Member/ PO Brazil France France France France France France France France RESIF Consortium Deutscher Wetterdienst (DWD) (= Nat. Met. Service of Germany) France Germany Global Runoff Data Centre (GRDC) at the Federal Institute of Hydrology (BfG) DLR/WDC-RSAT ISPRA JAXA Germany Germany Italy Japan Flagged as DataCORE Some resources flagged as DataCORE Not flagged as DataCORE Earth Observation Data Integration & Fusion Research Initiative, the University of Tokyo Japan Spanish State Meteorological Agency (AEMET) Spain KNMI The Netherlands University of Twente - Faculty of GeoInformation Science and Earth Observation (ITC) The Netherlands DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE and GCI Subgroup GEOSS-DataCORE original pledges (II) Institutions and projects GEO Member/ PO SeaDataNet The Netherlands Centre for Environment, Fisheries & Aquaculture Science UK Centre for Polar Observation & Modelling UK Department for Environment, Food and Rural Affairs UK Department of Agriculture and Rural Development (DARD) Rivers Agency UK Department of the Environment Northern Ireland UK Marine Scotland UK Some resources flagged as DataCORE Forestry Commission UK Not flagged as DataCORE National Biodiversity Network UK NERC British Antarctic Survey UK NERC British Geological Survey UK NERC Centre for Ecology and Hydrology UK Plymouth Marine Laboratory UK Proudman Oceanographic Laboratory UK Rutherford Appleton Laboratory UK Scottish Environment Protection Agency UK Sir Alistair Hardy Foundation for Ocean Science UK Agri-Food and Biosciences Institute UK Environment Agency UK CDIAC (Department of Energy) US Flagged as DataCORE DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE and GCI Subgroup GEOSS-DataCORE original pledges (III) Institutions and projects GEO Member/ PO AirNow (EPA) US Standard Earth Science Products (NASA) US NOAA datasets US Foreign Agriculture Serve Crop Explorer Interface (DoA) US Incorporated Research Institutions for Seismology (IRIS) US Landsat Global Land Survey (USGS) US Some resources flagged as DataCORE GCMD (USGEO) US Not flagged as DataCORE Datasets from atmospheric reanalyses, Datasets on atmospheric composition developed by pilot projects for the GMES Atmosphere Service ECMWF Several datasets EEA GMES Sentinel missions, Envisat, Earth Explorer, ERS ESA Different Meteosat, SAF and Metop products Eumetsat GBIF Data GBIF IAG IAG ISCGM ISCGM World Data System ICSU Flagged as DataCORE DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE and GCI Subgroup Contacting GEOSS-DataCORE providers – 1st stage (I) GEO Member/PO Organizations and Projects Contacted PO/Japan EEA DIAS (Earth Observation Data Integration & Fusion Research Initiative, the University of Tokyo) ISPRA Deutscher Wetterdienst (DWD) JAXA (ALOS, PALSAR) Spanish State Meteorological Agency (AEMET) DLR/WDC-RSAT (Polyphemus) Federal Ministry for Environment, Nature Conservation and Nuclear Safety ISCGM (The Global Map) EEA Brazil Japan Italy Germany Japan Spain Germany Germany Contacted? When? Response? When (1st Time)? 8/10/2012 8/10/2012 8/10/2012 9/10/2012 9/10/2012 10/10/2012 8/10/2012 10/10/2012 11/10/2012 11/10/2012 6/11/2012 INPE (DevCoCast & Earth Observation Images Catalog) 11/10/2012 14/10/2012 The Netherlands University of Twente - Faculty of GeoInformation Science and Earth Observation (ITC) 12/10/2012 France CTOH (LEGOS) 15/10/2012 ESA ESA (GMES Sentinel Missions, Envisat, ERS, Earth Explorer Missions) 15/10/2012 15/10/2012 OneGeology-Europe 15/10/2012 18/10/2012 World Data System Global Runoff Data Centre (GRDC) at the Federal Institute of Hydrology (BfG) 15/10/2012 19/10/2012 OneGeologyEurope ICSU Germany 15/10/2012 Are flagging or will flag shortly resources as DataCORE Expect to flag but not now Procedure to flag data is under review No response or no decision yet DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE and GCI Subgroup Contacting GEOSS-DataCORE providers – 1st stage (II) GEO Member/PO Organizations and Projects Contacted Eumetsat UK France France France France France / Europe ECMWF Contacted? When? Eumetsat UKGEO ICARE (CNES, INSU, Univ Lille, Region Nord-Pas-de-Calais) ETHER (CNES, INSU) SPIRIT, KALIDEOS (CNES, SPOT Image) AVISO (CNES, CLS) HERMES (ACRI, ESA) ECMWF Global Seismographic Network - Incorporated Research Institutions for Seismology (IRIS) SMOS (IFREMER, CNES) IAG (IRD) IFREMER (CERSAT) RESIF Consortium ECDS (Swedish Meteorological and Hydrological Institute) INTERACT: Royal Swedish Academy of Sciences (INTERACT Coordinator) PANGAEA: Alfred Wegener Institute for Polar and Marine Research (AWI) and Center for Marine Environmental Sciences (MARUM) from the University of Bremen 15/10/2012 16/10/2012 16/10/2012 16/10/2012 16/10/2012 16/10/2012 16/10/2012 16/10/2012 Germany Bundesanstalt für Gewässerkunde; UniBonn (GEOWOW FP7 Project) 11/12/2012 The Netherlands SeaDataNet 13/12/2012 US France France, IAG France / Europe France Sweden Germany Response? When (1st Time)? 13/12/2012 17/10/2012 17/10/2012 17/10/2012 17/10/2012 17/10/2012 24/10/2012 29/10/2012 29/10/2012 9/11/2012 13/12/2012 Are flagging or will flag shortly resources as DataCORE Expect to flag but not now Procedure to flag data is under review No response or no decision yet DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE and GCI Subgroup Contacting GEOSS-DataCORE providers – 2nd stage (I) GEO Member/PO Organizations and Projects Contacted When? Response? When (1st Time)? Japan DIAS (Earth Observation Data Integration & Fusion Research Initiative, the University of Tokyo) 12/12/2012 12/12/2012 Japan Germany JAXA (ALOS, PALSAR) DLR/WDC-RSAT (Polyphemus) 12/12/2012 12/12/2012 12/12/2012 14/12/2012 Germany Federal Ministry for Environment, Nature Conservation and Nuclear Safety 12/12/2012 PO/Japan ISCGM (The Global Map) 12/12/2012 Brazil INPE (DevCoCast & Earth Observation Images Catalog) 12/12/2012 12/12/2012 23/01/2013 OneGeologyEurope OneGeology-Europe 12/12/2012 12/12/2012 The Netherlands University of Twente - Faculty of GeoInformation Science and Earth Observation (ITC) 13/12/2012 14/12/2012 17/12/2012 ESA ESA (GMES Sentinel Missions, Envisat, ERS, Earth Explorer Missions) 13/12/2012 14/12/2012 UK UKGEO 13/12/2012 13/12/2012 23/01/2013 Are flagging or will flag shortly resources as DataCORE Expect to flag but not now Procedure to flag data is under review No response or no decision yet DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE and GCI Subgroup Contacting GEOSS-DataCORE providers – 2nd stage (II) GEO Member/PO Organizations and Projects Contacted When? Response? When (1st Time)? ECMWF ECMWF 13/12/2012 South Africa SAEON Data 13/12/2012 - INTERACT: Royal Swedish Academy of Sciences (INTERACT Coordinator) 17/12/2012 18/12/2012 Germany PANGAEA: Alfred Wegener Institute for Polar and Marine Research (AWI) and Center for Marine Environmental Sciences (MARUM) from the University of Bremen 17/12/2012 8/01/2013 Spain National Geographic Reference Equipment (IGN-CNIG) 17/12/2012 17/12/2012 France SPIRIT, KALIDEOS (CNES, SPOT Image) 17/12/2012 China Finer Resolution Observation and Monitoring - Global Land Cover 21/01/2013 US US SERVIR Landsat Global Land Survey 25/01/2013 26/02/2013 25/01/2013 Are flagging or will flag shortly resources as DataCORE Expect to flag but not now Won't flag neither in the short nor in the mid term No response or no decision yet DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE and GCI Subgroup Lessons learnt (so far) • Most of the entities contacted responded positively to our call. • Nevertheless, most of them have just reconfirmed their pledges and actions still need to be taken. • Those who are already taking some action appear not to completely understand what they actually have to do. We need simple step-by-step guidelines. • We cannot forget that GEOSS is built on a voluntary basis so we need to respond quickly at the questions and doubts raised by those who are willing to participate. Providers want a named Point-of-Contact they can interact with directly. • Following up the contributions to the GEOSS Data-CORE and to the GEOSS in general has to be, therefore, a continuous effort. We need to look at how the GEO Secretariat can provide dedicated support for this work. DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE and GCI Subgroup Future Plans • Follow-up the commitments by those already contacted, supporting them in registering and flagging their resources in the GEOSS. • Ask for further potential contributors to the GEOSS Data-CORE, especially to the Sprint To Ministerial demonstrators. • Ensure the real access to the GEOSS Data-CORE data, by checking the links provided in the metadata are available through the GEO Portal. • Launch a second reminder to those who still have not responded. DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE and GCI Subgroup GEOSS Data-CORE Timeline EEA GEOSS Data-CORE catalogue GEO VII Beijing Ministerial: Creation of GEOSS DataCORE ISPRA datasets INPE Image Catalog 2013 2011 2010 2012 Sprint to 2011 Plenary: GCMD in the GEOSS DataCORE October 2012: EC starts following up the GEOSS Data-CORE PANGAEA EXCOM27: > 600000 GEOSS DataCORE resources GEO-IX: > 14000 GEOSS DataCORE resources There are now over 600,000 GEOSS Data-CORE resources available in the GEOSS – and rising! DSWG Report to EXCOM-27, 19/03/2013 The GEOSS Data-CORE: Metadata Issues • To easily identify and access Data-CORE resources it is necessary for data providers to add, in addition to the geossDataCore tag, a link to the data resource or to a service enabling the view/download of the resource, and to make sure that this link is stable and is really working. • Considerable work is required to liaise with data providers to make them aware of the critical importance of this task. • A continuous effort also has to be made in order to keep the metadata updated. • And there are also many other issues ……. DSWG Report to EXCOM-27, 19/03/2013 The GEOSS DSWG and IIB therefore proposes that: • The GEO Secretariat should be the main Point-ofContact for entities looking to contribute resources to the GEOSS and in particular the GEOSS Data-CORE and issues associated with the quality and the correctness of the metadata accompanying the registered resources. • The GEO Executive Committee should instruct the Director of the GEO Secretariat to recruit an IT/Data Management specialist to help with the development of the GEOSS Data-CORE and the associated metadata. DSWG Report to EXCOM-27, 19/03/2013 The GEOSS DSWG and IIB Proposal • Ensuring that the GEOSS Data-CORE is a success is critical to the overall success of the GEOSS. • The implementation of the GEOSS Data-CORE will have a major impact on users, including those in Developing Countries. • The discovery of and access to GEOSS Data-CORE resources truly distinguishes the GEOSS from other the capabilities of other initiatives such as Google. • Hence the recruitment of such a specialist is not simply a matter of addressing technical issues, but also one of raising the visibility of the GEOSS and communicating its impact to the GEO Community and beyond. DSWG Report to EXCOM-27, 19/03/2013 Data Sharing in GEO Post-2015 The DSWG has begun discussions on possible data sharing strategies to be taken into consideration post-2015. Topics and/or issues include, (but are not limited to): • Should the Data Sharing Principles be updated; • What should be the responsibility and role of the GEO Secretariat in supporting the implementation of the GEOSS Data-CORE, the GCI and the Data Sharing Action Plan; • How can we improve the inclusion of commercial databases in the GEOSS; • What are the implication of new Earth Observation concepts and technologies, such as Citizens' Observatories, for the GEOSS and data sharing. DSWG Report to EXCOM-27, 19/03/2013 Data Sharing in GEO Post-2015: Updating the Data Sharing Principles Could the DSPs become a single Data Sharing Principle? That the societal benefits of Earth observations can only be achieved by: Full (FREE?), Open and Unrestricted Discovery and Access and INTEROPERABLE Use, Re-use and Re-dissemination of Documented Metadata, Data and Products of Known Quality with Minimal Time Delay DSWG Report to EXCOM-27, 19/03/2013 Data Sharing in GEO Post-2015: Evolving the GEOSS Architecture Accepting that society naturally organises itself to work within a "community framework", as communities recognise that they can derive much greater value if their individual efforts can be brought together within a "Community Framework", could the GEOSS architecture evolve from a "System of Systems" approach, to a "Community of Communities" philosophy. DSWG Report to EXCOM-27, 19/03/2013 DS Post-2015: Evolving the GEOSS Architecture A "Community of Communities" GEOSS CEOS R/S Community Community A Com. Portal ESA, NASA, NOA, JAXA, etc. Cit. Obs Domain Service Bus R/s In situ Archives Quality & Tagging Community X Composition ... Community B Access Com. Portal Semantic Cit. Obs Domain Service Bus R/s Discovery In situ Archives Brokering Infrastructure DSWG Report to EXCOM-27, 19/03/2013 Data Sharing in GEO Post-2015: Evolving the GEOSS Architecture A "Community of Communities" GEOSS Such a Framework: • builds on existing capacities, enhancing communities, respecting their mandates, but not supplanting them; • offers a high level of flexibility; • is open to new innovations as technology evolves, (so we can benefit from the next revolution!); • provides the potential for interoperability with cultural, social and economic information that will ultimately play an important role in decision making. But the societal benefits of such a Framework can only be achieved with the sharing of data.” DSWG Report to EXCOM-27, 19/03/2013 Summary GEO is working very actively: • to promote the benefits of full and open access to data; • To maximize the number of documented datasets made available on the basis of full and open access; • Expand the availability of detailed metadata; • Promote comprehensive documentation on data quality; • Compile lists of appropriate licenses and waivers; • Provide support to develop flexible policy frameworks. DSWG Report to EXCOM-27, 19/03/2013 Summary • The DSWG encourages all GEO Members and PO to make all resources currently pledged to the GEOSS Data-CORE visible in the GEOSS and to pledge further contributions. • The DSWG, supported by the IIB, asks EXCOM to instruct the GEO Secretariat Director to recruit an IT/Data Management specialist to help with the development of the GEOSS Data-CORE and the GCI. • EXCOM is also asked to identify a senior person from amongst its Co-chairs (or Members) and designate that individual as a Co-chair of the DSWG, to facilitate continued communication between the two groups on the key topic of data sharing. DSWG Report to EXCOM-27, 19/03/2013