CEOS System Engineering Toolset (CSET)
Space Data Management System (SDMS)
History: CEOS Visualization Environment (COVE) started in 2009
– Web-based suite of tools for searching, analyzing and visualizing actual & potential satellite
sensor coverage
– Philosophy: Simple, intuitive, easily accessible
– 2014 COVE Stats: ~2500 users, 20K page views, broadly utilized by countries worldwide
CSET is a Software Framework + Suite of Tools (Apps) that leverages a Common
Architecture, Unified Data Model, Common User Experience (App continuity)
1
2
Data Management and Data Integration Tools
o Space Data Management System (SDMS) - Extensible Data Management framework
Analysis Tools, Data Visualization & Presentation Tools
o
o
o
Rapid Acquisition Tool - Determines when satellites may get acquisitions of regions/locations
Coverage Analyzer - Determines number of actual acquisitions with given cloudiness threshold
COVE Google Earth visualization tool for satellites - Displays potential and coincidence paths for Earth
Observing Satellites. Data presentation, reporting, and charts
3
Informational Tools: Mission Instrument Browser, Data Policy portal, Labs…
CSET Application Portal
www.ceos-tools.org
Live in late November 2014
Analysis &
Visualization
Apps
Data Management
(SDMS) Apps
Space Data Management System (SDMS)
SDMS is a flexible Data Integration & Data Management Framework – Key is the ability
to quickly create applications (e.g., country/organization specific applications):
– Common Architecture & Data model, Security model, Project Model
– Search & Data Discovery Capability
o
o
Can pull in and search archives by metadata, criteria
Search the entire federated database
– Analysis & Tool Execution Capability
o
o
o
o
Tool Integration framework (Java/JS based tool wrapping)
Existing and custom tools to analyze data, e.g.:
GDAL, Google Earth Engine, Open Foris
Flexibility to work with different workflows/use-cases (mix-and-match)
GIS Tools
Community
– Data Visualization and Presentation Capability
– Big Data Framework (Hadoop – Local Big Data Cluster)
• Working toward model for low bandwidth connections
– Data housed, fetched, and processed remotely
– Tiered solution for different bandwidth levels
• SDMS is not the cure all…
Capabilities
SDMS
Framework
Columbia
GFOI
FAO
SDMS Interface
Project (Analysis)
… Data
… Tools
… Output
1
Global
SDMS Data
& Tools
2
3
Context specific
Work Area
User-specific
SDMS Data
& Tools
(Analysis Case)
4
Search & Filter
SDMS Tool Execution (Script Editor)
1
3
Global
SDMS Data
& Tools
Script
Editor
2
User-specific
Data & Tools
>> Initializing…
4
Execution
Console
SDMS Tool Execution (Terminal Interface)
1
Global
SDMS Data
& Tools
2
3
User-specific
Data & Tools
Terminal
Interface
GFOI SDMS Prototype
SDMS Integration - Google Earth Engine
Google Earth Engine Demonstrator:
– GEE is powerful satellite data acquisition and processing system
– SDMS has JavaScript wrappers to the GEE API (small subset)
o Allows heavy-lifting to be done at Google data centers
o Can download result subsets at user specified bounds and resolutions
– Benefits of integration:
o Data: Provides additional source of data (SDMS Unified Data Model)
o Tools: Additional processing capabilities
o Workflow: Can be a source of data that is then processed by other tools
– Current demonstrator prototype
o Allows fetching of Landsat 7,8 band TIF files and
combined thumbnail data
o Focus is on cloud cover removal
SDMS Status
•
•
•
•
CSET / SDMS development has been ongoing over the last year
Our initial release is a prototype/demonstrator (Amazon servers)
Alpha release to a small user group in December 2014
Looking for feedback, hoping CEOS SEO tools continue to be of
use to the broader community
• SDMS Live Demonstration
1. SDMS Interface Prototype – Concept of Operations
2. Google Earth Engine Prototype – L8 Cloud removal
3. GFOI Prototype - Kenya
POC: Brian Killough, Ph.D. brian.d.killough@nasa.gov
Brian Williams, Ph.D b.williams@ama-inc.com
Sanjay Gowda, Ph.D. gowda@ama-inc.com
Data Services Prototypes ... Kenya Demo


SEO is developing an online version of a generic data services tool for
future testing.
A demo of this tool is being worked with FAO to test processing and
product development for one Kenya scene.
DEMO Details
 Location: Kenya (Mt. Kenya National Park)
 Dataset: Landsat 7, (Path-168, Row-60)
 Step #1: LEDAPS Processing - TOA reflectance, cloud
screening (ACCA), Atmospheric Correction to SR
 Step #2: BRDF Correction - Use MODIS data?
 Step #3: Cloud/Shadow Removal - Tag pixels with a value
or color for statistics and viewing
 Step #4: SLC Gap Removal - Tag pixels with a value or
color for statistics and viewing
 Step #5: Forest Classifier (unsupervised, or MODIS
Vegetation Continuous Fields (VCF), or Random Forest)
 Final Product: Forest Map with Statistics (Clouds, SLC
Gaps, Forest, Non-Forest)
10