HRD In-Situ Ground System
Bridging Technologies Between EOS, NPP and the
Future
International TOVS Study Conference - XII
PI: Patrick Coronado
Code 935/Applied Information Sciences Branch
patrick.coronado@gsfc.nasa.gov
http://directreadout.gsfc.nasa.gov
Presented by: Gail Reichert
Code 935 /Applied Information Sciences Branch
gail.reichert@gsfc.nasa.gov
3/3/2002
1
NPP Direct Readout Lab (DRL)
Who?
• We are the NPP In-Situ Ground System element.
• As part of the NPP, our job is to design & prototype a Direct
Readout Ground System, which will
– Serve as an advanced technology test bed
– Act as a compatibility test unit for pre-launch spacecraft (e.g.,
NPP)
– Be back compatible with EOS spacecrafts and instruments
3/3/2002
2
The Problem
larger
more complex
More satellites +
+ more complex
instruments
data sets
= big headache (!)
for general users
more complex
+ processing
requirements
Our Goals:
Mission
Design
Develop
Integrate
Hardware/Software technologies needed to
acquire/process DB data
Overall:
&
3/3/2002
Test
Make available via
release to public domain
technology transfer
Be a bridge between satellite missions (Terra, Aqua, NPP…)
& general user community
“Pave the road” to NPP
3
Outline
• Real-Time Software Telemetry Processing System (RTSTPS)
• Real-Time Instrument Data Streaming (Simulcasting)
• Web site for distribution of software algorithms
– Level 0 (RTSTPS) + other protocol processing
– MODIS Level 1 (geolocation & calibration)
– Institutional & DB Level 2
• NPP ESTO Portal for Science, Technology and Environmental
Research (NEpster)
– Phase 1: Level 0 & Level 1 data
• Mobile NISGS: Advanced Technology Test Bed
– Phase 1: Land-going
*Charts with asterisks next to their page numbers signify pages with supplemental information.
3/3/2002
*4
Direct Readout Roadmap to NPP
POES/DMSP/SeaStar
Terra
Aqua
NPP
Now
AVHRR
SeaWiFS
OLS
MODIS
AIRS
AMSU
HSB
MODIS
VIIRS
CrIS
ATMS
S/C & Instrument Evolution
Standardization & Increasing RF, Modulation & Bandwidth Requirements
•L,S-band
•.665 - 2Mbps
•Bi-Phase L
•X-band
•13.1Mbps
•Viterbi
•OQPSK
•Custom Frame Formatters
& Ingest software
•Analog Custom Receivers
•NOAA Level1B (AVHRR)
•Limited Data Distribution
mechanisms
3/3/2002
•S/C specific STPS
•Level-0
•Return Link Processor
•Analog Configurable
Receiver
•X-band
•15Mbps
•OQPSK
•NRZM
•Reconfigurable RT-STPS
•Return Link Processor
•Digital re-Configurable
Receiver
•MODIS & AIRS Level-1
•MODIS Level-1
•DAAC & NEpster with
•DAAC & MODIS Simulcast
L0 & L1 data
•X-band
•15Mbps
•QPSK
•NRZM
•Viterbi
•Compression
•Reconfigurable RT-STPS
•Digital re-Configurable
Receiver (PC-based)
•ALL Instru. Level-1 Software
•DAAC & NEpster
with L0, L1 & EDRs
•Simulcast of all Instruments
Evolution of Concurrent Ground System Supporting Technologies and Algorithm Development
5
Real-Time Software Telemetry Processing System
• Standalone processing software system
• Takes raw satellite data stream from receiver & processes to
EOS Level-0 in real time:
– Synchronizes frames (CCSDS version 2)
– Applies Reed-Solomon error detection and correction
– Reassembles packets into 1+ Level 0 format data files
(by SCID, VCID &/or APID)
• Written in Java (runs on any computer with JRE 1.3+)
• Runs in batch &/or server mode
• Modular design, easy to reconfigure
– Processing modules linked by data pathways as set in configuration file
– Interactive control via graphical application viewer
3/3/2002
*6
RTSTPS Major Components
• Multi-threaded server:
– Runs continuously
– 1st thread listens for TCP/IP packets (unsynchronized telemetry)
– 2nd thread listens for user commands (load, shut down, get status info)
• Graphical application viewer
– Runs on remote computer or user’s local machine
– Sends load & shutdown commands
– Updates & displays status info
• Graphical editor (under development)
– Users may create/edit configuration files
– Config files may be on user's or server's computer
• Web viewer (planned)
– Connected to STPS server, shows status info
3/3/2002
*7
Real-Time STPS Context
Raw
Telemetry
SERVER
Frames
CADUs
VCDUs
B_PDUs
Packets
CORE
Frame
Synchronizer
Acquisition
Board
RS/CRC/PN
Decoders
Files
CCSDS
Services
Network
Ports
Status
Setup
Status
Remote User Interfaces
Editor
Setup
Files
Viewer
Web
see http://directreadout.gsfc.nasa.gov/projects/rt-stps/stps.htm
3/3/2002
8
Real-Time STPS versus STPS
written in
speed
reconfigurable
STPS
Java
C
as fast or faster
Yes
(most CCSDS compliant s/c)
Not
easily
Java tool
simple text
create PDS “on the fly”?
Yes
No
process AQUA data?
Yes
No
sort by time?
No
Yes
remove duplicate packets?
No
Yes
detect VCDU header errors?
No
Yes
next release
Yes
interface
handle NRZ-L/NRZ-M correction?
3/3/2002
RTSTPS
*9
Simulcasting
Realtime Flow
Low Rate Flow
Level-0
Ingest Computer
(PC/NT)
Receiver
Real-Time
Server
Products
Real-Time
Clients
Rate Buffer
Purpose: Distribute & visualize DB data in real time
Current System: Prototype, tested using Terra MODIS data
Planned: Multi-mission, multi-instrument
see http://directreadout.gsfc.nasa.gov/projects/simulcasting/simulcasting.htm
3/3/2002
10
Distribution of Software Algorithms
• Web site: http://directreadout.gsfc.nasa.gov/software_main.html
• Currently available in standalone mode:
– Level 0 (RTSTPS) + other protocol processing (STPS, Sorcerer,
MODIS band extractor, DPREP)
– MODIS Level 1: DAAC, IMAPP (offsite), winIMAPP (offsite)
– MODIS Fire Detection (Level 2)
– NEpster interface (NOI)
– Simulcasting Server/Client
• Soon to be available:
– MODIS NDVI (Level 2)
– Level 1, Fire Map, NDVI Viewers
• Planned:
– Institutional Level-2 algorithms (by Institution, not converted to
standalone mode)
– MODIS BRDF converted to standalone mode, with lessons learned
3/3/2002
11
Distribution of Software Algorithms (continued)
• To download any DRL-provided algorithm, user must fill out
User Account Form (for authentication purposes only)
• To download any software algorithm, user must electronically sign
software disclosure form for each algorithm
– Member LogIn Page
– Only have to sign form for each algorithm once
Algorithms for Future Missions:
• With Terra (& Aqua), we are “paving the road” to NPP in terms of
– Conversion of institutional algorithms to DB standalone mode
– Algorithm & product staging
– Distribution & version management
3/3/2002
* 12
Algorithms for Future Missions (continued)
• Don’t miss the following poster:
“Weather and Climate Information Available from EOS and
NPP Direct Broadcasts”, J. Dodge (NASA)
• Highlights algorithm & product roadmap from EOS
spacecrafts to NPP:
– MODIS As Real Time Camera - vegetation, coastal conditions, flooding,
fires, weather phenomena, …
– New Sensor Data For AQUA DB
– AIRS Temperature/Moisture Soundings
– AMSR-E Over US To Be Available
– Some Potential New Uses For AQUA Data
– NPP Data Products
3/3/2002
* 13
NPP ESTO Portal … (NEpster)
• Impetus: Unprecedented requirements for
– (for DB) Geographically dispersed collection
– Complex processing
– Very large data volumes
(for NPP Level 0 estimate 1 Gbyte/day/site)
• Concept:
– Virtually connected network of data archive systems that share & exchange
remote sensing data
– Peer-to-peer architecture
– Borrows from Napster, but relevant to other science areas as well
(Particle Physics, Medical Imaging, Astronomy…)
• Will be developed in 2 phases
3/3/2002
* 14
NEpster Phase I
Temporary
staging
archive
Broker
Metadata
Extractor
NOI
RODIN
(metadata)
Peer-to-peer data sharing
3/3/2002
see http://directreadout.gsfc.nasa.gov/projects/nepster/nepster.htm
15
Nepster Phase I (continued)
Components:
• Metadata Extractor:
– Automated, runs at DRL and remote archive sites
– Extracts metadata as new data is collected
– (or: remote site may choose to have regular downloads via ftp)
• RODIN: Direct Readout Lab database (MYSQL)
– Stores extracted metadata
– Allows searches on metadata fields (e.g., time, geolocation station)
• Intelligent Broker:
– Maps extracted metadata to RODIN database
– Resolves remote site policies (transparent to user)
• Nepster Observation Interface (NOI):
– Web-based data search interface
• Temporary Staging Archive:
– Stages remote data at DRL for possible future requests
– Convenient for remote sites with narrow “pipes”
3/3/2002
* 16
NEpster Phase II
“Deep” archive metadata store
DAAC
“deep”
archive
QuickTime™ and a
GIF decompressor
are needed to see this picture.
Real
Time
Provides:
• Real-Time notification of data acquisition at a remote site
• A web-based query mechanism for all participating acquisition sites and data repositories
• A temporary archive for remote sites with limited bandwidth
• Automatic routing between the user and data source
3/3/2002
* 17
Mobile NISGS
Will include:
1. RTSTPS
2. Simulcasting
3. NEpster
Will act as:
1. Advanced technology testbed
2. Compatibility test unit for
pre-launch spacecraft
See http://directreadout.gsfc.nasa.gov/projects/mobile_nisgs/mobile_nisgs.htm
3/3/2002
18