Aquarius/SAC-D Science Team Meeting
Aquarius Data Processing System (ADPS)
and Data Distribution
Fred Patt
July 19, 2010
Seattle, wA
Agenda
•
•
•
•
•
•
•
Overview
Science Data Processing
Level-2 Science Software Implementation
Level-2 Data Product Format
Data Archive and Distribution
Aquarius Web Site
What We Need
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
2 of 29
July 19-21, 2010
Aquarius Data Processing System
Overview
• The Aquarius Ground System at NASA/GSFC consists of the
Aquarius Data Processing System (ADPS) and the Aquarius
Command and Control System.
• The ADPS will support:
– Acquisition and ingest of Aquarius/SAC-D data from CONAE.
– Acquisition of dynamic ancillary required to support the science processing.
– Processing of Aquarius data to standard product levels using algorithms provided
by the Science Team.
– Quality control of data products using methods specified by the Science Team.
– Match-up of Aquarius data with in situ salinity measurements.
– Archive and distribution of all data products during the active life of the mission to
the science team and general science community.
– Delivery of source data (Level-1a) and “validated” Level-2 and Level-3 derived
data products to the PO.DAAC.
• The ADPS will be supported within the existing infrastructure of the
Ocean Biology Processing Group (OBPG).
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
3 of 29
July 19-21, 2010
Agenda
•
•
•
•
•
•
•
Overview and Status
Science Data Processing
Level-2 Science Software Implementation
Level-2 Data Product Format
Data Archive and Distribution
Aquarius Web Site
What We Need
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
4 of 29
July 19-21, 2010
Aquarius Data Processing Overview
Standard Data Processing
Raw Level-0
Downlink
File
Web-Based Telemetry Monitoring Tools
Level-0
Report
Level-0
Preprocessor
Level-0
SAC-D
HKT Data
Level-1a
Report
Level-0 to 1A
Level-1A
temp
Dynamic
Ancillary
Data
Level-1A Merge
Level-1A
orbit
Telemetry
Report
Level-1A to 2
Level-2
Static
Ancillary
Data
Bias Adjustment
Level-2 to 3
Smoothing
Level-3 Mapping
Automated Telemetry
Analysis Tool
Level-2 to 3
Binning
AVDS
In-Situ Match-up
Process
Level-2
Match-up
Predicted
Downlink
Times
SAC-D
HKT Data
CONAE-provided
Interactive Telemetry
Analysis Tool
Level-3 Bin
Level-3 Binning
Web-Based Browse,
Download and Order Tools
Level-3 Map
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
5 of 29
July 19-21, 2010
Science Data Processing
The Aquarius science data processing consists of the following steps:
•
•
•
•
•
•
•
•
•
Level-0 Preprocessor – process each HRD downlink file to produce a single time-ordered set
of Aquarius science blocks and removes duplicates– developed by Joel Gales - ADPS.
Level-0-to-1A – separate Level-0 files into orbits, unpack science data, incorporate overlapping
ephemeris and SAC-D HKT and convert Aquarius HKT – developed by Joel Gales - ADPS.
Telemetry Analysis – HKT data from Level-1A files analyzed for web display on the Aquarius
web site – developed by Susie Kennison, Norman Kuring and Liang Hong - ADPS/ACCS.
Interactive Telemetry Analysis - HKT files analyze using CONAE-provided telemetry viewer.
Level-1A Merge – consolidate Level-1A files from overlapping downlinks into a single, best-
quality full orbit product – Joel Gales - ADPS.
Level-1A-to-2 – perform calibration, atmospheric correction and salinity retrieval for Aquarius
science data – radiometer and scatterometer code delivered by RSS and JPL, respectively;
integration and input/output code developed by Joel Gales and Liang Hong - ADPS.
Level-2-to-3 Binning – Level-2 salinity retrievals for one day geographically projected and
collected into equal-area bins – existing OBPG software modified by Joel Gales- ADPS.
Level-2-to-3 Smoothing – Level-2 salinity retrievals for one day optimally interpolated into
equal-area bins – algorithm by Jonathan Lilly/ESR implemented by Joel Gales - ADPS.
Level-3 Binning and Mapping – Level-3 binned files aggregated to longer time periods
(weekly, monthly, etc.) and projected onto map grid – existing OBPG software.
All processing steps are run automatically when all input data sets are available.
All software exists in at least preliminary prelaunch form at this time.
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
6 of 29
July 19-21, 2010
Science Code Status
• Radiometer and Scatterometer 2009 code deliveries have been
integrated and installed in the data processing system.
• Latest versions of Level-1A and Level-3 software have been installed.
• Ancillary data to support processing have been acquired and ingested.
• The one-year (2007) simulated data set has been processed to Level-1A,
Level-2 and Level-3 products:
– http://oceancolor.gsfc.nasa.gov/cgi/aquarius (Level1A/2 browser)
– http://oceancolor.gsfc.nasa.gov/cgi/l3 (Level-3 browser)
– http://oceandata.sci.gsfc.nasa.gov/Aquarius (FTP-like interface)
• The PO.DAAC is downloading the simulated data products and making
them available (see poster by Gregg Foti).
• AVDS processing is in progress with the 2007 data set.
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
7 of 29
July 19-21, 2010
Science Code TBD
• From now until launch we need to:
– Integrate Radiometer and Scatterometer code updates into data
processing system.
– Incorporate additional quality flags into software and products.
– Develop ingest procedures for additional ancillary data required by
new code.
– Incorporate SAC-D HKT format into Level-1A processing.
– Verify output products with Science Team.
– Begin real-time mission simulation (August 2010 --> Launch).
– Incorporate additional prelaunch updates into the science software
(as needed).
– Participate in mission tests as scheduled.
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
8 of 29
July 19-21, 2010
Agenda
•
•
•
•
•
•
•
Overview and Status
Science Data Processing
Level-2 Science Software Implementation
Level-2 Data Product Format (separate presentation)
Data Archive and Distribution
Aquarius Web Site
What We Need
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
9 of 29
July 19-21, 2010
Level-2 Software Implementation
Implemented using software delivered by RSS and JPL
with I/O by Joel Gales
Capabilities:
•
•
•
•
•
•
Processes a Level-1A input product.
Performs radiometer Level-1B calibration and Level-2 processing using
latest code from RSS.
Performs scatterometer Level-1B calibration and Level-2 processing using
latest code from JPL.
Reads SST, wind, atmospheric, sea ice and solar flux data from dynamic
ancillary files.
Computes geolocation.
Generates HDF5 output product.
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
10 of 29
July 19-21, 2010
Level 2 Processing Flow
QMET
Ancillary
Files
Solar Flux
Ancillary
Data
SST
Ancillary
Files
Sea Ice
Ancillary
Files
Static
Ancillary
Files
Look-up
Tables
atm_tb_aquarius
Level-1A
Product
ATM
Ancillary
Files
mk_aquarius_
ancillary_data
Radiometer
Ancillary
Files
L2gen_scatter
Look-up
Tables
L2gen_aquarius
Scatterometer
ASCII Files
Level-2
Product
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
11 of 29
July 19-21, 2010
Dynamic Ancillary Data
Data Type
Source
Interface
Geopotential height, surface
Temperature at surface
Atmospheric pressure at surface
Relative humidity, entire atmosphere
Precipitable water
Cloud water
U/V wind 10 m
Soil moisture
Geopotential height (profile)
Temperature (profile)
Relative humidity (profile)
Cloud water mixing ratio (profile)
NCEP
Primary: tads1u.ecs.nasa.gov
Secondary: dss.ucar.edu
Sea Surface Salinity (model)
FSU/JPL
Primary: ourocean.jpl.nasa.gov/SSS/
Sea Surface Temperature
Reynolds (NCEP)
Primary: eclipse.ncdc.noaa.gov
Significant Wave Height
NCEP
Primary: polar.ncep.noaa.gov/pub/waves
Solar Flux and Flares at 1.4 GHz
USAF (NGDC)
Primary: www.swpc.noaa.gov/
ftpdir/latest/dayind.txt -- production
NSIDC
Primary:
polar.ncep.noaa.gov/pub/cdas/archive
-- for 2007 simulation
polar.ncep.noaa.gov/pub/cdas/
-- for production
Sea Ice Concentration/ Extent/Age
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
12 of 29
July 19-21, 2010
Agenda
•
•
•
•
•
•
•
Overview and Status
Science Data Processing
Level-2 Science Software Implementation
Level-2 Data Product Format
Data Archive and Distribution
Aquarius Web Site
What We Need
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
13 of 29
July 19-21, 2010
Aquarius Level-2 Product Content
• This will actually be a combined Level 1B / Level 2,
radiometer/scatterometer product.
• The current product specification incorporates updated parameter
lists based on the recent code deliveries from the radiometer and
scatterometer teams.
• Specific parameters to be output are configurable at runtime.
• Parameters are output per beam and block (1.44 sec)
• All output products include standard metadata, flags and geolocation
fields.
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
14 of 29
July 19-21, 2010
Radiometer Parameters (1 of 2)
Name
rad_TaH
rad_TaV
rad_Ta3
rad_toi_H
rad_toi_V
rad_toi_3
rad_toa_H
rad_toa_V
rad_far_TaH
rad_TbH
rad_TbV
SSS
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
Long Name
Radiometer Ta H polarization
Radiometer Ta V polarization
Radiometer Ta 3rd Stokes
Radiometer Top of the Ionosphere Tb H
polarization
Radiometer Top of the Ionosphere Tb V
polarization
Radiometer Top of the Ionosphere Tb +45
polarization
Radiometer Top of the Atmosphere Tb H
polarization
Radiometer Top of the Atmosphere Tb V
polarization
Radiometer Faraday Angle
Earth surface Tb H polarization
Earth surface Tb V polarization
Sea Surface Salinity
Implementation
Current
Current
Current
Current
Current
Current
Current
Current
Current
Current
Current
Current
15 of 29
July 19-21, 2010
Radiometer Parameters (2 of 2)
Name
rad_galact_Ta_dir_V
rad_galact_Ta_dir_H
rad_galact_Ta_dir_3
rad_galact_Ta_ref_V
rad_galact_Ta_ref_H
rad_galact_Ta_ref_3
rad_solar_Ta_dir_V
rad_solar_Ta_dir_H
rad_solar_Ta_dir_3
rad_solar_Ta_ref_V
rad_solar_Ta_ref_H
rad_solar_Ta_ref_3
rad_rain_Tb
rad_moon_Ta
rad_rough_corr
scat_rough_corr
rad_o2_wv_abs
rad_atm_emiss
rad_cosmic_Ta
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
Long Name
Radiometer galactic direct correction V polarization
Radiometer galactic direct correction H polarization
Radiometer galactic direct correction 3rd Stokes
Radiometer galactic reflected correction V polarization
Radiometer galactic reflected correction H polarization
Radiometer galactic reflected correction 3rd Stokes
Radiometer solar direct correction V polarization
Radiometer solar direct correction H polarization
Radiometer solar direct correction 3rd Stokes
Radiometer solar reflected correction V polarization
Radiometer solar reflected correction H polarization
Radiometer solar reflected correction 3rd Stokes
Rain effect on brightness temperature
Lunar effect on antenna temperature
Derived wind and/or roughness correction
Scatterometer-based TB roughness correction
O2/water vapor absorption
Atmospheric emission
Radiometer Cosmic Ta correction
Implementation
Current
Current
Current
Current
Current
Current
Current
Current
Current
Current
Current
Current
Planned
Planned
Planned
Planned
Planned
Planned
Planned
16 of 29
July 19-21, 2010
Scatterometer Parameters (1 of 2)
Name
Long Name
TOI Scatterometer normalized radar crosssection (NRCS) for VV polarization
scat_HH_ant
TOI Scatterometer NRCS for HH polarization
scat_HV_ant
TOI Scatterometer NRCS for HV polarization
scat_VH_ant
TOI Scatterometer NRCS for VH polarization
Scatterometer NRCS for VV polarization after
scat_VV_TOA
Faraday rotation correction
Scatterometer NRCS for HH polarization after
scat_HH_TOA
Faraday rotation correction
Scatterometer NRCS for HV polarization after
scat_HV_TOA
Faraday rotation correction
Scatterometer NRCS for VH polarization after
scat_VH_TOA
Faraday rotation correction
scat_wind_speed Scatterometer wind speed
scat_VV_ant
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
Implementation
Current
Current
Current
Current
Current
Current
Current
Current
Current
17 of 29
July 19-21, 2010
Scatterometer Parameters (2 of 2)
Name
Long Name
Delta Tb derived from scatterometer data for V
polarization and beam
Delta Tb derived from scatterometer data for H
dTB_H
polarization and beam
Kpc_VV
Kpc statistical uncertainty for VV NRCS
Kpc_HH
Kpc statistical uncertainty for HH NRCS
Kpc_HV
Kpc statistical uncertainty for HV NRCS
Kpc_VH
Kpc statistical uncertainty for VH NRCS
wind_uncertainty
Estimated wind speed error
dTB_V_uncertainty Uncertainty in delta-Tb-V
dTB_H_uncertainty Uncertainty in delta-Tb-H
scat_total
NRCS for total power
Kpc_total
Statistical uncertainty for total power NRCS
dTB_V
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
Implementation
Planned
Planned
Planned
Planned
Planned
Planned
Planned
Planned
Planned
Planned
Planned
18 of 29
July 19-21, 2010
Agenda
•
•
•
•
•
•
•
Overview and Status
Science Data Processing
Level-2 Science Software Implementation
Level-2 Data Product Format
Data Archive and Distribution
Aquarius Web Site
What We Need
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
19 of 29
July 19-21, 2010
ADPS Data Archive and Distribution
• Overview of the currently operational web-based browse, order and
distribution tool that is available to support the distribution of
Aquarius data within the Aquarius Data Processing System that also
serves as a data quality monitoring tool for both internal and
external use.
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
20 of 29
July 19-21, 2010
Aquarius Level-1-2 Data Browser
http://oceancolor.gsfc.nasa.gov/cgi/aquarius
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
21 of 29
July 19-21, 2010
Aquarius Level-1-2 Data Browser
http://oceancolor.gsfc.nasa.gov/cgi/aquarius
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
22 of 29
July 19-21, 2010
Aquarius Level-3 Data Browser
http://oceancolor.gsfc.nasa.gov/cgi/l3
All Beams - daily
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
23 of 29
July 19-21, 2010
Agenda
•
•
•
•
•
•
•
Overview and Status
Science Data Processing
Level-2 Science Software Implementation
Level-2 Data Product Format
Data Archive and Distribution
Aquarius Web Site
What We Need
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
24 of 29
July 19-21, 2010
Aquarius Website
Consolidated
Data access,
Information,
Services and
Community Feedback
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
25 of 29
July 19-21, 2010
Aquarius Wiki
Online Documentation and Software Descriptions available online at:
http://aquarius.nasa.gov/
Under Data and Documents Link labeled
“Aquarius Ground System and Mission Operations Wiki Documentation”
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
26 of 29
July 19-21, 2010
Agenda
•
•
•
•
•
•
•
Overview and Status
Science Data Processing
Level-2 Science Software Implementation
Level-2 Data Product Format
Data Archive and Distribution
Aquarius Web Site
What We Need
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
27 of 29
July 19-21, 2010
What We Need
•
•
•
•
Review and analysis of the 2007 simulation products
Agreement on the ancillary data sets and sources.
Agreement on the Level-2 parameters.
Agreement on the at-launch processing baseline (e.g., NCEP
winds).
• Schedule and responsibility for algorithm and software updates
between now and launch.
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
28 of 29
July 19-21, 2010
Backup Slides
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
29 of 29
July 19-21, 2010
Ground System Interfaces
S-Band
S-Band
ASI Malindi
Wallops Svalbard Alaska McMurdo
NASA Ground Network
Commands
Real-Time
Telemetry &
Coordination
Data
Archive
CUSS
Algorithms &
Science Code
Science Team
and
Algorithm Providers
AVDS
Aquarius Validation
Data System
Evaluation
Products
RF
Services
Cordoba
Aquarius
Command
Schedules
And Reports
Schedule
Confirmation
And Reports
ASI Matera
ADPS
ACCS
Calibration &
Instrument
Aquarius
Data
Processing
System
Aquarius
Command &
Control
System
Configuration
Change Requests
Insitu Data
Bias Adjustments
Principal Investigator
and
Science Planning Team
Aquarius Instrument Testbed
Flight Software Updates
Instrument
Engineering
Team
Level 1, 2, 3
Aquarius Data
PO.DAAC
Aquarius Match-up
Data
1- SAC-D Ground System to Aquarius Ground System ICD
2- SAC-D / NASA PSLA, Network Requirements Document & Scheduling Interface
3- SD - Observatory Ground IF Specification Document
4- Aquarius Science Team to Aquarius Ground System ICD
5- AQ - Aquarius Ground System – Aquarius Instrument Team Anomaly ICD
and Aquarius Test bed User Manual
6- Aquarius Data Product User Guide (PO.DAAC) & Format Specification (ADPS)
7- Ocean Data Processing System Data and Software Management Plan, ATBD’s
and Software User’s Guides
8- A V D S to A D P S ICD
9- Aquarius Ground System to PO.DAAC ICD
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
X-Band
Mission
Operations
CODS Command
Center
Aquarius
Data, SAC-D
Telemetry
and Orbit
Products
Ancillary Data
Sources
X-Band
CONAE
Aquarius L1, 2, 3 Data
Ancillary Data
Feedback
SeaDAS
Software
Data Users
and
Science Team
30 of 29
July 19-21, 2010
Aquarius Software Status
For each software module, we will show:
• High-level flow (inputs and outputs).
• Current status and capabilities.
• Remaining work to be done.
Input File
Intermediate
Data
Processing
Module
Output File
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
31 of 29
July 19-21, 2010
Level-0 Preprocessor
Implemented by Joel Gales
Level-0
Downlink
File
l0gen
Current Capabilities:
1.
Processes a Level-0 downlink file
at CONAE to remove duplicate
blocks and sort by time.
To be done:
1.
Final data quality checks.
Level-0
File
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
32 of 29
July 19-21, 2010
Level-0 to 1A
Implemented by Joel Gales
SAC-D
Ephemeris
Data
Current Capabilities:
Level-0
File
SAC-D
Telemetry
1.
2.
l1agen
3.
4.
Processes a Level-0 multi-orbit
file to multiple Level-1A orbit files
with overlap.
Inputs SAC-D ephemeris and
telemetry from separate files.
Unpacks Aquarius science data.
Converts Aquarius telemetry
to engineering units.
5.
Level-1A
Files
To be done:
1.
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
Generates HDF5 L1A files.
Implement actual SAC-D
telemetry input file formats when
these are known.
33 of 29
July 19-21, 2010
Level-1A Merge
Implemented by Joel Gales
Level-1A
Files
l1amerge
Current Capabilities:
1.
Merges multiple Level-1A files
for the same orbit from
successive downlinks.
To be done:
1.
Data quality checks to keep only
“best” data.
Level-1A
Product
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
34 of 29
July 19-21, 2010
Level-2 to 3 Binning
Level-2
Products
Implemented by Joel Gales
based on current OBPG
binning software
Current Capabilities:
1.
l2bin
2.
3.
Level-3
Binned Beam
Product
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
Processes multiple Level-2
products (e.g., for 1 week).
Average samples for each beam
in 1 degree bins.
Outputs a Level-3 binned
product per beam.
To be done:
1.
Add flag exclusion rules.
35 of 29
July 19-21, 2010
Level-2 to 3 Smoothing
Implemented by Joel Gales
based on algorithm from
Jonathan Lilly
Level-2
Products
Current Capabilities:
1.
2.
l2smooth
3.
Level-3
Smoothed
Product
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
Processes multiple Level-2
products (e.g., for 1 week).
Generates smoothed values in 1
degree bins.
Outputs a Level-3 smoothed
product.
To be done:
1. Additional smoothing
algorithms or options.
2. Add flag exclusion rules.
36 of 29
July 19-21, 2010
Level-3 Binning
Level-3
Binned
Products
Implemented by Joel Gales
based on current OBPG
binning software
Current Capabilities:
1.
l3bin
2.
3.
Processes multiple Level-3
binned products.
Averages samples in common
bins.
Outputs a single Level-3 binned
product.
Level-3
Binned
Product
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
37 of 29
July 19-21, 2010
Level-3 Mapping
Level-3
Binned
Product
Implemented by Joel Gales
based on current OBPG
binning software
Current Capabilities:
1.
2.
l3bin
Reads a Level-3 binned product.
Outputs a Level-3 mapped (latlon gridded) product.
Level-3
Mapped
Product
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
38 of 29
July 19-21, 2010
OBPG Computing Facility
NASA Goddard Space Flight Center
Greenbelt, Maryland
Server location: Building 28 Room W220
- 900 square feet dedicated to the project
- Raised computer floor with under-floor A/C
- Primary and backup A/C units
- 80 KVA Powerware UPS unit with dual 50 KVA
PDUs, additional 5 KVA rack-mounted UPS units
- Connections to GSFC high speed networks: SEN
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
39 of 29
July 19-21, 2010
OBPG System Hardware
Distribution and Storage
Servers (web)
Distribution Servers
(ftp)
Aquarius Telemetry
Viewer
Production: 3 processing nodes
75 storage nodes 1.03 PB
8 storage nodes 49.3 TB
1 server 250GB
2 Windows PC
Aquarius Command
Planning Tool Servers
Ingest Servers
2 SeaSpace workstations 78
GB
1 storage node 5.1 TB
2 processing nodes 586 GB
AQOPS Servers (web)
Production: 2 processing nodes
Processing Cluster
46 processing nodes 15 TB
4 storage nodes 10.9 TB
Extreme Networks
Black Diamond 8810
Gigabit Ethernet switch
Database Servers
480 ports
2 superstore nodes 16.4 TB
Forum Servers (web)
Production: 2 processing nodes
Subversion CM Servers
2 superstore nodes 586 GB
Backup Servers
2 storage nodes 19.2 TB
Development Servers
2 superstore nodes 10 TB
Network Support
Servers
Mission Operations
Systems
Testing Cluster
User Desktops
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
Cal/Val & QC Systems
40 of 29
July 19-21, 2010
Aquarius Backup Strategy
• Data
• On-site
• Downlinks, L0, L1a data files
• primary copy on RAID6 storage node
• mirror copy on separate RAID6 storage node, at file creation
• Science Products
• primary copy on RAID6 storage node
• mirror copy if desired, or regenerated from L1a
• Off-site
• Downlinks: stored at CONAE
• L1A archived at PO.DAAC
• Source Code
• On-site
• primary copy on subversion server, RAID6
• mirror copy on backup Subversion server, RAID6, nightly mirror
• Off-site
• Subversion data copied to external disk, taken off-site every 1 month
• Multiple copies, given to Project Manager, SE, Processing Manager
• Other
• On-site
• AQOPS, Forum: database mirrored to backup server nightly
• Telemetry Viewer, other: image backups periodically, manual
• Off-site
• AQOPS, Forum: databases included in source code off-line backups
• Telemetry Viewer, other: image backups stored off-line like source code
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
41 of 29
July 19-21, 2010
IT Security Plan
•
ADPS and ACCS: covered under the same plan as other Ocean Biology
Processing Group (OBPG) systems.
•
NASA IT Security Plan Identification
– System name: Sciences and Exploration Directorate (SED) MultiProgram/Project (MPP) IT Science Systems
– System Abbreviation: SED MPP
– System Unique Identifier: CD-014-L-GSF-6004
– Information System Type: General Support System
– Information System Security Categorization: Low (transitioning to
medium within the year)
– Information System Operational Status: Operational
•
Effective 5/31/2007. Passed Certification & Accreditation in Summer 2007 in process of being upgraded
•
Plan designated Sensitive But Unclassified by NASA
•
Implementation questions: addressed off-line
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
42 of 29
July 19-21, 2010
Ground System Staffing
NASA Management - Gene Feldman: Oversight of the overall effort, communication/coordination with other NASA organizations and
centers and CONAE; oversight of science support, algorithm integration, QC, validation and in situ data support.
Contractor Management - Fred Patt: Oversight of contract support staff, including hiring, tasking, review of deliverables and
schedules; ADPS interface definition and product format specification.
Mission Operations - Susie Kennison: Technical lead for instrument operations and commanding support; interface to instrument
engineers and CONAE mission operations.
Data Systems Engineering - John Wilding: Technical lead for the data system integration effort; overall system design,
implementation and testing.
Software Engineering – Joel Gales: Technical lead for the level conversion / product generation software, including development of
the Level 0-to-1 software and integration/packaging of the science (Level 2 and 3) processing software to be provided by the algorithm
developers.
Data Systems Support - Dan Knowles: Integrate the product generation software into the existing automated processing system,
develop ingest modules, develop database tables for archive products, develop schedules and recipes for data acquisition, production
and migration.
Systems Administration - Paul Smith: Specify and acquire new processing and data storage hardware, integrate into existing data
processing facility, verify/upgrade network capability for external data transfers.
Quality Control and Validation - Liang Hong: Working with the Science Team, develop methodologies, tools, procedures and reports
for operational QC; specify data and develop algorithms and tools for data and science algorithm validation.
In Situ Data Support - Jeremy Werdell: Work closely with the Aquarius Validation Data System (AVDS) to develop methods and tools
for data analysis and quality control, building upon the existing SeaWiFS Bio-optical Archive and Storage System (SeaBASS).
Archive and Distribution - Norman Kuring: Incorporate Aquarius product browse, search, order and distribution into existing webbased Ocean Color capability and work with JPL PO.DAAC regarding “institutional” archive support requirements.
ADPS and Data Distribution
Aquarius/SAC-D Science Team Meeting
43 of 29
July 19-21, 2010