HOPE
0.02
HOPE
200
REPT
MagEIS
Medium x2
MagEIS
Low
REPT
MagEIS (4)
New LX file
available
New LX processing
code, new
dependency
Get LX data, place
in file system
Update
database
Do LX -> LX+1
processing
Process
Control
Database
Query database for
processing step needed
Contains file
version info,
location info,
processing step
info (codes,
required input
files,
dependencies)
Quality Control Loop
Have LX+1 data,
place in file system
The ECT-SOC handles data from 2
HOPE, 2 REPT and 8 MagEIS
instruments with 4 data levels of
several data products per level (~160+
data products) and a range of ancillary
data (ephemeris, magnetic field).
Update
database
Filename example: rbsp-a-hope_pre_L2_20130506_vX.Y.Z-R.cdf where:
X – interface, representing significant change on codes /contents
Y – quality, representing change in contents with no impact on codes
Z – bug fix/revision, representing minor changes
R – release (optional) to group a given collection of files into a single release.
ECT-SOC Functional Diagram
•
The ECT-SOC has two components:
– SOC-CTG (Command, Telemetry and GSE): Provides
instrument team’s GSE functionality during I&T,
commissioning and normal operations, communicates
with MOC for commanding / data and housekeeping
reception; quicklook data & HK
– SOC-SDC (Science Data Centre): Receives Level 1 data
from the SOC-CTG and performs all higher level data
processing, ingests ancillary data and maintains ECT data
website
ECT-SOC Data Products:
Levelz
L0
Description
Raw de-commutated telemetry data from
MOC.
L1
L0 + sorted, time-tagged, instrument
separated units of count rate.
L2
L1 + calibrated and corrected (bkg. deadtime, etc.), physical units.
L3
L2 + B-field derived science products (pitch
angles, moments, L-sorted).
L4
L3 + PSD units derived using B-field
models, magnetic coordinates.
•
Level 0 - Raw science telemetry data from MOC (Mission Ops Center @ APL)
•
Level 1 – Time sorted (MET-UT conversion) files per real day. Science, HK data in engineering units. ECT Team internal only.
•
Level 2 – Calibrated and corrected Science data in physical units, per detector, sector, full time resolution. Publically available.
http://www.rbsp-ect.lanl.gov/data_pub/rbspa(b)/(hope)(mageis)(rept)/level2
•
Level 3 – Magnetic field derived Science products. Under development, not public yet
•
•
–
Pitch angle information added for each detector, sector. One default pitch angle/energy data product per instrument. PAPCO allows for arbitrary pitch angle/energy data product.
–
Magnetic model information added to ephemeris (model field, magnetic coordinates). One default L-sorted data product per instrument. PAPCO allows for arbitrary L-sorted product.
–
Plasma moments (HOPE)
Level 4 – Data converted into PSD units and sorted according to the three adiabatic invariants – μ, K, L* . Under development, not public yet
–
Instrument inter-calibration on each RBSP S/C
–
Inter-calibration between RBSP S/C
–
Inter-calibration with ancillary data (LANL GEO, GPS, etc)
Ancillary Data – data from other instruments (LANL GEO, GPS, GEOS etc) will be included at their best quality availability. ECT-SOC will use what
is available, but will not be responsible for any data processing work for these data.
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy’s NNSA
Such processing cannot be handled
“manually” but requires a database
driven data processing and versioning
scheme.
This scheme is fully automated and
keeps track of all file and processing
versions – any specific file version can
always be re-created.
Any new file added to the system
triggers all required processing steps.
Am change in the version of a data
file, support file or processing code
triggers all required re-processing and
version number updates.
The quality control loop is an integral
part of this scheme and also tracked in
the process database.
ECT Web Portal
ECT-SOC Web Work Flow
The ECT-SOC at LANL is the central SOC for the ECT Team
and will handle all commanding, telemetry and data
processing for HOPE, MagEIS and HOPE.
Launch PNG Walk
(pre-loaded
bookmarks for ECT,
Ancillary summary
plots)
Browse Plots
(can launch
Autoplot with
product)
Launch Autoplot
(pre-loaded bookmarks for
ECT data, Ancillary data,
products)
Complex
Plot
(many
panels)
Get Digital
data
(PRBEM CDF
files)
Export
(Excel, ASCII,
bin, .. Can be
scripted)
Data is provided is self describing
format (CDF, Rich ASCII). Autoplot
provides rendering and some analysis
capability.
Direct “java-bridge” access to data
into IDL or Matlab allows seamless
access to anything Autoplot can render.
Make Summary Plots
(publish to local or
remote site)
Autoplot is Java based requiring a
modern Java environment, which is
standard on Unix, Windows and Mac.
Autoplot uses the Webstart protocol
which handles all install and update tasks.
ECT-SOC approach is to provide an
integrated data browse (PNG Walk)
and rendering (Autoplot) environment.
For detailed analysis data can be
imported into a variety of other
languages and/or software packages
(PaPCo, TDAS, Matlab, IDL).
Simple
Plot
(File,
variable)
ECT-SOC Quality Control
•
ECT-SOC Data
Product Details
MagEIS
High
ECT-SOC Processing
RBSP-ECT consists of three
coordinated sensor types:
•HOPE
•MagEIS
•REPT
ECT measures electrons
(continuously) and ions (with
composition up to 50 keV) from ~20 eV
to ~10’s of MeV with energy
resolution, and pitch angle coverage
and resolution required for mission
success.
Relative Electron Flux
friedel@lanl.gov
Relative Electron Flux
RBSP-ECT Instruments
SM31C-2348
ECT SOC website:
http://www.rbsp-ect.lanl.gov/
Use own tools
(“I’ll do it my
way”)
Autoplot Website:
http://autoplot.org/
Same Loop for all
L0->L1->L2->L3->L4
processing
UNCLASS IFIED
Autoplot / PNGWalk Examples
ECT = Energetic particle, Composition and Thermal plasma
R. H. W. Friedel ; M. G. Henderson, Steven Morley, G.D. Reeves and J. Niehof (LANL)
PSD/MagEphem Processing
Scientific Data Processing for the ECT Suite on NASA’s Van Allen
Probes mission
Los Alamos is leading the Science Operations Center (SOC) for the Energetic Particle, Composition, and Thermal Plasma Suite (ECT)
for NASA's upcoming Radiation Belt Storm Probe Mission (RBSP). Here we present the ECT team's facilities for making data available
to both the ECT team and the public. The ECT-SOC will make extensive use of resources such as Autoplot, which will be tightly
integrated into a modern version and process control system developed at LANL.
In order to support the scientific investigations at the heart of the RBSP mission we make use of existing LANL resources to provide
data in PSD format at constant adiabatic invariants, using the LANLGeoMag library and processing pipeline developed for the LANL
DREAM (Dynamic Radiation Environment Assimilation Model).
ECT-SOC website: http://www.rbsp-ect.lanl.gov
• The ECT SOC has a mandate to produce the magnetic ephemeris for the RBSP mission.
Available on ECT website, direct data path: http://www.rbsp-ect.lanl.gov/data_pub/MagEphem
• “pre” preliminary data files are being produced 4 weeks into the future (OP77Q, T89Q).
• “def” definite data files are produced with a 1-6 day lag depending on availability of the definite
kernel from APL (OP77Q, T89Q routinely, T89D,TS04D on demand).
• Also produce rbspa-pp and rbspb-pp files of orbit # & perigee-perigee times (use in autoplot).
• Autoplot can use these MagEphem files to add extra tickmarks to any autoplot plot.
• Files are “Info Rich”, L, L*, coords, footpoints, etc etc..
• “QinDenton” real-time solar wind, magnetospheric indices and “W” parameter files (provided
by APL) are used for dynamic model inputs and are available for context.
http://www.rbsp-ect.lanl.gov/data_pub/QinDenton/
• The ECT SOC has a large investment in infrastructure and codes (LANLGeoMag) to
convert data from physical space (energy, pitch angle, position) to phase space density
(PSD) at fixed adiabatic invariants – μ, K, L* for use in modeling, data assimilation and
science studies
• Uses MagEphem files and energy/pitch angle data matrices as input together with in-situ
magnetic field data
• Provides the input files for downstream modeling (DEAM code)
• Provides inverse transformation of PSD data to physical space along satellite trajectories.