NOAA Testbeds and Proving Grounds:
Fostering Transitions between Research
and Applications/Operations
Paula Davidson, Ph.D.
Chief Scientist
Office of Science & Technology
NOAA/National Weather Service
Co-chair, NOAA Testbeds & Proving Grounds Coordinating Committee
1
Background
NOAA operates a number of testbeds (TB) and operations and services proving
grounds (PG), but lacked a systematic approach for their function, mode of
operation, and reporting results
March 2011- NOAA Line Office Transition Managers approved guidelines and
chartered a Coordinating Committee for Testbeds & Proving Grounds (TB/PG)
•
Guidelines:
•
•
•
•
Roles and responsibilities of TB/PG
Function and execution
Governance
Coordination:
•
Increasing R&D leveraging, collaboration for R2O
Increasing reliance on TB/PG for advancing R2O/A/U
• NOAA re-chartered COMT (2013)
• CSTAR FFO links results to testing in OPG; use of NWS Virtual Laboratory (FY14)
• Planning for Next Generation Global Prediction System
Brief working definitions of TB/PG:
TB-- for developmental testing
PG-- for pre-deployment (experimental) testing R&D
of advanced operations, services and science
and technology capabilities
2
Current NOAA TB/PG
NOAA Testbed and Proving Ground Portal
AWT
CTB
GRPG
COMT
HWT
JHT
DTC
HMT
OPG
http://www.testbeds.noaa.gov
JCSDA
SWPT
(thanks to : Rich Lataitis & Barb deLuisi)
3
Framework for Transition:
Phased Implementation into NWS Operations
Phase
Key Q
Key Metric
Facility
Does it work?
Peer-reviewed
Publication
TB Developmental
Works with
operational
systems?
Feasibility/ Engineering Testbed with
Analysis Successful
operations-like
environment
Experimental
PG Testing
Meets operational
performance
criteria?
Go/No Go based on:
Objective Performance
(e.g. accuracy)
Subjective Feedback
Production Readiness
R&D
Testing
Operations
Universities,
Government
Labs, Private
Industry
Operational
proving ground
for clinical tests
and full “dress
rehearsal”
Maintains required Objective criteria:
Operations
performance?
accuracy and reliability
4
Coordinating Committee
 Comprised of TB/PG managers and LOTM-designated LO focal points:
 Co-chairs: Paula Davidson (NWS LO focal point) Mike Uhart (OAR LO focal point)
 Initial meeting (NWS and OAR members): Seattle AMS, Jan 2011
 Featured Activities:
 Monthly virtual meetings featuring TB/PG recent tests/results
 Coordination/Outreach: revamped TB/PG coordination website www.testbeds.noaa.gov,
coordinated SEE inputs, tutorials on guidelines, testing protocols , coordinated AOs
 Organize annual science-themed TB/PG workshops
 Adopted recommended approach to metrics for recognizing progress, Fall 2012:
 Relevance, quality, and effectiveness/efficiency
 Accomplishments and performance metrics provided in annual TBPG progress reports
 Summary publication (Ralph, Intrieri, et al.): The emergence of weather-focused testbeds
linking research and forecasting operations. BAMS, 94, 1187-1210, 2013
5
Annual Workshops
 Highlight/promote integrated testing opportunities in NOAA TB/PG
 Overviews from current/emerging TB/PG
 Integrating Science Themes:
 3rd Annual TBPG Workshop: Intense Precipitation Events (2012)
 4rd Annual TBPG Workshop: High-impact Environmental Events (2013)
 5th Annual TBPG Workshop: Advances in Environmental Prediction (2014)
 Best paper (Science-theme) competition
 Awarded to Faye Barthold (WPC, 2012); Tom Workoff (WPC, 2013); Chad
Gravelle (OPG, 2014)
 Criteria - excellence, relevance, presentation
6
FY 2014 Activities
 TBPG Workshop: College Park, MD, April 16-18
 Science Theme: Advances in Environmental Prediction
 Best Paper: Chad Gravelle, “Transitioning the GOES-R Fog and Low Stratus
Products from Research To Operations Through the NWS Operations
Proving Ground”
 Closer connection with CSTAR , MDL’s Virtual Laboratory and pilot
activities: advanced testing
 Increasing coordination, external engagement
 Combined Special Testing Opportunities
 In-residence and virtual testing
 Consolidation on Website: Announcements of Opportunity
7
Thank You!
Contact Information:
Paula Davidson – Paula.Davidson@noaa.gov
8
Backup
9
FY13 Highlights: COMT
• COMT results featured in a special issue of JGR-Oceans:
10
FY13 Highlights: DTC
Hurricane-WRF (HWRF) testing for operational implementation at NCEP
• HWRF provides guidance to the
National Hurricane Center for
the North Atlantic and Eastern
North Pacific basins
• Upgraded HWRF increased skill
in forecasts of hurricane
intensity
• Regional model 27/9/3 km
• HWRF has 8 components, many
used in other applications
• Developmental Testbed Center
roles:
– Support code to community
– Code management
– Testing and evaluation (R2O)
Operational forecasts
http://www.emc.ncep.noaa.gov/gc_wmb/vxt/
g
11
FY13 Highlights: GOES-R PG
• NWS Central Region Fog & Low Stratus Evaluation, Aug-Dec 2012
•
Products demonstrated successfully – scheduled for delivery via NWS operational systems
• High latitude & Arctic testing
•
snow/fog, low stratus/volcanic ash, aviation applications
• GOES-R Algorithm Development & Testing
•
Simulated satellite imagery: lightning mappers, nearcasting data, convective initiation,
cloud-top cooling
• Partnerships
•
NWS: NCEP/AWT, HWT, WPC, OPC, Regions; NESDIS : SAB; CIRA, CIMSS, NASA/SPoRT, DoD,
private sector…
Heritage GOES Fog
Detection
GOES-R Algorithm
Applied to GOES
GOES-R Algorithm
Applied to GOES-R
courtesy
Mike Pavalonis,
NESDIS/STAR
12
FY13 Highlights: HMT
Flash Flood and Intense Rainfall Experiment (July 8-26, 2013)
• 26 forecasters, researchers
and model developers from
WPC, NSSL, ESRL and HMT
collected to examine
alternatives for short term
QPF and flood prediction
QPE
NAM- X Nest
HRRR
• Several operational and
research models tested
12-hr mean QPF
valid 2013 07/18 0Z
NAM Nest
NSSL WRF
HRW ARW
Operational forecasts
http://www.emc.ncep.noaa.gov/gc_wmb/vxt/
13
FY13 Highlights: HWT
Experimental Warning Testing**
• LAPS Space and Time Multiscale Analysis System (STMAS)
•
Forecast return moisture and instability very useful for determining if storms will change
intensity in 0-2 hours
• WDSSII Multiple-Radar/Multiple Sensor (MRMS) applications
•
•
•
2-minute updates of MRMS products helps keep situational awareness high
Greater efficiency in storm diagnosis
Aided in positioning of storm-based warning polygons
• Hail Size Discrimination Algorithm (HSDA)
•
•
Very beneficial in breaking down areas of
severe vs. non-severe hail
Spatial extent of giant hail tended to be too
large at times
• GOES-R Algorithm Development and
Testing
•
Simulated satellite imagery: lightning mappers,
nearcasting data, convective initiation, cloudtop cooling
** additional highlights from experimental forecast testing not shown
14
FY 13 Highlights: JCSDA
Improved prediction of snow depth
by NOAH/GFS
•U of Arizona effort, funded by NOAA’s JCSDA
external research program
•Collaboration with NESDIS CRTM and NWS
land data assimilation teams
•Improved roughness length led to increased
skill in snow depth prediction by GFS/NOAH
and WFR/NOAH
•Implemented in GFS/NOAH land model
(latest release, v 3.5).
Black: observations
Green: control GFS
Yellow: GFS + improved roughness length
Red: GFS + all improvements
averaged (over 100-120degW, 50-60degN) for 3/2011
JCSDA annual report; figure courtesy of X. Zeng, U Arizona
Testing new snow depth algorithms led to improved snow
characterization in land-surface component (NOAH) for
NCEP’s operational GFS (release v3.5) and WRF models. 15
FY 13 Highlights: JHT
Uhlhorn , Hurricane Research Division:
Stepped-frequency microwave
radiometer bias-corrected hurricane
winds
Ginis/Bender, U Rhode Island/
Geophysical Fluid Dynamics Laboratory:
Improved hurricane models
11 Joint Hurricane Testbed projects completed their two
years of testing during the 2013 hurricane season
16
FY 13 Highlights: OPG
• Virtual AWIPS proof of concept
• GOES-R Fog/Low Stratus Tool
• Support for AWC TB Experiment
• Programming for Winter Hazard Message Simplification
• Finalizing Test Plan with NASA SPoRT/GOES-R for
Operational Readiness Evaluation in March 2014
• Test plan developed for Hazards Services
Hydro Tool
17
FY13 Highlights: SWPT
• Tested upgrades to operational Wang-Sheeley-Arge Enlil model for
prediction geomagnetic storms, now implemented
• Tested new coronal mass ejection analysis tool, now implemented
• Tested Space Environmental Anomalies Expert System Real Time
for GEO satellite operators, now implemented
• Evaluated and selected magnetosphere/geospace model for
transition to operations (with NASA)
• Tested ionosphere and thermosphere
products
• Ovation Prime aurora forecast model, now
implemented
• Upgrade/expansion to US Total Electron Content
model – N America domain ready for
implementation; global product in testing
18