DTC Management Board Meeting
24th January 2013
Bill Kuo
1
Topics for DTC MB meeting:
 AOP 2013 plan and priorities
 Execution of AOP 2013:
 NOAA-UCAR Cooperative Agreement
 Transition to next phase of DTC
 DTC Science Advisory Board membership
 DTC sponsors’ funding and priorities
 DTC Executive Committee (EC) meeting preparation
 Challenges
2
Planning for AOP 2013
 Preliminary guidance for DTC is ‘flat budget,’ however, potential cuts
are possible.
 DTC has taken into account the initial guidance provided by DTC MB
during the October 2012 MB meeting
 Final budget for DTC will most likely not be known until Spring 2013
 DTC Task Leads will present:
 Accomplishments in 2012
 Proposal for 2013
 DTC MB will need to decide on:
 What tasks fall within the 85% budget level?
 The ranking of tasks between 85% to 100% budget level
3
DTC funding sources (in $K)
Funding source
NOAA/OAR
FY2011
2,994
FY2012 Difference
2,925
-69
NOAA/HFIP
USWRP
GSD
708
281
250
460
190
250
-248
-91
AFWA
NCAR
NSF
718
250
100
855
250
100
137
129
5,430
268*
5,298
139
-132
Carry-over
Total
* Due to period of performance for some projects, some funding is
‘committed carry-over’ for AOP 2012.
4
DTC budget allocations (in $K)
Funding allocation
Director’s Office
Visitor Program
Mesoscale Modeling
Hurricane
Data Assimilation
Ensemble
Verification
Total
FY2011
FY2012 Difference
755
746
-9
200
1,037
1,197
200
1,029
1,012
-8
-185
772
846
623
569
827
915
-203
-19
*292
5,430
5,298
-132
*Note verification support for T&E activities is included in the Verification task
in AOP 2012. In AOP 2011, they resided with task areas conducting the test.
5
Execution of AOP 2013
 DTC AOP has adopted the period of performance of March 1 –
February 28.
 Current NWS-UCAR Cooperative Agreement (CA) will end by
August 2013. NWS indicated that:
 NCAR budget through August 2013 can be transferred via NWS-
UCAR CA
 Remainder of funds needs to be transferred using OAR-NSF CA
(subject to NSF cost-recovery fee)
 NOAA will administer a competitive RFP in 2013 for the next
DTC CA
 When will the process be completed?
 How would this impact AOP 2014 planning?
6
DTC Science Advisory Board
 DTC Science Advisory Board consists of 14 members.
 Six SAB members whose terms will expire by June 2013:






Brian Colle, SUNY Stony Brook
James Doyle, NRL
Bob Dumais, ARL
Cliff Mass, U. of Washington (chair)
Tom Henderson, ESRL
David Bright, AWC
 DTC MB needs to discuss:
 Who should be retained?
 Nomination of new members
 Is the operation of SAB effective? Suggestion for change?
DTC Science Advisory Board
STC Sponsors’ Funding and Priorities
 DTC funding sponsors include NOAA/OAR, NOAA/HFIP, USWRP,
GSD, AFWA, NCAR, NSF
 HFIP, USWRP, AFWA and NSF have provided guidance on the
allocation of their funding to support specific tasks that fall within the
core areas of DTC
 NOAA/OAR, GSD, and NCAR have allowed flexibility to allocate
funds according to priorities set by DTC MB
 Recently, questions have been raised with regards to the allocation of
NOAA/OAR funds:
 Should NOAA (or OAR) provide guidance on the priority for
NOAA/OAR funds (instead of the entire DTC MB)?
 This change will have a significant impact on DTC operation, as
NOAA/OAR is the largest funding source for DTC
9
DTC EC Meeting Preparation
 DTC EC meeting will be held 12 Feb 2013 in Silver Spring
 Executive Committee will:
 Review and approve DTC AOP 2013 and priorities
 Review and approve DTC SAB membership
 DTC MB needs to identify important issues/topics for DTC EC
discussion
 Next phase of DTC, RFP process, use of NOAA/OAR funds
 Future direction of DTC
 DTC partnership between NOAA, AFWA, NCAR, and NSF
 Others?
10
Challenges
 Some DTC task areas are becoming subcritical due to budget reduction:
 Should we consider consolidation of task areas?
 Should DTC focus on certain aspects (e.g., physics)?
 SAB recommended DTC put greater emphasis on T&E at the expense of
community support.
 Are we ready to take a large step back from conducting tutorials for our
publically-released software packages?
 Should we reduce the number of packages that we support to the community?
 How can DTC be more effective in R2O?
 Development and testing of next-generation NWP systems
 Migration toward a unified modeling system (a recommendation from the
UCACN)
 Development of 10-year strategic plan for EMC
 Establishment and operation of an “ECMWF-like” facility
11
Summary
 DTC has come a long way toward establishing a community
facility with a robust management structure and planning
process, strong community connection and partnership among
sponsors.
 Joint decision making process by the DTC MB is a very
important mechanism for:
 Setting priorities for DTC
 Maintaining the partnership among sponsors
 Setting future direction of DTC
12
Community Interactions
Louisa Bogar Nance
13
Outline
 Software Systems
 Community Outreach Events
14
Software Systems
Framework for bringing together operational capabilities and
research innovations to accelerate the transition of new technology
into operations by facilitating carefully-controlled extensive T&E
Close collaboration between DTC & developers
is critical to the success of this work!
15
Software System Philosophy
 Shared resource w/ distributed development that includes
capabilities of current operational systems
 On-going development maintained under mutually agreed upon
software management plan
 Code repository maintained under version control software
 Protocols for proposing & approving modifications to the software
 Testing standards
 Code review committee
 Additional testing standards to more thoroughly check integrity of
evolving code base
16
Accelerating R2O Transitions for GSI
Before and After: An Example
2008
GSD initiates merging the cloud analysis code to
the existing GSI
2009
GSD/DTC commit code changes to the GSI trunk
as a trial case to set up the GSI R2O transition
procedure
2010
2011
Can we wait this long for
any other R2O transition?
Issues related to:
• Version control
• Code portability
• Development coordination
• Formal code commit
procedure for external groups
• Coding standards
• Standard pre-commit tests
2012
17
Accelerating R2O Transitions for GSI
Before and After: An Example
2008
GSD initiates merging the code to the existing GSI
2009
GSD starts to use the GSI repository
Establish DTC community GSI repository with the
multi-platform feature
Establish EMC operational GSI repository
Lessons learned through the
2010
DTC’s experience
with
DTC
leads GSI
effort toare
form the GSI Review Committee
(GRC) and set up the GSI R2O procedure (including
GSD/DTC commit code changes to the GSI
being
applied to all other
software
repository syncing)
trunk(s) as a trial case to set up
the GSI R2O
transition procedure
systems we work
with!
DTC commits portability related changes to the GSI
All GRC members start to follow the R2O
transition procedure
Now, it takes about one week for GSI Review
Committee to review a code change proposal and
about one day to commit.
2011
trunk(s)
GRC finalizes the R2O procedure
2012
18
Current Software Systems
 WRF – NWP model + pre- and post-processors
 UPP – New package in 2011 – community code mgmt plan in process of being
implemented
 Model Evaluation Tools (MET) – verification package
 Gridpoint Statistical Interpolation (GSI) data assimilation system, including
GSI-hybrid capability
 WRF for Hurricanes - set of tools for tropical storm forecasting, including
coupled atmosphere and ocean system & stand alone GFDL vortex tracker
 Modular end-to-end ensemble system (repository & code mgmt plan
established during AOP 2011)
 NOAA Environmental Modeling System (NEMS) (repository established
during AOP 2011 – working towards code mgmt plan)
Software system management for GSI, NEMS and ensemble systems has
greatly benefitted from having DTC staff members, Hui Shao and
Eugene Mirvis, co-located with EMC staff!
19
Making New Capabilities Available to
Operations (2009-present)
WRF (including atmospheric component of HWRF)
 Enhanced interoperability for NMM-E, including moving nest
 Radiation – RRTMG
 Cumulus –Tiedtke, NSAS and Grell (uncoupled only)
 New capabilities (3-nest) and physics updates from AOML/HRD
HWRF
 Extension of POM coupling to Eastern Pacific (URI)
GSI
 NCAR/MMM’s aerosol optical depth data assimilation function
 Numerous contributions from GSD (e.g., cloud analysis) and GMAO (both
new features and enhancements to existing features)
MET
 Baldwin-Elmore spatial significance tool (DTC Visitor Program)
 Capability to convert TRMM satellite data into MET readable format
20
Contributions to Operational Software
Systems
Ensemble system
 Bias correction and downscaling for SREF
Ensemble Kalman Filter
 Working w/ EMC & ESRL to set-up a code management plan –
DTC will likely be a major contributor to this effort
NEMS
 Enhanced portability (DTC staff and DTC Visitor Program)
 Co-leading movement towards common repository for external
NCEP libraries
21
Publically-Released Packages
Philosophy
 Periodic releases made available
to the community that include
latest developments of new
capabilities & techniques
 Additional testing, including
multiple computing platforms
and compiler options
 Centralized support (in
collaboration with developers)




Software downloads
Documentation
Email helpdesk
Tutorials (online and onsite)
Current Packages
 WRF
 UPP
 HWRF
 GFDL vortex tracker
 GSI
 MET
22
Registered Users
Software
Initial
Release
Registered
Users 2009
Registered
Users 2013
WRF
Dec ‘00
~1,400*
17,225*
MET
Jan ’08
~300
~2,000
GSI
Sept ’09
0
679
HWRF
Aug ’11
0
500
GFDL vortex tracker
Aug ‘12
0
217
*All WRF users required to re-register starting in 2008 –
number corresponds to those who have registered since 2008
23
AOP 2012 Code Releases and Tutorials
Software
Code releases
Onsite Tutorials
WRF
v3.4 – 6 Apr 12
v3.4.1 – Aug 12
28 Jan – 1 Feb ’13
UPP
v2.0 – 1 Nov 12
28 Jan – 1 Feb ’13
HWRF
v3.4a – 29 Aug 12
none
GFDL vortex tracker
v3.4a – 29 Aug 12
none
GSI
v3.1 – 20 Jul 12
21-23 Aug 12
MET
v4.0 - 25 Jun 12
4-5 Feb 12
24
AOP 2013 Code Releases and Tutorials
85-100%
Software
Code releases
Onsite Tutorials
WRF
v3.5 – Apr 13
v3.5.1 – Aug 13
none
UPP
v2.1 – Apr 13
none
HWRF
v3.5a – Jun 13
none
GFDL vortex tracker
v3.5a – Jun 13
none
GSI
v3.2 – Jul 13
none
MET
v4.1 - Apr 13
v4.2 - TBD
Annual
25
AOP 2013 Code Releases and Tutorials
Just beyond 100%
Software
Code releases
Onsite Tutorials
WRF
v3.5 – Apr 13
v3.5.1 – Aug 13
none
UPP
v2.1 – Apr 13
none
HWRF
v3.5a – Jun 13
Annual
GFDL vortex tracker
v3.5a – Jun 13
Annual
GSI
v3.2 – Jul 13
Annual
MET
v4.1 - Apr 13
v4.2 - TBD
Annual
26
Software System for AOP 2013 – 85%
ID
Activity Description
MM1
WRF/UPP repository maintenance, public release & user support (includes only
ARW portion)
MM2
NMME user support
MM5+ NEMS - implement code management plan, repository maintenance, enhance
EN2 portability and physics options
HU1
HWRF repository maintenance, public release and user support
HU2
HWRF physics interoperability
HU3
HWRF scripting maintenance
DA1
GSI code management & repository maintenance, public release & user support
DA4
GSI-hybrid coordination, code management and repository maintenance
DA7
Community-base GSI observations pre-processing capability
VX1
Community support - MET release (including MET-TC), user support & annual
tutorial
27
Discussion Items for Software Systems
 Maintenance of SREF code repository does not currently fall
within the 100% scenario (only NEMS portion of work
currently part of plan) - what does this mean for the future of
this repository?
 Community support – are we ready to take a large step back
from conducting tutorials for our publically-released software
packages?
 What type of planning should DTC be doing wrt potential
upcoming transitions of HWRF to NEMS?
28
Community Outreach Events
Important mechanism for bringing together research and operations
to discuss how to work together to advance NWP
29
Outreach efforts – AOP 2012
 DTC-sponsored events
 Mesoscale Modeling
 Annual WRF Users Workshop (25-29 Jun 2012)
 Ensembles
 Mini-workshop w/ GIFS-TIGGE working group (June 2012)
 DTC & NUOPC Ensemble Design Workshop (10-12 Sept 2012)
 Verification – invited presentations
 UnidataTriennial Users Workshop (3; July 2012)
 EarthcubeWorkshop (Dec 2012)
 Full-day tutorial for Turkish Met Service
30
DTC & NUOPC Ensemble Design
Workshop
BAMS paper by Scott Sandgathe, Brian Etherton,
Barb Brown & Ed Tollerud
Focus: Quantification & characterization
of uncertainty
Main conclusion: more scientific
approach is needed to answer ensemble
design questions
Plan for the future:
1.Establish standard set of metrics that
will allow for useful inter-comparison
of ensemble formulations & dealing
w/ uncertainty
2.Establish a small set of target
parameters
3.Establish a global ensemble data
archive for research (e.g. Ensemble
ICs and perturbations, forecasts from
major centers etc.)
4.Establish clean experimental program
31
DTC-Sponsored Events – AOP 2013
85% Scenario:
 Annual WRF Users Workshop
Proposed but did not make 100%
 Verification workshop
 GSI Workshop
Given the DTC’s mission to serve as a bridge between the research and
operational NWP communities, are we stepping back too much from
sponsoring workshops?
32
Mesoscale Modeling
Jamie Wolff
Collaborators:
NOAA’s Environmental Modeling Center
NOAA’s Earth System Research Laboratory
NCAR’s Mesoscale and Microscale Meteorology Division
North Carolina State University
División de Energías Renovables, CIEMAT, Madrid, Spain
University of Washington
33
Mesoscale Modeling
AOP 2012 Activities
Activity Description
WRF-based community code maintenance and support:
Repository maintenance, email support, code releases, tutorial
Status
Ongoing
Physics interoperability for WRF-based system
In progress
Enhancement of NEMS-based code management:
Technical discussions, friendly user release, FSOE for internal T&E
In progress
Establish a Mesoscale Model Evaluation Testbed (MMET)*:
Define process for R2O transition, provide datasets and baseline results for
cases of interest
Complete
Continue to conduct extensive T&E through comprehensive research
innovation inter-comparisons and Reference Configuration designation:
AFWA: WRF version difference and LIS input data set impact*
NOAA: Surface drag parameterization schemes impact on a High Resolution
Window WRF-ARW baseline configuration
AFWA –
Complete
NOAA –
In progress
34
Key Accomplishments
Inter-comparison Testing and Evaluation
MMET
35
WRF Testing and Evaluation (T&E)
 End-to-end system: WPS, WRFDA, WRF, UPP, and MET
 Test Period: 1 July 2011 – 29 June 2012
 Retrospective forecasts: 48-h warm start forecasts initialized every 36
h w/ DA
 Domain: 15-km CONUS grid
 Evaluation:
 Surface and Upper Air ((BC)RMSE, bias)
 Temperature, Dew Point Temperature, Winds
 Precipitation (GSS, frequency bias)
 3-h and 24-h accumulations
 GO Index
 Statistical Significance Assessment
 Compute confidence intervals (CI) at the 99% level
 Apply pair-wise difference methodology
 Compute statistical significance (SS) and practical significance (PS)
36
WRF Inter-comparison T&E
 Functionally similar operational environment testing
 WRF Data Assimilation and 6-hr warm start
Current AFWA Op Configuration
Microphysics
WRF Single-Moment 5 scheme
Radiation SW and LW
Dudhia/RRTM schemes
Surface Layer
Monin-Obukhov similarity theory
Land-Surface Model
Noah
Planetary Boundary Layer
Yonsei University scheme
Convection
Kain-Fritsch scheme
 WRFDAv3.3.1 + WRFv3.3.1 w/ LoBCs from LIS w/ Noahv2.7.1
 WRFDAv3.4 + WRFv3.4 w/ LoBCs from LIS w/ Noahv2.7.1
 WRFDAv3.4 + WRFv3.4 w/ LoBCs from LIS w/ Noahv3.3
 Evaluation included:
 Impact assessment of WRF system version
 Performance assessment of the LIS input data set
37
Background Error Files
Season
Dates of cold start runs
Summer
20110723 – 20110804
Fall
20111016 – 20111030
Winter
20120123 – 20120207
Spring
20120401 – 20120415
 Used gen_be to produce seasonal background error covariance
files
 Cold start cases initialized at 00 and 12 UTC daily for ~15 days
during each season (GFS only - no SST or LIS)
 Pseudo single observation test
2012072118
Summer:2012011906
Winter:
 Resulting analysis increment from a single observation of the v-
component of the wind with 1 m/s innovation
WRF v3.3.1 – v3.4 Results
 SS (light shading) and PS (dark shading) pair-wise differences for the annual
aggregation of surface temp, dew point and wind BCRMSE and bias aggregated over the
full set of cases and the entire integration domain
39
Regional Temperature Bias Verification
WRF v3.3.1
v3.4 w/
w/Noah
Noahv2.7.1
v2.7.1
00 UTC 12h forecast
00 UTC 24h forecast
40
GO Index
Version Difference
41
Key Accomplishments
Inter-comparison Testing and Evaluation
MMET
42
Testing Protocol Motivation
 Wide range of NWP science innovations under development
in the research community
 Testing protocol imperative to advance new innovations
through the research to operations (R2O) process efficiently
and effectively.
 Three stage process:
1) Proving ground for research
community
2) Comprehensive T&E
performed by the DTC
3) Pre-implementation testing
at Operational Centers
43
Mesoscale Model Evaluation Testbed
(MMET)
 What: Mechanism to assist research
community with initial stage of testing
to efficiently demonstrate the merits
of a new development
 Provide model input and
observational datasets to utilize for
testing
 Establish and publicize baseline
results for select operational
models
 Provide a common framework for
testing; allow for direct
comparisons
 Where: Hosted by the DTC; served
through Repository for Archiving,
Managing and Accessing Diverse
DAta (RAMADDA)
www.dtcenter.org/eval/mmet
44
MMET Cases
 Initial solicitation of cases from DTC Science Advisory Board Members and Physics
Workshop Participants – great response and enthusiasm towards endeavor
 Cases current available within MMET
 20090228 – Mid-Atlantic snow storm where North American Mesoscale (NAM) model produced high
QPF shifted too far north
 20090311 – High dew point predictions by NAM over the upper Midwest and in areas of snow
 20091007 –High-Resolution Window (HIRESW) runs underperformed compared to coarser NAM
model
 20091217 – “Snowapocalypse ‘09”: NAM produced high QPF over mid-Atlantic, lack of
cessation of precipitation associated with decreasing cloud top over eastern North Carolina
 20100428-0504 – Historic Tennessee flooding associated with an atmospheric river
event
 20110404 – Record breaking severe report day
 20110518-26 – Extended period of severe weather outbreak covering much of the mid-west and into
the eastern states later in the period
 20111128 – Cutoff low over SW US; NAM had difficulties throughout the winter of breaking down
cutoff lows and progressing them eastward
 20120203-05 – Snow storm over Colorado, Nebraska, etc.; NAM predicted too little precipitation in
the warm sector and too much snow north of front (persistent bias)
45
User Case #1 (Jimenez and Dudhia)
20100428-20100504 – Extended case focused on historic Tennessee
flooding event
Forecasts: WRF v3.4 ARW baseline configuration namelist from DTC
WRF v3.4 ARW namelist with topo_wind=1 activated
CONUS domain at 15km resolution
Utilized IC and BC files provided by DTC for model initialization
Utilized observation files provided by DTC for verification
46
User Case #1 (Jimenez and Dudhia)
Wind Speed Time Series
47
User Case #1 (Jimenez and Dudhia)
Wind Speed Error (topo_wind=1)
00 UTC 20100428 through
00 UTC 20100504
(every 3 hours)
Underforecast
Overforecast
Average wind speed
across the domain
• topo_wind=1
• Observed
48
User Case #1 (Jimenez and Dudhia)
Wind Speed 6-day Average Error
Status of testing:
• Overall 6-day domain average with
topo_wind=1 smaller than default
• Reduces diurnal mean bias but does not
capture full diurnal amplitude
• Looking into reduction of convective mixing
and vertical transport of momentum causing
overall lower speeds
Default
topo_wind=1
49
Proposed Activities for 2013
50
Mesoscale Modeling
AOP 2013 Proposed Activities
ID
Activity Description
MM1 WRF/UPP community code maintenance and support
MM2
MM5
MM6
MM7
WRF-NMM support
Enhancement of NEMS-based code management
Continue to conduct extensive T&E through comprehensive
research innovation inter-comparisons and Reference Configuration
designation
MM8 Continue implementation and maintenance of MMET
HMT Add 2 Atmospheric River cases from HMT to MMET
51
AOP 2013 Proposed Activities
Testing and Evaluation
 Extensive inter-comparison T&E and RC designations
 MM6: AFWA - Two WRF-ARW configurations
 MM7: NOAA - Two NEMS-NMMB configurations
 NAM physics suite vs. Thompson mp
 Additional Western US verification focus for HMT
 MM8: Expansion of Mesoscale Model Evaluation Testbed (MMET)
 Establish NMMB baselines for all cases (existing and new)
 Maintain infrastructure
 Update baselines with new versions
 Work with community to ensure utilization
 Add additional cases
 4 more cases from EMC priority list
 2 HMT Atmospheric River cases
52
Hurricane
Ligia R. Bernardet
External collaborators:
NOAA Environmental Modeling Center
NOAA Geophysical Fluid Dynamics Laboratory
NOAA Atlantic Oceanographic and Meteorological Laboratory
NCAR Mesoscale and Microscale Meteorology Division
University of Rhode Island
University of California – Los Angeles
Florida State University
53
Hurricane AOP 2012 Activities
Activity Description
Status
HWRF repository maintenance, public release and
user support
Ongoing
HWRF interoperability – Thompson microphysics
In progress
HWRF FSOE to match 2012 operational
Competed
HWRF 2012 operational Reference Configuration
Completed
T&E FSOE: HWRF cumulus sensitivity
Completed
T&E FSOE: HWRF atmos-ocean fluxes
Completed
Sensitivity experiments: Thompson microphysics in
HWRF
Current– will complete in Feb
Diagnostics of large scale environment in HWRF
Completed
54
POM Flux Test
55
Background
HRD (Uhlhorn and Cione) compared HWRF retro forecasts for 2011 against
buoys and showed that HWRF ocean does not respond (=does not cool as
much as obs) when storm goes by
• Fluxes from HWRF atmosphere to ocean are truncated in POM (75%)
• DTC ran 2012 season: control HD12 (75% fluxes) and modified HDFL (100%)
56
Atlantic track and intensity
Track ME: HD12 and HDFL very similar
Int MAE: HDFL SS better at 3 lead times
Int bias: HD12 lowers intensity and helps
overintensification at long lead times
Hurricane Leslie (12L) is the storm with
largest impact (large and slow)
Pacific impact is much smaller (POM 1D)
57
Leslie bias and 09/04 00Z case
• HD12 and HDFL tracks are similar
• HDFL reduces intensity (as expected).
• Is it because of low SST under storm?
58
Leslie bias and 09/04 00Z case
48-h SST control – flux exp
At 48 h, control has cooler
SST than flux exp (contrary
to linear interpretation)
More
mixing
More
intensity
More SST
cooling
Less SST
cooling
Less
intensity
X
Less mixing
X = storm center
59
Cumulus sensitivity test
60
Test of HWRF sensitivity to cumulus schemes
Tested HWRF SAS,
new SAS, Tiedtke,
Kain-Fritsh
Track
12
24
36
48
60
72
84
96
108
120
12
24
36
48
60
72
84
96
108
120
HNSA
HKF1
HTDK
Intens
HNSA
HKF1
HTDK
HWRF SAS performs
best for track;
differences in
intensity have little
statistical significance
Statistical Significance
95%
Green= HWRF SAS better
Red = HPHY SAS worse
61
Case study: Katia init 09/02/11 18 Z
Tracks: similar
Intensity: different (HPHY, HTDK intensify)
78-h forecast isotachs (E-W x-section)
HPHY
SHIPS diagnostics
of shear: initially
similar, later
different. Intensifiers
have lower shear.
Highlights cumulus
effects on and
control on
intensification
HKF1
HNSA
HTDK
62
Large scale diagnostics
63
Background
 Motivation
 EMC is preparing to implement basinscale HWRF in ‘14/15
 Extensive collective work in data assimilation, moving nests, trans-
Atlantic POM
 Need to identify large scale errors – Vx of HWRF 3D fields never
done before
Example of basinscale domain
 DTC diagnostic study
 Evaluated cold-started basinscale HWRF
large scale fields
 Identified issues that deserve further
investigation (hypotheses)
 Created benchmark
64
Methodology
BHWRF
forecast
fields
~730 possible forecast cases from
2011060318 to 2011112506
570 forecast cases
Cold-started from GFS analysis
Run by EMC
Compute
paired
differences
Accumulate differences by
forecast lead time
GFS
analysis
fields
615 forecast cases
PRE13HI
surface pressure
skin temperature
3D temp
3D u and v
3D rel. hum.
3D sp. hum.
3D geopotential
65
Highlight: 600-hPa zonal wind speed
Basinscale bias
September 2011 – 72-h forecast
African jet too weak in HWRF
GFS Bias
September 2011 – 72-h forecast
In GFS jet displaced to south
66
Highlight: surface temperature
Basinscale bias
June 2011 – 24-h forecast
HWRF cold over dry continental areas
Suggests issue with inland ice
GFS Bias
June 2011 – 24-h forecast
No significant biases
67
Thompson microphysics
68
DTC-EMC collaboration in MP
 Interoperability
 EMC (S. Trahan) has created the basic interoperability
 Ability to advect various microphysics mixing ratios and number concentrations
(Ferrier only advects one species)
 New nest-parent interpolation routines which communicate all microphysics
variables (for Ferrier or other microphysics)
 DTC improving MP-radiation interface
 Testing by DTC
 Irene and Earl, with stationary and moving nests
 Winter storm with single domain and stationary nest
 Debugging
 Tests, diagnostics, code analyses uncovered bugs in nest-parent interpolation
 EMC corrected; work in progress
69
HWRF w/Thompson MP (winter storm)
Most recent problem
solved: snow coming from
grid1 into grid2 has a sharp
discontinuity (also cloud ice
number concentration).
Caused by an array
dimensioned incorrectly
70
Radiation code issues: DTC work
 The sum of ice and snow mass is passed from MP to radiation
 Their radius is assumed to be small at cold temperatures
 Effectively, snow is counted as small particles, with massive (and
incorrect) impact on shortwave radiation reflection
 Solution: compute effective radii of cloud ice, snow, cloud
droplets in manner consistent with microphysics scheme – for
Thompson, Ferrier etc.
 Implemented in WRF-ARW in RRTMG (RRTMG being tested
by EMC for 2013 HWRF)
 Will transfer to HWRF *and* NMM-B
71
Hurricane AOP 2013 Activities
ID
HU1
Activity Description
HWRF repository maintenance, public release and support
HU2
HU3
HU4
HWRF interoperability
HWRF scripts maintenance
HWRF Tutorial (currently beyond 100%)
HU5
HU7
HU6
T&E: HWRF FSOE – innovation testing
T&E: Diagnostics and sensitivity experiments for HWRF
T&E: HWRF FSOE – innovation testing (currently beyond
100%)
72
T&E for AOP 2013
Is it cost effective to maintain a HWRF FSOE to do 1 T&E this year??
Recommend expanding to at least 2 T&E activities
73
Diag & sensitivity exper - possibilities
 Work with the research and operational communities to determine priorities
 Follow up on 2012 large-scale diagnostic work (hypothesis were formulated and
require testing)
 Noah LSM and initialization of ice-covered land
 Could improve surface temperature inconsistencies and African jet placement
 Prepare for storm surge and flooding coupling
 Repeat basinscale evaluation with newer datasets (GSI-Hybrid)
 Expand diagnostics to precipitation (CMORPH and Stage IV near/over land)
 Feature-based analysis of subtropical high and upper level high/lows
 Switch tool to MET, which now provides spatial verification
 Statistics conditional on SHIPS: how does model perform when shear is high/weak;
when moisture is high/low etc
 SAB suggestions
 Intercomparison with other HFIP models
 Acquire/organize datasets for case studies
74
T&E- possibilities
 Work with the research and operational communities to
determine priorities
 Noah LSM (beyond diagnostic)
 Could improve surface temperature inconsistencies and African jet
placement
 Laureano thesis indicates improvement for landfalling storms
 Thompson microphysics
 Improvement noted in parallel COAMPS could also benefit HWRF
 Test innovations devised by HFIP grants program, if available
 Da-Lin’s vertical level distribution
 Fovell’s modified physics
75
Data Assimilation Task
Hui Shao
AFWA, NCEP/EMC, NOAA/ESRL, NASA/GMAO,
NCAR/MMM, AOML/HRD, University of Oklahoma
76
Data Assimilation: AOP 2012 Activities
Activity Description
Software Systems: GSI code management &
repository maintenance, public release & user
support
Software Systems: GSI tutorial
Status
Ongoing
T&E: GSI baseline tests for AFWA
T&E: GSI-hybrid for HWRF
T&E: NCAR DART EnKF System (2011 leftover)
T&E: Impact of Radio Occultation Data on HWRF
Forecasts
Ongoing
Ongoing
Completed
Ongoing
Completed
77
GSI Baseline Tests for AFWA
Motivation: Assist AFWA with determining appropriate initial
configuration of GSI for operational implementation (proper
set-up and definition of background error covariance.
Black-box Run of GSI in SLP forecasts
Between 5/14/2012 And 7/25/2012
3.0
Sea Level Pres. (mb)
2.5
T51A (GSI) RMSE
T91A (WRFDA) RMSE
T51A (GSI) BIAS
T91A (WRFDA) BIAS
2.0
1.5
1.0
0.5
0.0
0
-0.5
6
12
18
24
30
FORECAST HOURS
36
42
48
78
Mechanism for AFWA-DTC
Communications
• Benchmark
• Oper config
• Benchmark
• Parallel run
config
• Archived data
/background
for retro runs
AFWF real-time parallel GSI runs:
Updates/changes are periodically brought into
parallel runs. Focus on evaluating the overall
performance of GSI.
DTC real-time & retrospective GSI runs using
functionally-similar operational environment:
Focus on testing incremental changes.
• Real-time: “sync” testbed, uncover the issues
• Short-term retrospective: test individual
changes, tackle the issues
• Extensive retrospective: impact study w/
SS, test research/developmental components
• DA system
switch
• Oper config
(updated)
• Benchmark
• Developmental
config
(suggested
from the DTC)
Pathway to “O”
AFWF real-time operational WRFDA runs.
79
Functionally Similarity Check
Only differences are input fields (background
and observations) and individual changes to be
tested.
AFWA Northern Hemisphere Domain
14
AFWA(UKMET)-Bias
DTC(GFS)-Bias
AFWA(UKMET)-RMSE
DTC(GFS)-RMSE
15
10
Bias/RMSE (Variable Unit)
Bias/RMSE (Variable Unit)
20
5
0
0
-5
2
4
6
8
10
DTC BIAS
10
AFWA RMSE
8
DTC RMSE
6
4
2
0
-2
Variables
AFWA BIAS
12
0
2
1
Variable Name
T
2
3
T-120 T-131
6
8
10
Variables
Background Comparison
Variable No.
4
Analysis Comparison
4
UV
5
UV220
6
UV231
7
q
8
9
Psq-120
180,181,187
80
Real-Time Runs
AFWA GO index:
where Sw is the sum of the skill scores, weighted by lead time,
for wind speed, dew point temperature, temperature, height at
various levels and surface, and mean sea level pressure.
N>1: GSI+ARW better
GSI: GFS BE+GPSRO
GSI: NAM BE
(no GPSRO)
N<1: GFS+ARW better
GSI+ARW runs switched to AFWA parallel run configuration
81
Retrospective Runs: What caused the drop?
N>1:NAM
NAMBEBE/GFS
N>1:
better BE (No GPSRO) better
GFS BE (No GPSRO) better
RAP BE better
N<1:1:GFS
GFSBE+GPSRO
BE+GPSRObetter
better
N<
 NAM BE: Northern Hemisphere BE
computed based on NAM forecasts.
 GFS BE: Global BE computed based on
GFS forecasts.
 RAP BE: Global BE tuned for the RAP.
combination of global/regional (balance =
GFS, Lengthscales/variance = NAM)
NAM BE
GFS BE
RAP BE
NAM BE
GFS BE
RAP BE
Temperature Analysis RMSE
Wind Analysis RMSE
82
Retrospective Runs: Background Errors (BE)
GFS BE
NAM BE
Vertical Lengthscale
Horizontal Lengthscale
Standard Deviation
BE factors for Stream Function
Analysis Inc. from
single T obs test
Messages Passed to the “Operations”
 For Northern Hemisphere, the NAM BE or tuned global BE w/
regional scaling is recommended at current stage.
 The DTC is testing the impact of application specific BE, by
comparing with the NAM BE experiments. Based on the
pending results, application specific BE may be recommended
(resolution may play a role here!).
 For Southern Hemisphere, BE should be examined separately
since the model errors are expected to be larger than those in
Northern Hemisphere.
Action taken: AFWA is going to test the NAM BE in their real-time
parallel runs.
84
GSI-Hybrid Test for HWRF
 Global GSI-hybrid implemented May 2012. Regional system is still under
development.
 DTC member of HFIP tiger team targeting at the 2014 implementation of
GSI-hybrid for HWRF (EMC (team lead), ESRL, AOML, U of O)
 DTC T&E activities focusing on:
 Building up baseline tests for cross comparison with other teams’ work:
 Cross covariance contributed by the ensemble BE under current NCEP
global GSI-hybrid setup
 GSI-hybrid versus GSI benchmark runs
 Testing alternative/reference configurations:
 Partial/full cycling runs versus cold start/warm start runs
 Varying weights of the static BE and ensemble BE
85
GSI-Hybrid Benchmark Tests
NoDA
GSI-3DVAR
GSI-Hybrid
Best Track
• GFS ensemble input: DTC tests show mixed
results on both track & intensity forecasts.
• GSI-hybrid using regional ensembles is under
investigation and will be added to these
benchmark tests.
• Varying weights of static BE and ensemble BE
and their impacts are under investigation.
86
Partial Cycling of GSI-hybrid
 Case study of the 1-d cycling of the GSI-hybrid prior to
the TCvital time shows positive impact on track and
intensity forecasts.
1-d Cycling
 NCEP/EMC and NOAA/ESRL are running warm-start
(6hr HWRF forecast as background) and full cycling
through TC life time. Cross examination will be done
once their initial test results are available.
GFS
1-d Cycling
No cycling
1-d Cycling
Best track
87
Data Assimilation AOP 2013 Activities
ID
Activity Description
DA1
GSI code management & repository maintenance, public release &
user support
DA2
GSI Tutorial (currently beyond 100%)
DA3
GSI Workshop (currently beyond 100%)
DA4
GSI-hybrid coordination, code management and repository
maintenance
DA5
GSI baseline for AFWA
DA6
GSI-hybrid for HWRF
DA7
Community-base GSI observations pre-processing capability
88
Community-base GSI Observations Preprocessing Capability
• GSI observations are required in BUFR format. Conventional data go
through a sophisticated QC procedure prior to being assimilated by
GSI. Satellite data are dumped into a data trunk and their QC and bias
correction procedure are performed inside GSI.
• The data format conversion and the data QC procedure have become
interest of active GSI community users, especially those who would
like to ingest additional data types.
• AFWA little_r data files are in ASCII format and, along its preprocessing tool, are used already by AFWA operations and some
community users.
• NCEP/EMC has worked with AFWA to adopt the NCEP/NCO preprocessing package. However, the code has not reached the
community code standard.
89
Community-base GSI Observations Preprocessing Capability: Wish list
•
•
•
•
•
•
Code portability
User friendly interface
Configurable setup
Modularized code
Documentation
User support
• Flexibility to
ingest new
observation
types
• Independent
format
converter
(multiple input
format)
• Configurable
platformspecific QC
• Different
model
background
for QC
NCEP/AFWA Pre-processing System
90
Plans for Setting Up an Initial Community
Capability
• Review and test the current AFWA/NCEP data collection and processing
procedures in an operational environment.
• Establish a DTC code repository to facilitate version control of the preprocessing capability development.
• Design an acceptable workflow system and user interface that will meet
AFWA’s operational requirements.
• Develop essential scripting and coding to accommodate the desired initial
capability of the pre-processing procedures, such as adding initial
configuration management to the scripts and codes requested by AFWA.
• Develop an automatic and portable configuration and compilation utility so
that the system can be easily ported to different computing environments.
• Begin to document the established community capability.
Resources allocated to this activity reduced from original proposal –
longer time frame required to achieve ultimate goals
91
GSI Baseline Test for AFWA
 DTC will be testing developmental capabilities – focus of tests will be
determined by consultation with AFWA. Potential ideas include:
 ARW BE (domain specific)
 Increased model top
 New NCEP regional bias correction (BC) scheme for radiance data (by
blending global coefficients and ozone information with regional BC)
 Moisture channel radiance QC and impacts
 Radiance channel selection
 GPSRO assimilation
 Only limited configurations can be tested. Running short-term tests
(<monthly) will allow more capabilities to be tested but with less
confidence in the results.
92
GSI-hybrid Tests for HWRF
 Aspect of GSI-hybrid to be tested will be determined and coordinated
through consultation with EMC and appropriate developers (the
“tiger” team). Potential focus:
 Input from regional ensembles
 Optimal weighting for static and ensemble BE
 Full cycling of GSI-hybrid
 Moving domain for GSI-hybrid
 Only limited configurations can be tested (case studies but testing
more configurations, longer-period of tests but limited testing
components).
93