Title: Quantitative Precipitation Forecast (QPF) Verification, Part I
Section 6
Total # of Pages:
Development URL: http://deved.meted.ucar.edu/Hydro/verification/QPFverif-I
Section Narration Duration: XXminXXsec
Special Development Notes:
Project Scientist: Matt Kelsch
Instructional Designer: Lon Goldstein
SMEs: Julie Demargne, Tish Soulliard, Dave Myrick, Dave Novak,
Keith Brill, Ken Pomeroy
Bullets/Onscreen text: Blue
Annotation #s and text in right column: light green
Middle column label/graphics-not onscreen: purple
Matt comments highlight-Blue
Lon comments/highlight-gray
Programmer notes-green with gray highlight
Table of Contents
6.0
Summary
6.1
Using the Tools
6.2
Verification Questions
6.3
Resources
Summary
Using the Tools
Verification Questions
Resources
6.0 Summary
QPFs are difficult to verify
We will re-use graphics from other section in this section.
because of the different types
of QPF and the
characteristics of both the
forecasts and the verifying
observations. As a result,
good verification typically
requires a well thought-out
methodology and an
examination of more than one
1
d:\106741942.doc
Documen
Section 6
Estimation of Observed Precipitation
type of verification score.
Verification is done for three
primary reasons: 1)
evaluating the forecasts, 2)
comparisons between
forecast systems, and 3)
identifying and correcting
forecast errors thus resulting
in forecast improvement.
With all of the QPF
verification tools out there, it
was only possible to discuss
some of the more common
ones used widely by NOAA’s
NWS. Those are the
Hydrometeorological
Predication Center (HPC)
QPF verification, the National
Precipitation Verification Unit
(NPVU), Boise Verify
(BOIVerify) and other
Western Region efforts,
Interactive Verification
Program (IVP) and the
Ensemble Verification
System (EVS).
Currently, and certainly in the
future, these are not the only
verification choices. However,
we hope that these provide
enough different types of
verification approaches to
demonstrate how to perform
a good verification study.
Once you know the best
approaches to verification, it
should not be too difficult
adapting to new tools.
6.1 Using the Tools
The HPC verification tool is
typically used for the big
picture look at the data often
used as a QPF starting point.
The website offers a look at
NWP model QPF as well as
2
Will re-use selected graphics from sections 3-5 here.
d:\106741942.doc
Section 6
Estimation of Observed Precipitation
the HPC QPF itself. These
are presented for several
quantities and can be viewed
graphically for certain time
periods.
NPVU provides the tools to
evaluate local office QPF
performance for gridded
forecasts. Combined with the
HPC information, a user can
get a sense of how much
value is added to the QPF by
local office operations.
NWS Western Region tools
like BOIVerify are also useful
for evaluating QPF on the
local scale. But BOIVerify
offers more sophisticated
verification measures. It also
provides the ability to cater
verification scores to
specified regions and time
periods better than the other
tools described.
NWS AWIPS verification
tools like IVP and EVS will
continue to evolve so we did
not detail them too much in
this module. With these
tools, important feedback
from verification is integrated
directly into the forecast
process.
6.2 Verification Questions
It is important to know what
questions you are trying to
answer so that you can set
up an effective methodology.
For example, “how good are
my QPFs,” may be too
vague. What you really may
want to ask are questions like
these:
How much error is there in
my QPF compared to the
3
d:\106741942.doc
Section 6
Estimation of Observed Precipitation
analyses? Do my QPFs
show skill over NWP models
in the short range, and
climatology in the long
range? At what time horizon
do the QPF errors become
unacceptably large? Are
there events or seasons
when certain types of error or
bias are more evident? How
is our office’s forecast
reliability for typical events
compared to high-impact big
events?
These are the types of
questions that can help guide
a forecaster to make useful
evaluations and subsequent
improvements to QPF.
6.3 Resources
As you refine your QPF
verification strategy, we hope
the guidelines in this module
will be useful. Important new
and more complex measures
of forecast quality will
become available through
some of the programs
discussed here, as well as in
other programs.
As mentioned in section 1,
this is one of several COMET
training modules dealing with
verification of either QPF or
streamflow forecasts. The
first, Introduction to
Verification of Hydrologic
Forecasts, provides a through
overview about common
verification measures in
hydrology, both simple and
complex. The next verification
module titled Techniques in
Hydrologic Forecast
Verification looks at
4
d:\106741942.doc
Section 6
Estimation of Observed Precipitation
verification methodology
using the AWIPS tools IVP
and EVS and is scheduled to
publish in summer 2010. QPF
Verification 2 will cover some
more advanced measures
that deal with problems such
as timing issues.
5
d:\106741942.doc