The Evaluation of National Fire Danger Rating (NFDRS) Zone Trend Forecasts
Produced by the National Weather Service in 2002
An evaluation of National Fire Danger Rating (NFDRS) zone trend forecasts produced by
six National Weather Service (NWS) were evaluated at the conclusion of the 2002 fire
season by the Predictive Services Section at the Northwest Interagency Coordination
Center. This was the third year in which a detailed assessment was performed in
accordance with Section V paragraph five of the Memorandum of Understanding
between the Pacific Northwest Wildfire Coordinating Group (PNWCG) and the NWS.
The goal is to improve fire weather services in the Pacific Northwest through monitoring
and evaluation.
This was done by comparing observed and forecast weather data archived in the Weather
Information Management System (WIMS). NWS forecasts were compared against the
persistence error for a particular weather station. The persistence error refers to the
numeric difference (or the change) from one day to the next for a particular weather
element at approximately 1400 LDT. For example, if the observed temperature was 75
degrees F one day and 78 degrees F the next, the persistence error is 3 degrees F. The
elements verified were temperature, relative humidity, and wind speed. Changes in wind
speed and relative humidity have a much greater impact on NFDRS than temperature.
Since persistence forecasts can be determined without any meteorological input, NWS
skill should result in forecast accuracy that consistently exceeds the persistence error. A
total of 100 RAWS stations were evaluated between June 1 and September 30: 23 in the
Medford fire weather district, 22 in Portland, 20 in Pendleton, 17 in Spokane, 12 in
Seattle and 6 in the Boise district.
A spreadsheet was constructed for each RAWS to calculate daily persistence and forecast
errors for temperature, relative humidity and wind speed. Once the persistence and
forecaster errors were calculated for each day, the average error for each was calculated
over the entire fire season. These values were then input into the following formula to
calculate forecaster improvement over persistence.
Forecaster improvement = (persistence error – forecaster error)/persistence error * 100
A negative value indicates a forecast worse than persistence. The higher a positive value,
the better the forecast in relation to persistence. Perfect forecasts would result in a 100
percent forecaster improvement over persistence.
These figures were then entered into a summary spreadsheet that divided the 100 RAWS
into NWS fire weather districts and fire weather zones. Additional information, such as
the beginning and ending dates of NFDRS forecasts and the number of forecasts
generated during the four-month period were also included.
The table below summarizes the improvement over persistence of NWS forecasts during
the 2002, 2001 and 2000 fire seasons. In general, NWS forecasts have improved
slightly, but are still below the accuracy standards established in the Memorandum
of Understanding between the NWS and the Pacific Northwest firefighting agencies.
The negative red figures in the wind column indicate forecast skill less than that of
persistence.
Office
Seattle
2002
2001
2000
Spokane
2002
2001
2000
Portland
2002
2001
2000
Pendleton
2002
2001
2000
Medford
2002
2001
2000
Boise
2002
2001
2000
MOU Fcst
Standards
Improvement over Persistence
Temperature
Relative Hum.
Wind
25.89
23.24
22.42
20.59
21.17
19.10
5.23
-11.18
-18.12
25.56
20.69
19.43
14.52
17.89
8.64
-6.86
-13.94
-17.59
27.36
26.71
29.54
22.45
18.53
18.72
-7.94
-11.09
-11.69
35.12
31.26
28.76
22.45
20.62
19.06
-1.49
-6.94
-15.88
31.75
28.80
25.75
18.15
14.92
14.54
-11.83
-2.53
-6.29
29.28
32.25
N/A
20.51
23.54
N/A
7.31
0.37
N/A
35.00
25.00
10.00