Applying Fuzzy Clustering Analysis to Assess
Uncertainty and Ensemble System
Performance to Cool Season High-Impact
Weather
BRIAN A COLLE
MINGHUA ZHENG │ EDMUND K. CHANG
NCEP
CMC
ECMWF
MEAN
ANA
2015 Northeast
Winter Storm
+3 Day forecast
Verifying at 0000 UTC 25 Jan 2015
Motivation
Should forecasters utilize ensembles beyond ensemble mean/spread,
probs, and anomalies?
Some Key issues:
1. Forecasters need ensemble tools to condense useful information from
ensembles to understand predictability issues in the forecast.
2. Forecasters need more guidance about potential storm scenarios,
biases, and outliers, especially for high impact weather events
3. Important, since ensembles can fail for major weather events, and we
need to quantify more when and how this happens.
Tools and analysis:
 Ensemble sensitivity analysis (see Zheng et al. 2014).
 Apply an efficient ensemble tool (fuzzy clustering analysis) to quickly
separate forecast scenarios among the large ensemble set.
 Evaluate different ensemble models’ performance in forecasting
winter storms over East Coast for medium-range forecast and find
out on average which model is more reliable when expecting a severe
storm.
ECMWF (50-member) mean (shaded in mm) and
spread (contoured) for the day-7 24-h Precipitation
Initialized 1200 UTC 16 December 2013
EOF1 Pattern for the day 7 Precipitation and
Analyzed Precipitation (in mm)
(See Zheng et al. 2014 for Ensemble Sensi details)
Ensemble Sensitivity Based on EOF1 Pattern
Forecasters likely want to know the various forecast
scenarios: Our attempt using Fuzzy Clustering
 Data:
-

-
TIGGE Ensemble forecast data: NCEP (20 mem) + CMC (20 mem) +
ECMWF (50 mem)
Analysis: NCEP operational analysis
Variables: MSLP and Z500
Historical cases selections: 124 (114 for region 2 – East coast)
cyclone cases (minimum pressure <996 hPa) from 2007 to 2014 cool
seasons (NDJFM) using Hodges cyclone tracker.
Method:
Empirical Orthogonal Function (EOF) analysis
To quantify ensemble forecast variance patterns
Fuzzy clustering analysis
To group ensemble members based on EOF PCs (Harr et al.
2008)
Ensemble Means (Green = CMC, Red = NCEP, and
Blue = EC: Valid at 1200 UTC 27 UTC January 2015
CPC 24-h Precipitation and day 3 ensemble mean precipitation for
forecasts initialized at 1200 UTC 24 January 2015.
Case study: 3-Day forecast, IT: 1200 UTC 24 Jan
2015 UTC 00; VT: 1200 UTC 27 Jan 2015
MSLP spaghetti plot for 996 hPa at Jan
27h 2015 12Z
Ensemble mean (contours) and spread
(shades) of MSLP , [hPa]
STEP1: EOF analysis of MSLP on 90 members of
forecasts at valid time (VT)
EOF1 MSLP anomaly pattern, [hPa]
EOF2 MSLP anomaly pattern, [hPa]
57.1%
23.8%
STEP2: group ensemble members into 5 clusters
based on PCs using Fuzzy clustering scatter plots
+PC1: Deeper
+slight N
-PC1: Weaker
+slight S
+PC2: West
+slight deeper
-PC2: East
+slight weaker
Group 2
Weaker+NE
Group 3/ANA
Shift to NE
Group 4
Weaker+SW
Group 5
Deeper+SW
STEP3: Pick up a contour line and plot group mean
summary based on the partitions of clusters
EM
NE/Weaker(2)
NE Shift (3)
SW/Weaker(4)
SW/Deeper(5)
Analysis
STEP4: Look at ensemble sensitivity to know where
upstream to look for any important differences
between cluster solutions
STEP5: Look at the evolution of the clusters upstream
and how they compare to analysis as they become
available…
EM
NE/Weaker(2)
NE Shift (3)
SW/Weaker(4)
SW/Deeper(5)
Analysis
Day 1 Forecast
Day 1.5 Forecast
Historical evaluations using 124 (114) cyclone cases

Region 1
Day 3 forecast
NCEP/CMC/EC: 7/4/9
Region 2
124 cyclone cases for region 1
114 cyclone cases for region 2
8 out-of-envelope or outlier
cases are not included.
Day 6 forecast
Percentage of each model ensemble’s members w.r.t. its total
ensemble member falling into the Group ANA for region 2
Day 3 forecast
Day 9 forecast
Percentage of each ensemble's members in Group ANA
RMSE and Spread relation (RMSE/SPRD) for
PC1 and PC2 for regions 1 and 2
1 is the best
Under-dispersed
Over-dispersed
Under-dispersed
Over-dispersed
Percentage of cases each model misses Group ANA
The ratio of cases with Group ANA the same as Group EM to
the average cases
ANA more likely than average
to be in Group EM
ANA less likely than average
to be in Group EM
Conclusions
 Ensembles have difficulties with major cyclone events. We still need forecasters






in the loop, but they need more tools.
Case study of “2015 Northeast Winter Storm” shows that fuzzy clustering can
quickly separate different forecast scenarios.
For day 6 (9) forecasts, in about 7% (13%) of the cases, the analysis falls outside
the envelope of the multi-model ensemble.
ECMWF members have the highest probability to be included in the analysis
group for 3 and 6 days forecast.
The analysis is not more likely to be in the Ensemble mean group than other
groups (except for day 3 forecast), suggesting that focusing on the ensemble
mean may be misleading in many cases
The RMSE/SPREAD relations show that ECMWF ensemble is less underdispersed than NCEP and CMC for day 3 and day 6. For day 9, all individual
models are severely under-dispersed.
The multi-model ensemble is less under-dispersed than any individual models
for day 6 and day 9.
 This research is supported by NOAA CSTAR program.
 Fuzzy clustering website:
http://dendrite.somas.stonybrook.edu/CSTAR/Ensemble_Sensitivity/FC_Main.html