SUMMARY OF THE FOURTH
CLOUD RETRIEVAL EVALUATION
WORKSHOP
Rob Roebeling, Bryan Baum, R alf Bennartz, Ulrich Hamann, Andrew Heidinger,
Jan Fokke Meirink, Martin Stengel, Anke Thoss, Andi Walther, and Phil Watts
by
T
he Cloud Retrieval Evaluation Workshops
(CREWs) provide a forum for international cloud
remote sensing scientists to share their experience with state-of-the-art cloud parameter retrievals
from satellite, airborne, and surface observations. The
overarching objectives of CREW are to enhance our
knowledge on quantitative cloud parameter retrievals
from state-of-the-art algorithms and identify shortcomings that need focused attention as a community.
Continual improvement in the global description
of cloud properties optimizes these algorithms for
near-term (nowcasting), short- to medium-term
(weather forecasting), and long-term (regional and
climatological analyses) applications, as well as for
potential improvements in the cloud and convection
AFFILIATIONS: Roebeling
and Watts —EUMETSAT, Darmstadt,
Germany; Baum, B ennartz, and Walther—Space Science and Engineering Center, University of Wisconsin–Madison, Madison, Wisconsin; Hamann —MeteoSwiss, Locarno, Switzerland; Heidinger—
Center for Satellite Applications and Research, NOAA/NESDIS,
Madison, Wisconsin; Meirink—Royal Netherlands Meteorological
Institute (KNMI), De Bilt, Netherlands; Stengel—Deutscher Wetterdienst, Offenbach, Germany; Thoss —Swedish Meteorological
and Hydrological Institute, Norrköping, Sweden
CORRESPONDING AUTHOR: R. A. Roebeling, EUMETSAT,
Eumetsat Allee 1, D-64295 Darmstadt, Germany
E-mail: rob.roebeling@eumetsat.int
DOI:10.1175/BAMS-D-14-00184.1
In final form 25 September 2014
©2015 American Meteorological Society
AMERICAN METEOROLOGICAL SOCIETY
FOURTH CLOUD RETRIEVAL EVALUATION
WORKSHOP (CREW-4)
What: A joint Asian–European–U.S. workshop gathered
about 70 research scientists to review existing
and new approaches to infer cloud parameters
from passive and active satellite observations. The
priorities of this workshop were to compare and
improve level-2 and level-3 cloud products from
different teams, increase commonality between
these products, and define scientific focal points
and collaborations for the next workshop in 2016.
When: 4–7 March 2014
Where: Grainau, Germany
parameterizations adopted in weather and climate
models.
The Fourth CREW (CREW-4) was the held in
Germany in March 2014. The workshop was attended
by about 70 participants from various universities,
research institutes, and satellite agencies in Asia,
Europe, and the United States. The workshop was
organized along four thematic sessions: “Cloud
parameter retrieval methods,” “Cloud parameter retrieval evaluations,” “Nowcasting and severe
weather applications,” and “Cloud parameter datasets for weather and climate research.” The cloud
parameter retrieval evaluations were facilitated by
a common database that comprises, for a number
of “golden days,” cloud property retrievals from
different algorithms for passive imagers (SEVIRI,
MODIS, AVHRR, POLDER, and AIRS; see Table 1 for
APRIL 2015
| ES71
a complete list of acronyms) and other cloud measurements that serve as a reference (e.g., CALIOP, CPR,
and AMSU-E).
HIGHLIGHTS. A noticeable finding was the
increased number of research groups that now
implement optimal estimation methods in their
operational retrievals. In addition, some research
groups have started to combine observations from
both passive and active instruments. While the
active sensors provide information for only a very
small portion of the imager swath, these observations
are critical for improving global cloud parameter
retrievals.
The preliminary results presented on the assessments of error estimates produced by some of the
retrieval schemes were an important step toward
quantifying these estimates in a more systematic
manner. These assessments reveal that error estimates compare reasonably well in multiple algorithm
ensembles or against the true uncertainty between
retrieved and observed cloud parameters.
The evaluation of aggregation methods and
filtering rules reveal that the manner of aggregating
or filtering level-2 data creates systematic differences in level-3 products that tend to vary regionally
depending on climate regions and/or surface conditions. Although the differences are smaller than those
between level-2 retrievals, they are not negligible.
For the community to be able to truly compare their
gridded level-3 products, a set of common filtering
and aggregation rules need to be identified and integrated by the various teams.
OUTCOME OF CREW- 4. Cloud parameter
retrieval methods continue to advance by improving
physical approaches, adopting new retrieval methods, or exploiting new types of observations. The
session “Cloud parameter retrieval methods” reported on updates made to existing operational
cloud parameter retrievals (e.g.,
MODIS06–006, CM-SAF), as well
Table 1. List of acronyms.
as on newly developed retrieval
Acronym
Meaning
methods (e.g., ESA Cloud_cci), such
as optimal estimation methods and
AERONET
Aerosol Robotic Network
methods that combine observations
AIRS
Atmospheric Infrared Sounder
from passive and active instruments.
AMSU-E
Advanced Microwave Sounding Unit for Earth Observing
The presentations in the session
System
“Cloud parameter retrieval evaluAVHRR
Advanced Very High Resolution Radiometer
ations” covered the latest results
BSRN
Baseline Surface Radiation Network
of intercomparison and validaCALIOP
Cloud–Aerosol Lidar with Orthogonal Polarization
tion assessments of cloud paramCGMS
Coordination Group for Meteorological Satellites
eters. Since CREW-3 in 2012, more
Cloud_cci
Cloud Climate Change Initiative
groups now perform these types
of assessments (Hamann et a l.
CM SAF
Satellite Application Facility on Climate Monitoring
2014; Stengel et al. 2015). Figure 1
CPR
Cloud Profiling Radar
shows an example of the CREW
CREW
Cloud Retrieval Evaluation Workshop
type of assessment made for cloud
DWD
Deutscher Wetterdienst
optical depth retrievals from polarESA
European Space Agency
orbiting satellite observations from
EUMETSAT European Organization for the Exploitation of
the VIIRS instrument. Moreover,
Meteorological Satellites
the validations are performed for
GEWEX
Global Energy and Water Cycle Experiment
different types of cloud conditions,
GRUAN
Global Climate Observing System (GCOS) Reference
giving better insight to the strengths
Upper Air Network
and weaknesses of the algorithms.
GSICS
Global Space-Based Inter-Calibration System
Initial results were presented of an
assessment of the error estimates
ICWG
International Clouds Working Group
produced by some of the retrieval
MODIS
Moderate Resolution Imaging Spectroradiometer
schemes.
POLDER
Polarization and Directionality of the Earth’s Reflectances
The special session “Nowcasting
SEVIRI
Spinning Enhanced Visible and Infrared Imager
and severe weather applications”
TCDR
Thematic Climate Data Record
highlighted how cloud parameter
VIIRS
Visible Infrared Imaging Radiometer Suite
retrievals can be used to predict
ES72 |
APRIL 2015
Fig. 1. Example evaluation of polar satellite retrievals of cloud optical depth, showing the retrieval results from
six different algorithms: that is, Interface Data Processing Segment (IDPS), Goddard Space Flight Center
(GSFC), Clouds from AVHRR Extended Daytime Cloud Microphysical Properties (CLAVR-x/DCOMP), Polar
Platform System Cloud Physical Properties (PPS/CPP), and Langley Research Center (LaRC).
severe storms by making use of the local dynamics
and the temporal resolution of the geostationary data.
The session “Cloud parameter datasets for weather
and climate research” reported on conditions and
requirements that need to be satisfied for the generation of well-characterized cloud parameter data
records and on the use of these so-called thematic
climate data records (TCDRs) in several climate
monitoring and climate model evaluation studies.
More focused discussions on cloud retrieval
principles and the validation of cloud parameters
were held within three clusters of parallel breakout
sessions. The topics of these sessions were i) retrieval
methods, ii) retrieval and uncertainty evaluations,
and iii) nowcasting and climate applications.
CREW endorsed the need for accurate calibration
of all passive imager measurements. We hope that
GSICS will provide such calibration (visible, near
infrared, and infrared) for all passive imagers. The
community decided to form subordinate working
groups to improve the retrievals, for example, of
multilayered clouds, for the treatment of vertical
cloud inhomogeneity, and in polar regions. The
AMERICAN METEOROLOGICAL SOCIETY
participants also discussed forming multi-algorithm
ensembles to assess uncertainties and sensitivities
of, for example, error estimates. There was consensus on the need to streamline intercomparison and
validation activities across research groups. Among
other recommendations, CREW suggests that the
measurement community work toward establishing
a network of climate anchor reference sites that
operate several reference networks simultaneously
(e.g., GRUAN, BSRN, AERONET). The research into
using cloud parameter products in severe weather
applications generated much discussion and was
encouraged through case studies focusing on convection, fog, and severe fires. The breakout session on
climate applications discussed ways to accommodate
a common approach for generating global decadal
gridded (level 3) cloud parameter records and in
particular recommended focusing on the definition
of a set of essential filtering rules for different cloud
parameters and further to investigate the propagation
of error estimates in level-3 products. In support of
the GEWEX Cloud Assessment, CREW will seek to
advance the production of long-term datasets that are
APRIL 2015
| ES73
well characterized in their strengths and weaknesses.
The need was acknowledged for preservation of these
data in formats that are widely accessible and to adopt
self-describing common data formats.
The participants strongly supported the proposal to
establish CREW as an International Clouds Working
Group (ICWG) within the Coordination Group for
Meteorological Satellites (CGMS). At the 42nd plenary
session of CGMS, which was held in China in May
2014, the CGMS-42 plenary endorsed the formation
of an ICWG, making CREW a formal entity. The community is now comfortable enough with the CREW
forum to collaborate well beyond what has occurred to
date and to promote software exchange. It decided to
adopt the format of subordinate working groups that
RECOMMENDATIONS FROM THE MEETING
• Improve cloud models used in retrievals to more
accurately reflect reality, particularly ice crystal models,
vertical inhomogeneity, and multiple layers.
• Explore the potential of combining different types of
observations in level-2 cloud retrievals methods.
• Explore the definition of a set of essential filtering
rules in level-3 aggregation methods for different cloud
parameters.
• Work toward the characterization of uncertainties in
level-2 and level-3 products.
• Explore production of multi-algorithm ensembles to
assess uncertainty/sensitivity.
• Explore the production of long-term datasets aimed at
stability and accurate assessment of product strengths
and weaknesses.
• Use common ancillary data and validation procedures
for level-2 and level-3 data.
• Establish subordinate working groups to make progress
on a variety of outstanding issues: for example,
multilayered clouds, severe weather applications, and
aggregation methods.
ES74 |
APRIL 2015
could make efficient progress on a variety of topics and
report on their findings at the next CREW meeting. In
the final discussions, the priority research themes for
the coming years were defined. In connection to these
themes, a list of potential topics for subordinate working groups and leads was drafted. The next CREW will
be held in Lille, France, in 2016.
MORE INFORMATION. More detailed information on the CREW workshops can be found on the
CREW website (www.icare.univ-lille1.fr/crew/). The
passive imager retrievals and the reference data in the
common database are available via our FTP site and
can be downloaded (www.icare.univ-lille1.fr/register
/register.php). When asked for a “short description of
your project,” please state that you want to have an
account created for the Cloud Retrieval Evaluation
Workshop.
ACKNOWLEDGMENTS. The intercomparison and
evaluation of retrievals schemes, done as preparatory work
to this workshop, was part of the EUMESTAT Fellowship
project cosponsored by EUMETSAT. Financial and organizational contributions for holding this workshop were
made by EUMETSAT, ESA, DWD, and CM SAF.
REFERENCES
Hamann, U., and Coauthors, 2014: Remote sensing of
cloud top pressure/height from SEVIRI: Analysis of
ten current retrieval algorithms. Atmos. Meas. Tech.
Discuss., 7, 401–473, doi:10.5194/amtd-7-401-2014.
Stengel, M., and Coauthors, 2015: The Clouds Climate
Change Initiative: Assessment of state-of-the-art
cloud property retrieval schemes applied to AVHRR
heritage measurements. Remote Sens. Environ.,
doi:10.1016/j.rse.2013.10.035, in press.