Technical Introduction
Jörg Trentmann
Satellite Application Facility on
Climate Monitoring (CM SAF)
Deutscher Wetterdienst
June 2012
Overview
•Introduction to
•CM SAF data
•Software tools (cdo, R)
•Example scripts
•Suggestions/ideas for exercises
June 2012
Introduction
Are you planning to use CM SAF data during the
CM SAF Event Week?
No
Yes
(but will use
CM SAF data later)
No
(no plans to use
CM SAF data)
June 2012
Introduction
Are you a registered user of CM SAF?
Yes
No
Have you ever ordered CM SAF data through the Web
User Interface (WUI: www.cmsaf.eu/wui)?
Yes
June 2012
No
CM SAF data files
How does a CM SAF data file look like, have you ever
seen a CM SAF data file??
Yes
June 2012
No
CM SAF data files
The file names of CM SAF data follow a certain naming
convention:
June 2012
CM SAF data files
The file names of CM SAF data follow a certain naming
convention:
June 2012
CM SAF data files
Example:
June 2012
Data Formats
hdf
(http://www.hdfgroup.org/HDF5/)
+ supports additional data
compression
+ commonly used in
satellite data community
+ provides a standard
– not very common
– only limited software
available to access the
data
– complex to read data
June 2012
Data Formats
hdf
(http://www.hdfgroup.org/HDF5/)
+ supports additional data
compression
+ commonly used in
satellite data community
+ provides a standard
– not very common
– only limited software
available to access the
data
– complex to read data
June 2012
netcdf
(http://www.unidata.ucar.edu/software/netcdf
+ growing number of
meteorological /
climatological users
+ wider range of software
available to access the
data, incl. GIS
+ standard for climate data
available
– no data compression,
larger files
Data Formats
hdf
(http://www.hdfgroup.org/HDF5/)
netcdf
(http://www.unidata.ucar.edu/software/netcdf
It is recommended to order CM SAF
+ supports additional data
+ growing number of
data in netcdf-format
via the Web
compression
meteorological/climatolog
+ commonly used in
User Interfaceical(WUI)!
users
satellite data community
+ wider range of software
+ provides a standard
available to access the
data, incl. GIS
– not very common
+ standard for climate data
– only limited software
available
available to access the
data
– no data compression,
– complex to read data
larger files
June 2012
Software tools
The free, open-source climate data operators (cdo)
and the statistical software tool R are the main tools
supported to analyse CM SAF data.
•cdo: https://code.zmaw.de/projects/cdo
•R: http://www.r-project.org
June 2012
Software tools
Have you ever used the
climate data operators (cdo) / R?
June 2012
cdo
R
Yes
Yes
Very little
Very little
No
No
tools: cdo
Climate Data Operators
• developed at Max Planck Institute for Meteorology, Hamburg
• freely available including extensive documentation:
https://code.zmaw.de/projects/cdo
• works with multiple file formats, including grib, netcdf etc.
• collection of operators for processing climate (model) data e.g.
spatial interpolation
data selection
Subsampling
statistical and arithmetical functions
…
• is accessible from the ‘command line’ (‘prompt’)
• cdo does currently not include a graphical user interface and can
not visualize data
June 2012
tools: R
R
• programming language and software environment for
statistical computing and graphics: http://www.r-project.org/
• includes an interface to netcdf
• has extensive statistical and graphical capabilities, mainly
through additional packages
• supports the use of scripts
• extensive online-documentation available on the official
webpage and on other webpages
• has been developed for use in statistics (biology, medicine,
etc.);
• has not been developed for use in meteorology / climatology:
-> terminology sometimes not intuitive
-> rather slow for large data sets
June 2012
tools: cdo + R
Which operating system are you using?
XP
7
Linux/Unix
Windows
2000
June 2012
Vista
MAC OSX
tools: cdo + R
Is cdo / R available on your computer system?
June 2012
cdo
R
Yes
Yes
No
No
tools: cdo + R
• A collection of software scripts including example CM
SAF data is provided on the Course page
• scripts are based on Windows, rearranging them for
Unix/Linux/Mac-operating systems should be straight
forward.
• Before running these scripts, install cdo and R on your
local computer system (see further instructions on the Course Page)!
• Suggestion: install panoply (visualize netcdf-files) and
Notepad++ (text-editor).
• Have a look at the screencasts (provided on the Course
page) that describe the use of the scripts!
June 2012
Step-by-step
1. Order data in netcdf-format on a regular lon-lat grid from the Web
User Interface
June 2012
Step-by-step
1. Order data in netcdf-format on a regular lon-lat grid from the CM
SAF Web User Interface
2. Combine the individual time steps into one netcdf-file (cdo)
June 2012
Step-by-step
1. Order data in netcdf-format on a regular lon-lat grid from the CM
SAF Web User Interface
2. Combine the individual time steps into one netcdf-file (cdo)
3. Extract the time series for a single location (cdo)
4. Calculate the temporal / spatial means (cdo)
5. Calculate the multi-year monthly averages (cdo)
6. Calculate the monthly anomalies (cdo)
June 2012
Step-by-step
1. Order data in netcdf-format on a regular lon-lat grid from the CM
SAF Web User Interface
2. Combine the individual time steps into one netcdf-file (cdo)
3. Extract the time series for a single location (cdo)
4. Calculate the temporal / spatial means (cdo)
5. Calculate the multi-year monthly averages (cdo)
6. Calculate the monthly anomalies (cdo)
7. Plot time series data (R)
8. Analyse the time series data, e.g., calculate average annual cycles,
trends (R)
9. Visualize the 2D Data, e.g., monthly means / temporal means /
anomalies (R)
June 2012
Step-by-step
Surface Solar Radiation,
Offenbach, Germany
Examples
Time Series Analysis
Regional Plot
June 2012
Now it is your turn!
• Use the scripts and the data provided
• Order more data that fits your purpose using the WUI
• Best way of learning new tools is using them!
June 2012
Now it is your turn!
• Use the scripts and the data provided
• Order more data that fits your purpose using the WUI
• Best way of learning new tools is using them!
• Some ideas to start with:
- Have a look at your home town:
What is the average annual cycle of cloud cover, solar
radiation, humidity?
Is there a trend in solar surface radiation?
Are there temporal/spatial correlations between solar
radiation, cloud cover, humidity in your country?
Which was the ‘best’ summer / winter in the last five years?
June 2012
Now it is your turn!
• Use the scripts and the data provided
• Order more data that fits your purpose using the WUI
• Best way of learning new tools is using them!
• Some ideas to start with:
- Have a look at your home town:
What is the average annual cycle of cloud cover, solar
radiation, humidity?
Is there a trend in solar surface radiation?
Are there temporal/spatial correlations between solar
radiation, cloud cover, humidity in your country?
Which was the ‘best’ summer / winter in the last five years?
- Which region in your country has the highest cloud cover (e.g.,
in July)? Is this the region with the highest solar radiation?
June 2012
Now it is your turn!
• Use the scripts and the data provided
• Order more data that fits your purpose using the WUI
• Best way of learning new tools is using them!
• Some ideas to start with:
Create
own
project, we are
- Have
a look at your
your home
town:
to annual
learncycle
from
you!
Whatcurious
is the average
of cloud
cover, solar
radiation, humidity?
Is there a trend in solar surface radiation?
Are there temporal/spatial correlations between solar
radiation, cloud cover, humidity in your country?
Which was the ‘best’ summer / winter in the last five years?
- Which region in your country has the highest cloud cover (e.g.,
in July)? Is this the region with the highest solar radiation?
June 2012
Now it is your turn!
Any Questions?
We will be back later today (14 UTC) in
Centra for further comments / questions.
See also the course page:
• Download CM SAF Sample File
including instructions (screencasts).
• Forum at Course page for further
discussions.
Have fun playing with the data!
June 2012