EUMeTrain LSA SAF Week
Ricardo Silva, IM
Session 2 – Messing around with the data
MONITORING
WEATHER
CLIMATE
SEISMIC ACTIVITY
TO
A SUSTAINABLE DEVELOPMENT AND A SAFER WORLD
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Who am I?
Ricardo Silva
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Where do you come from?
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
How often do you use LSA SAF products?
Regularly
Occasionally
Never used, but plan
to start
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Session 2 Summary

Obtaining LSA SAF products

General Product Information

Documentation

File structure

Quick Visualization using HDFView

GIS tools for visualization and analysis

Quantum GIS

GRASS GIS

ILWIS

Tool for georeferencing LSA SAF products

Using programming tools – Python
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Obtaining LSA SAF products

Near Real Time Dissemination via EUMETCast

Dissemination on-demand

http:landsaf.meteo.pt

Products are available for registered users



Registration is free of charge, but may take up to 3 working
days for approval
Downloads use the FTP protocol
Several download methods according to desired temporal
extent and download settings
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
How do you obtain LSA SAF products?
EUMETCast
LSA SAF
(website, FTP,
helpdesk)
Other?
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Obtaining LSA SAF products

Dissemination options

Ordering methods

Random slot ordering



Long series ordering



The user can select individual time slots for several products
Automatic dissemination
Ordering of long time series of products.
Requires human intervention and may take up to one week
Dissemination methods

LSA SAF FTP server


The order is placed in the LandSAF server for the user to download
User's personal FTP server

LSA SAF's system transfers the order directly to the user's FTP server
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Product Documentation

Product User Manual (PUM)

Main documentation source





Physical phenomena
Algorithm basis
Output data specifications
Etc.
Product Output Format (POF)

Details concerning description of metadata, file naming
formats, etc.

Validation Report (VR)

Algorithm Theoretical Basis Document (ATBD)
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Product file structure

LSA SAF products use the HDF5 format




http://www.hdfgroup.org/
Hierarchical Data Format (HDF)

Groups – Containers for datasets and other groups

Datasets – Multidimensional arrays of a homogeneous type

Attributes – Metadata on groups and datasets
HDF files:

Are self-describing. Complex data relationships and dependencies

Can contain binary data

Support n-dimensional datasets
Each element in the dataset may itself be a complex object
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Product file structure

LSA SAF HDF5 files:

A common set of attributes on the root group

general information

A dataset for the main product (LST, DSLF, etc)

Additional datasets for cell metadata (Quality flags, etc.)

Each dataset may have:



Specific attributes (Scaling factor, missing value, etc.)
Different dimensions (Most datasets are 2D)
Different data types (Most datasets are 16bit signed integers)
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Quick visualization of products

HDFView software
http://www.hdfgroup.org/hdf-java-html/hdfview/index.html#download_hdfview

Java-based HDF Viewer

Inspection of a HDF5's structure and data

Visualize data in spreadsheet or image mode

Basic statistics, line plotting and histogram generation

Export to image formats
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Have you used HDFView before?
Yes
No
HDFWhat?
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
HDFView
1. Datasets
2.Attributes
3.Table View
4.Table options
5.Image view
6.Image options
7.Pixel values
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
HDFView
Inspecting file and dataset attributes
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
HDFView

When visualizing
products take note:
1. Cell values are unscaled
2. The 'missing value'
values have no physical
meaning
3. Product is not
georeferenced (line,
column coordinates
only)
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
GIS tools for visualization and analysis



Quantum GIS

User friendly interface

Many plugins created by the community
GRASS GIS

Many raster analysis modules

Python scripting
ILWIS


Direct import of LSASAF products through the
GEONETCast toolbox
Other tools (ArcGIS, Mapinfo, SAGA, etc.)
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Quantum GIS

http://www.qgis.org/

Open source multi-platform GIS software

User-friendly interface

Reads many input formats



Raster files, Vector files, Web services, etc
On-the-fly reprojection for vector and raster data
Large and friendly community of users and
developers
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Quantum GIS
Integration of different data sources
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Quantum GIS
Quick inspection of relationships between data
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
GRASS GIS

Geographic Resources Analysis Support System




http://grass.fbk.eu
Open source multi-platform GIS software
Hundreds of modules for raster and vector
processing
Reads many input formats

Can read HDF5 directly (won't be georeferenced)

It is preferable to georeference the HDF5 file first
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
GRASS GIS
Modular and flexible user interface
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
GRASS GIS

Raster calculator


Allows for computing a combination of logical and algebraic
expressions
Example: Calculate the arithmetic mean LST values for a specific day (using
hourly sample)

24 raster files each with the LSA SAF LST product for the hourly timeslots
Approach #1: Using only cells where there is enough data for the whole day
(In GRASS, any math performed with NULL cells always results in a NULL value for these cells
(
a+ b+ c
)
3
Approach #2: Using all the cells where there is at least one value in the day
(
if (isnull (a), 0, a)+ if (isnull (b), 0, b)+ if (isnull (c) , 0, c)
)
if (isnull (a), 0,1)+ if (isnull (b), 0,1)+ if (isnull(c) ,0,1)
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
GRASS GIS
Approach #1
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
GRASS GIS
Approach #2
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
GRASS GIS

Most commands are available in two interfaces:

Graphical – Good for visualization and exploratory analysis

Command-line – Nice for automating repetitive tasks
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Are you used to creating small scripts for
automating tasks?
Yes
Not really
I might start using them
In the future...
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
GRASS GIS

Python scripting example:

Automatically import and rename 191 LST files for a
given day
GRASS 6.4.1 (world4326):~ > python
>>>
>>>
>>>
>>>
>>>
>>>
>>>
...
...
...
...
...
...
# example importing all the LST geotiffs into GRASS
import grass.script as grass
from glob import glob
import os
import datetime as dt
for path in glob('/home/geo2/test_data/lsasaf/modificados/georefs/*LST*.tif'):
fName = os.path.basename(path)
nameFields=fName.split('_')
product, area = nameFields[3:5]
timeslot = dt.datetime.strptime(nameFields[5], '%Y%m%d%H%M')
newName = '_'.join((product, area, timeslot.strftime('%Y_%m_%d_%H_%M')))
grass.run_command('r.in.gdal', flags='oe', overwrite=True, input=path, output=newName)
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
ILWIS

Integrated Land and Water Information System

http://52north.org/communities/ilwis/

Open Source GIS software

Available only for MS Windows (as of now)


Imports LSA SAF products, using the GEONETCast
Toolbox
Several modules for raster and vector processing
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
GEONETCast Toolbox

GEONETCast Toolbox

http://52north.org/communities/earth-observation/geonetcast-toolbox-software

Available as a toolbox for ILWIS

Developed by ITC – University of Twente

Enables easy access to and management of
GEONETCast data


various satellite and environmental data and/or resulting
products
Tailored for users running GEONETCast reception
stations
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
ILWIS/GEONETCast


Installation

Follow the GEONETCast Toolbox's user manual

Read section 1.1 and 1.2 CAREFULLY
Importing LSA SAF products

Beware of file names:

Files obtained through GEONETCast (EUMETCast)



S-LSA_-HDF5_LSASAF_MSG_LST_Euro_201111060000.bz2
Everything will work OK
Files obtained through LSA SAF's on-demand dissemination



HDF5_LSASAF_MSG_LST_Euro_201111060000.bz2
GEONETCast Toolbox expects the 'S-LSA_-' prefix and will throw errors
Solutions:
 Uncompress → rename → recompress (time consuming and boring)
 Modify the GEONETCast's batch import scripts
 Contact LSA SAF's helpdesk and they'll send you the modified scripts
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
ILWIS/GEONETCast
Modular user interface, integrating the GEONETCast
Toolbox
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
ILWIS/GEONETCast
Use native NGP projection of the LSA SAF products
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Have you used these GIS tools before?
Yes, some of them
No
I might start using them
In the future...
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
LSA SAF Product Coordinate System

Normalized Geostationary
Projection (NGP)



Projection of the underlying
MSG LRIT source data
Important parameters for
MSG satellite:

Satellite height above
ellipsoid: 35785831 m

Central meridian: 0º
Documented on
EUMETSAT's LRIT and
HRIT Global Specification

http://www.eumetsat.int/groups/cps/documents/document/pdf_cgms_03.pdf
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
LSA SAF Product Coordinate System


LSA SAF products hold coordinate system
information in the attributes of the root group
But HDF5 files have no standard way of defining
coordinate systems

It's hard to automatically import LSA SAF products into
a Geographical Information System (GIS) software AND
have the data correctly georeferenced
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Visualization of LSA SAF products
HDFView is nice
for a quick inspection
GIS tools are nice for
visualization, analysis
map making, etc.
Some GIS tools
can open HDF5 directly
Most GIS tools cannot
recognize LSA SAF
coordinate information
The files will not be
projected
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
BUT ...
It won't be possible to
overlay different data
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
UNLESS …
URL: http://www.meteo.pt
LSASAF Georeferencer Tool

A graphical tool for georeferencing LSA SAF
products

Still in-development, but already usable

Open development


Source code hosted on a public repository

https://github.com/ricardogsilva/LSASAF-georeferencer
Uses open source tools


Python, GDAL, Proj, Qt4
Multi-platform
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
LSASAF Georeferencer Tool

Convert LSASAF HDF5 files to GeoTiff




GeoTiff is a Tiff based interchange format for georeferenced raster
imagery
Reproject from the NGP projection to other coordinate
systems
Apply scaling factor and missing value information to the
output file
Output GeoTiff files are suitable for further processing in
GIS software

Process multiple files simultaneously

Also has a CLI interface, useful for scripting
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
LSASAF Georeferencer Tool

Graphical Window usage
example
1. Select the file(s) to process
2. Load file information
3. Choose the dataset(s) to be
processed
4. Choose the output
projection
5. Choose the output directory
6. Process the files
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
LSASAF Georeferencer Tool

Command line usage example
# display usage information
ricardo@geo2:~$ ./georef_hdf5.py -h
# process all the LST files in ~/input_dir
ricardo@geo2:~$ ./georef_hdf5.py -x -o ~/my_output_dir ~/input_dir/*LST*
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Guess who is developing the LSA SAF Georeferencer Tool?
Ricardo Silva
(me) :)

Some other
dude...
Contributions, bug reports, etc. are very much
welcome!
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Programming tools – Python

Python(x, y)


http://code.google.com/p/pythonxy/
A (free) scientific and engineering development software
for




numerical computations,
data analysis
data visualization
Bundles Python and a big set of extra modules in a single
package

Includes modules for manipulating HDF5 files
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Programming tools – Python

Some useful python modules for working with LSA SAF products:



NumPy

The fundamental package needed for scientific computing with Python

Provides an N-dimensional array object and specialized data types and functions
Scipy

A library built to work with NumPy arrays

Provides efficient numerical routines for interpolation, statistics, linear algebra, etc.
Matplotlib



2D plotting library which produces publication quality figures
PyTables

Package for managing HDF5 files

Designed to efficiently and easily cope with extremely large amounts of data
Gdal

A translator library for raster geospatial data formats.

Has read/write functionality for many raster formats (HDF4/5, GeoTiff, GRIB, netCDF, etc.)

Allows assigning georeferencing information
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Programming tools – Python

Example interactive session for manipulating a LST product
ricardo@geo2:~$ python
>>> import tables
>>> import numpy as np
>>> import matplotlib.pyplot as plt
>>> product = tables.openFile('path_to_HDF5_file')
# list attributes of the root group
>>> product.root._v_attrs
# list attributes of the LST dataset
>>> product.root.LST.attrs
# import the LST dataset into a 32bit float numpy array
>>> lst = np.array(product.root.LST, dtype=np.float32)
# convert missing values to nan and apply scaling factor
>>> lst[lst == product.root.LST.attrs['MISSING_VALUE']] = np.nan
>>> lst = lst / product.root.LST.attrs['SCALING_FACTOR']
# plot the data
>>> plt.imshow(lst, interpolation='nearest')
>>> plt.colorbar()
>>> plt.show()
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
Programming tools – Python

The script's output:
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt
END of session 2
Thank you
for attending!
Enjoy the rest of
the LSA SAF week
Ministério da Ciência, Tecnologia e Ensino Superior
Instituto de Meteorologia, I. P. Rua C – Aeroporto de Lisboa 1749-077 Lisboa – Portugal
Tel.: (351) 21 844 7000
Fax: (351) 21 840 2370
e-mail: informacoes@meteo.pt
URL: http://www.meteo.pt