Lossless Compression of Meteorological Data in GRIB Format
R. Lorentz
Fraunhofer Institute for
Scientific Computation and Algorithms (SCAI)
Germany
The main content of this talk is the lossless compression of meteorological data in the GRIB1
format. First, the general issues involved when compressing data for archival and transmission
purposes are discussed. Then we review the possible data file formats and the data they include.
GRIBZip is software for the lossless compression of GRIB files, which was developed at the
Fraunhofer Institute SCAI in cooperation with the German Weather Service. . We report on the
compression factors and the throughput it achieves on two sources of data. The first is weather
forecasting data from the German Weather Service (DWD) for Europe. The DWD will be using
GRIBZip in their production runs. The next source is data from climate research at the German
High Performance Computing Centre for Climate- and Earth System Research.
The lossless compression factors obtained by GRIBZip depend very much on data to be
compressed, especially on the distance between grid points. The finer the grid is, the better is
the compression. An example is results for forecasts made by the German Weather Service
(DWD) with their local model, so called LME. The distance between grid points is 7
kilometers. The compression factor is defined to be the size of the source file divided by the
size of the compressed file. The compression was carried out on a 3 Ghz Linux PC
.
File
LM-E (320MB)
2D compression factor
2,65
3D compression factor
3,17
Total time
22 Sek.
Speed in MB/sec
16 MB/Sek
The decompression was about 20% slower than the compression.
Finally, we briefly discuss ongoing research, which includes the compression of data on
global triangular grids and spectral data using experimental programs.
This talk reports on work done together with R. Iza-Teran, M. Rettenmeier and C. Thole.