Ancillary File Tutorial
Keir Bovis June 2013
© Crown copyright Met Office
Overview of the tutorial
• What is an ancillary file ?
• Review and summarise some important ancillary
files used in NWP.
• Creating ancillary files using the Central Ancillary
Program (CAP)
• Creating ancillary files from PP formatted data
and NetCDF.
© Crown copyright Met Office
What is an ancillary file ?
• Ancillary files are the mechanism by which external
data sources are entered into the Unified Model (UM).
• Ancillaries files typically comprise static data held on
disk in UM fields file format.
• Ancillary files may hold data relating to model
orography, soil and vegetation types, and climatologies
for sea surface temperature and sea ice amongst
others.
• At the UK Met Office, the Central Ancillary Program
(CAP) creates ancillary files by reading post-processed
source data and writing them in UM fields file format.
© Crown copyright Met Office
Ancillary files
SST/ice climatology
Soil moisture/snow climatology
Vegetation surface type and LAI
Soil parameters
Model orography (topography)
Land-sea mask
© Crown copyright Met Office
Assumed
underpinning
hierarchy
Land-sea mask
• Name: qrparm.mask
• Key field:
• Land mask stored as a logical field (1=land, 0=sea)
• Application:
• Key ancillary used to differentiate between land and sea points.
• The field is usually stored in model start dumps.
• Source:
• International Geophysical Biosphere Programme (IGBP).
http://edc2.usgs.gov/glcc/globe_int.php
• 1km resolution obtained from Advanced Very High Resolution
Radiometer (AVHRR) data spanning April 1992 to March 1993.
© Crown copyright Met Office
Land-sea mask
Sea points
Land points
Note the absence of inland lakes. Majority of these are defined as
land points with a surface type of inland water and not as sea
points.
© Crown copyright Met Office
Land-sea mask: post-processing
• The default land-sea mask generated by the Central Ancillary
Program (CAP) may not be suitable for use in NWP models.
• For example, very small lakes and islands may lead to
numerical instabilities and run-time failures in NWP models.
Alternatively, some land points are missing in the IGBP data
set, e.g. Hawaii, Chatham Islands, etc.
• A facility exists with the CAP to allow the user to override the
default mask and reassign land to sea point and vice-versa.
• The overrides may be manually created using a facility in the
CAP or by using the graphical IDL-based tool ‘edit_lsm’ (we
will see both approaches in the workshop later).
© Crown copyright Met Office
Model orography (topography)
• Name: qrparm.orog
• Key fields:
• Mean orography in grid box (metres above sea level), standard deviation and
three gradient orography fields.
• Application:
• These fields are used in gravity wave drag scheme, radiation scheme and
hydrological inundation models. Stored in model start dumps.
• Source:
• Global models use the GLOBE dataset at 1km resolution.
http://www.ngdc.noaa.gov/mgg/topo/globe.html
• UK regional models use restricted military DTED dataset.
• Alternative data sets include the Shuttle Radar Topography Mission (SRTM)
data set http://www2.jpl.nasa.gov/srtm/cbanddataproducts.html
© Crown copyright Met Office
Model orography – mean field
© Crown copyright Met Office
Orography: post-processing
• Filtering is applied to remove scales shorter than 6km so that
features too small to generate flow blocking and gravity
waves are not included.
• Inland lakes are automatically detected and given values as if
they were land points. If a high lake such as Lake Victoria is
given a height of zero then it is likely that the model will fail.
© Crown copyright Met Office
Soil parameters
• Name: qrparm.soil
• Key fields:
• Soil moisture volume content at critical, wilting and saturation points.
• Application:
• Describe the physical properties of the soil and are important in correctly
determining the surface energy / soil moisture balance and the exchange of
moisture and heat between the atmosphere and land surface.
• Source:
• NWP previously used Wilson & Henderson-Sellers (WHS) at resolution 1°
latitude X 1° longitude originally designed for climate models.
• More up-to-date Harmonised World Soils Database (HWSD) at 1km resolution
is now used operationally. As quality is variable it is supplemented with other
regional soils datasets using an optimal interpolation scheme.
http://www.fao.org/nr/water/news/soil-db.html
© Crown copyright Met Office
Soil parameters – key soil fields
Critical point
Saturation point
Soil pores filled
with water and soil
is said to be
saturated. There is
no air left in the
soil.
Accurate specification of these
fields is important as they are
used in calculating the
movement of water through the
implemented soil hydraulic
scheme
Irrigation Water Management: Introduction to irrigation, FAO.
© Crown copyright Met Office
After drainage
has stopped,
large pores in
soil are full of air
and smaller are
full of water.
Wilting point
The point at
which a plant
wilts, there is not
enough water in
the soil for the
plant root to
extract.
Soil parameters – soil hydraulics
• Soil hydraulic schemes are used to parameterise the
rate of infiltration of water through the soil by describing
the relationship between volumetric soil moisture and
soil suction.
• Two different schemes have been implemented in Met
Office models, Clapp and Hornberger and van
Genuchten each requiring there own soil ancillaries.
• Correct parameterisation of each scheme will ensure
correct water ponding and run-off over the ground
surface.
© Crown copyright Met Office
Vegetation fraction
• Name: qrparm.veg.frac
• Key fields:
• Fraction of surface type expressed as a value (0-1).
• Application:
• Describe the fraction of each grid box that occupies each surface type
defined within the land-surface model.
• Used in land-surface model and surface analysis.
• Source:
• International Geophysical Biosphere Programme (IGBP).
http://edc2.usgs.gov/glcc/globe_int.php
© Crown copyright Met Office
Vegetation fraction types
land-surface scheme implemented in the UM recognises 9 surface types:
Broad leaf tree
Shrub
© Crown copyright Met Office
Needle leaf tree
Urban
C3 grass
Inland water
Bare soil
C4 grass
Land ice
Vegetation fraction types
The impact of incorrect specification of surface types affects evapotranspiration in
NWP models.
© Crown copyright Met Office
Veg fraction: post-processing
• Source IGBP data is interpolated onto the model grid
and the fraction of each of the IGBP classes present is
calculated.
• Not every IGBP grid point has a defined class and final
totals are adjusted to remove any areas of missing data.
IGBP classes consisting of less than 1% of the grid box
are eliminated, the area reallocated to other classes.
• The fraction totals of the 9 UM surface types are
calculated by mapping IGBP classes to UM surface
types using mapping weights.
© Crown copyright Met Office
Veg fraction: post-processing
17 IGBP vegetation classes
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
Evergreen Needleleaf Forest
Evergreen Broadleaf Forest
Deciduous Needleleaf Forest
Deciduous Broadleaf Forest
Mixed Forest
Closed Shrublands
are to
Open Shrublands
mapped
Woody Savannas
Savannas
Grasslands
Permanent Wetlands
Croplands
Urban and Built-Up
Cropland/Natural Vegetation Mosaic
Snow and Ice
Barren or Sparsely Vegetated
Water Bodies
© Crown copyright Met Office
9 fractional UM classes
be
on to
1.
2.
3.
4.
5.
6.
7.
8.
9.
Fraction of broadleaf trees
Fraction of needleleaf trees
Fraction of C3 grass
Fraction of C4 grass
Fraction of shrub
Fraction of urban
Fraction of water
Fraction of bare soil
Fraction of ice
IGBP -> UM mapping weights for
vegetation fraction totals
UM surface types
Broadleaf
Needleleaf
C3 Grass
C4 Grass
Shrub
Urban
Water
Bare soil
Ice
Evergreen needleleaf
0.0
70.0
20.0
0.0
0
0.0
0.0
10.0
0.0
Evergreen broadleaf
85.0
0.0
0.0
10.0
0.0
0.0
0.0
5.0
0.0
Deciduous needleleaf
0.0
65.0
25.0
0.0
0.0
0.0
0.0
10.0
0.0
Deciduous broadleaf
60.0
0.0
5.0
10.0
5.0
0.0
0.0
20.0
0.0
Mixed forest
35.0
35.0
20.0
0.0
0.0
0.0
0.0
10.0
0.0
Close shrub
0.0
0.0
25.0
0.0
60.0
0.0
0.0
15.0
0.0
Open shrub
0.0
0.0
5.0
10.0
35.0
0.0
0.0
50.0
0.0
Woody savanna
50.0
0.0
15.0
0.0
25.0
0.0
0.0
10.0
0.0
Savanna
20.0
0.0
0.0
75.0
0.0
0.0
0.0
5.0
0.0
Grassland
0.0
0.0
66.0
15.7
4.9
0.0
0.0
13.5
0.0
Permanent wetland
0.0
0.0
80.0
0.0
0.0
0.0
20.0
0.0
0.0
Cropland
0.0
0.0
75.0
5.0
0.0
0.0
0.0
20.0
0.0
Urban
0.0
0.0
0.0
0.0
0.0
100.0
0.0
0.0
0.0
Cropland/natural mosaic
5.0
5.0
55.0
15.0
10.0
0.0
0.0
10.0
0.0
Snow and ice
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
100.0
Barren
0.0
0.0
0.0
0.0
0.0
0.0
0.0
100.0
0.0
Inland water
0.0
0.0
0.0
0.0
0.0
0.0
100.0
0.0
0.0
IGBP class
© Crown copyright Met Office
Composite mapping
1-to-1 mapping
Leaf Area Index (LAI)
• Name: qrparm.veg.func
• Key field:
• The Leaf Area Index (LAI) is a measure of leaf area on each of the vegetative
surface types.
• Application:
• LAI used in land-surface model in the determination of surface moisture and
heat fluxes, through impact on transpiration, controlling the partitioning
between surface sensible and latent heat fluxes.
• Source:
• Seasonal climatology derived from monthly means of the Moderate Resolution
Imaging Spectroradiometer (MODIS) global 4km LAI product, from 2005-2009.
ftp://primavera.bu.edu/pub/datasets/MODIS/MOD15_BU/C5/LAI/data/monthly/
4km/
© Crown copyright Met Office
Leaf Area Index (LAI)
LAI is the total one-sided area of leaves in a vertical column, within a unit area of
ground
© Crown copyright Met Office
MODIS Broadleaf trees LAI
July climatology
Time-varying zonal mean
Boreal forest growth
December climatology
SH
NH summer
© Crown copyright Met Office
NH
MODIS needleleaf trees LAI
July climatology
Time-varying zonal mean
Boreal forest growth
December climatology
SH
© Crown copyright Met Office
NH summer
NH
SST & sea-ice climatologies
• Name: qrclim.sst, qrclim.seaice
• Key fields:
• The skin SST is the temperature of the top few micrometres over sea points at
12 monthly intervals.
• Fraction of sea-ice in sea over at 12 monthly intervals.
• Application:
• Both fields used in land-surface model and in construction of surface analyses.
• Source:
• The Met Office Hadley Centre's sea-ice and Sea Surface Temperature (SST)
data set, HadISST1, created from monthly globally-complete fields of SST and
sea-ice concentration on a 1 degree latitude-longitude grid from 1870 to-date
http://www-hc/~hadobs/www.hadobs.org/hadisst/
© Crown copyright Met Office
Sea surface temperature
climatology
Zonal mean of 12 month climatology
SH
NH summer
© Crown copyright Met Office
NH
Monthly climatology for July
Sea-ice climatology
Zonal mean of 12 month climatology
Monthly climatology for January
The sea ice thickness field is arbitrarily assigned values of 2m in the Arctic and 1m in
the Antarctic. However, a separate dataset exists for models that have variable sea-ice
thickness.
© Crown copyright Met Office
Soil moisture climatology
• Name: qrclim.smow
• Key fields:
• The soil moisture content in each soil layer at 12 monthly intervals.
• Application:
• The soil moisture climatology is used during the creation of a soil
moisture analysis.
• Different climatologies will exist depending on soil parameters and soil
hydraulic scheme used.
• Source:
• The climatology is created by running the Joint UK Land Environment
Simulator JULES model (standalone land-surface model) off-line for 10
years following a 6 year spin-up period. http://www.jchmr.org/jules/
© Crown copyright Met Office
Soil moisture climatology
Created using WHS soil parameters
Created using HWSD soil parameters
We hope the new climatologies are more realistic in the climatologies derived from
the HWSD soil parameters. For example we see drier response over sandy soils in
Portugal and finer structures as a result of increased HWSD resolution.
© Crown copyright Met Office
Snow climatology
• Name: qrclim.smow
• Key fields:
• The snow amount over land (kg/m-2) at 12 monthly intervals.
• Application:
• This ancillary is available for model initialisation.
• Source:
• Climatology created by constructed from Willmott and Rowe (WR) and
source climatologies from the Atmospheric Model Intercomparison
Project (AMIP) at 1° X 1° resolution.
© Crown copyright Met Office
Snow climatology
Zonal mean of 12 month climatology
SH
NH winter
© Crown copyright Met Office
NH
Monthly climatology for February
Implementation of ancillary
files
• We’ve looked at the different types of ancillary files used in
the unified model and now we move onto examine how
these files are stored and used.
• Specifically we will look at:
• How they are created using the CAP
• Other methods of creating them
• Briefly how to visualise them
© Crown copyright Met Office
Creating ancillary files
•
There are two methods used to create ancillary files:
1. Create them using the Central Ancillary Program (CAP).
2. Importing processed data into ancillary fieldsfile format
from
• a PP (Post-processed) format
• A NetCDF format.
•
We’ll look at the CAP first and then other methods of
importing data.
© Crown copyright Met Office
The Central Ancillary
Program (CAP)
• The CAP comprises a suite of FORTRAN, C and UNIX shell
scripts that are run to construct ancillary files from a postprocessed data source.
• Ancillary file content is generated according to a predetermined list of fields implemented in the code.
• It is has been previously ported to IBM, NEC and Cray
platforms.
• Can also run on LINUX for use in the workshop that follows.
• All code is version controlled using an subversion repository
and FCM.
© Crown copyright Met Office
pptoanc
• This UM utility may be used to create an UM ancillary format
fieldsfile from a PP data source.
• PP files may have been used in IDL to carry out processing
on ancillary file data.
• The PP file must be converted back to fieldsfile format before
it can be used in the UM.
• It is run from the command line
• pptoanc -hpf -n <namelistFile> <PPinputFile> <ancillaryFile>
• An example namelist file is shown in the next slide.
© Crown copyright Met Office
pptoanc namelist
&SIZES
field_types=1,
n_times=1,
nlevels=70,
n_pp_files=1,
field_code=287,
stash_code=57,
nlevs_code=70,
len_intc=15,
len_realc=6
/
&LOGICALS
single_time=.true.
pack32=.true.,
pphead=.true.
field_order=.true.
lwfio=.true.
/
&FIRST_VT
fvhh=0,fvdd=0,fvmm=0,fvyy=0
/
&INTERVAL
year360=f,ivhh=0,ivdd=0,ivdd=0,ivmm=0,ivyy=0
/
&LAST_VT
lvhh=0,lvdd=0,lvmm=0,lvyy=0
/
&HEADER_DATA
fixhd(2)=1
fixhd(3)=1
fixhd(12)=600
/
© Crown copyright Met Office
Number of field types, times and max levels in PP
file. Mapping of UM stash codes to PP field codes.
Number of levels for each code. Length of integer
and real constants array (15 & 6 for ancillaries)
All input fields valid at a single time, pack real fields
using 32 bit numbers print out pp headers read in,
fields are ordered by time, output ancillary file or
dump well-formed.
Defines the first validity time of the ancillary data.
Defines the time interval between the validity times.
Defines the last validity time of the ancillary data.
Used to overwrite values in the fieldsfile header: submodel=atmosphere; vertical coordinate. type=hybrid;
model version number x 100 + release.
Xancil – creates ancillary files
from NetCDF sources
• Written by Jeff Cole at the Department of Meteorology,
University of Reading.
• Xancil takes NetCDF data and converts it into UM ancillary
format.
• Comprises a front-end Tcl/Tk GUI with the same look and feel
as xconv.
• Back-end processing is a compiled FORTRAN source single
executable.
• Xancil contains none of the CAP functionality and cannot
operate on un-processed source data for the purpose
ancillary file creation.
© Crown copyright Met Office
Xancil overview
Pre-defined set of
known ancillary
files to create
Selected ancillary
file has a predefined set of
known fields
Symbolic mapping
to fields in
NetCDF source
© Crown copyright Met Office
Viewing ancillary files
• They can be viewed graphically using xconv or using
in-house IDL routines such as pp_contour.
• Contents can be dumped using the UM ‘pumf’ (print
UM file) utilities.
• Different versions of ancillary files can be compared
using the UM ‘cumf’ (compare UM file) utility.
© Crown copyright Met Office
Practical session
• By the end of the practical session you should be able to:
• Compile the CAP on a LINUX platform.
• Create a default land-sea mask for use in a global NWP model.
• Be able to manually edit the default land-sea mask to reassign
land and sea points.
• Experiment in modifying a land-sea mask using the graphical
edit_lsm tool.
• Please copy the pdf below to access the tutorial:
/home/h01/frke/CAPworkshop.pdf (use evince to view it).
• Please ask if you get stuck or have any questions..
© Crown copyright Met Office
References
• Brutsaert,W. (1982). `Evaporation into the atmosphere. Theory,
history and applications'. D. Reidel Publishing Company, Dordrecht.
• Calder,I.R., Wright,I.R. and Murdiyarso,D. (1986). `A study of
evaporation from tropical rainforest - West Java'. Journal of
Hydrology, 89, pp13-23.
• Cosby, B.J., Hornberger, G.M., Clapp, R.B. and Ginn, T.T. (1984). `A
statistical exploration of the relationships of soil moisture
characteristics to the physical properties of soils.` Water Resources
Research, 20 pp682-690.
• Dickinson, A. and Wilson, C.W. (1991). `Interpolation Techniques
and Grid Transformations used in the Unified Model'. (Unified Model
Documentation Paper (UMDP) S1. Unpublished paper.)
• Dingman,S.L., Barry,R.G., Weller,G., Benson,C., Le Drew,E.F. and
Goodwin,C.W. (1980). `Climate, snow cover, microclimate and
hydrology'. In An Arctic ecosystem: The coastal tundra at Barrow
Alaska, Brown,J., Miller,P.C., Tieszen,L.L. and Bunnel,F.L. (eds).
Dowder, Hutchinson and Ross, Stroudsburg,
© Crown copyright Met Office
References
• Eagleson (1970) `Dynamic Hydrology'. McGraw Hill, New York.
• Evans,R (1995) 'Creation of Data Fields Required for Effective
Roughness Parameterisation'. Forecasting Research Technical
Report No. 146. (Unpublished Paper).
• FAO (1995). ‘The digital soil map of the world’. FAO, Rome.
• Folland, C.K. and Parker, D.E. (1990). `Observed Variations of Sea
Surface Temperature' in `Climate-Ocean Interaction', M.E. Sclesinger
(ed).
• Halldin,S., Saugier,B. and Pontailler,F.Y. (1984). `Evapotranspiration
of adeciduous forest. Simulation using routine meteorological data'.
Journal of Hydrology, 75, pp323-341.
• Houldcroft,C.J., Grey,W.M.F., Barnsley,M., Taylor,C.M. and Los,S.O.
New vegetation albedo parameters and global fields of backgroudn
albedo derived from MODIS for use in climate model. To be
published, currently available from Department of Geography,
Swansea University, Singleton Park, Swansea, SA2 8PP.
© Crown copyright Met Office
References
• Li,D. and Shine,K.P. (1995) ‘A 4-Dimensional Ozone Climatology for
UGAMP Models’. UGAMP Internal Report No. 35 (Unpublished
paper).
• Lindroth,A. (1985). `Canopy conductance of coniferous forests
related to climate'. World Resources Research, 21, pp297-304.
• Lu, S., Ren, T, Gong, Y and Horton,R (2007) An improved model for
predicting soil thermal conductivity from water content at room
temperature. Soil Science Society of America, 71, pp8-14.
• Mason, P.J. (1985). `On the parameterization of orographic drag',
Proc. of the seminar on physical parametrizations for Numerical
Models, Reading, 9-13 September 1985, ECMWF.
• Meteorological Office (1991) `Meteorological Glossary`, HMSO.
• Monteith,J.L. (ed) (1976). `Vegetation and the Atmosphere, 2'.
Academic Press, London.
© Crown copyright Met Office
References
• Parker, D.E, Jackson, M., Horton, E.B. (1995) Forthcoming Climate
Research Technical Note.
• Posey,J.W. and Clapp,P.F. (1964). `Global distributions of normal
surface albedo'. Geofis. Int., 4, pp333 348.
• Peters-Lidard,C.D., Blackburn,E., Liang,X and Wood,E.F. (1998) The
effect of soil thermal conductivity parameterization on surface energy
fluxes and temperatures. Journal of the Atmospheric Sciences, 55,
pp1209-1224.
• Post,W.M. Emanuel,W.R., Zinke,P.J. and Stangenberger,A.G. (1982).
‘Soil carbon pools and world life zones’. Nature 298 pp156-159.
• Robinson,D.A. and Kukla,G. (1984). `Albedo of a dissipating snow
cover'. Journal of Climatology and Applied Meteorology, 23, pp1626
1634.
• Rockwood,A.A. and Cox,S.K. (1978). `Satellite infrared surface
albedo over northwestern Africa'. Journal of Atmospheric Sciences,
35, pp513 522.
© Crown copyright Met Office
References
• Scholes,R.J., Skole,D. and Ingram,J.S. (1995). ‘A global database of
soil properties: Proposal for implementation. IGBP-DIS working paper
#10, University of Paris, France.
• Shuttleworth,W.J., Gash,J., Lloyd,C.R., Moore,C.J., Roberts,J.,
Marques Filo,A.O., Silvia Filo,V.P., Ribeiro,M.N.G., Molion,L.C.B.,
De Sa,D.A., Nobre,J.C.A., Cabral,O.M.R., Patel,S.R. and De
Moraes,J.C. (1984). `Eddy correlation measurements of energy
partition for the Amazonian forest'. Quarterly Journal of the Royal
Meteorological Society, 110, pp1143 1162.
• Sud,Y.C. and Smith,W.E. (1984). `Ensemble formulation of surface
fluxes and improvement in evapotranspiration and cloud
parameterizations in a GCM'. Boundary Layer Meteorology, 29,
pp185 210.
• Thompson,N., Barrie,I.A. and Ayles,M. (1981) . `The
Meteorological Office rainfall and evaporation calculation system:
MORECS. Hydrological Memorandum No. 45, Meteorological Office,
Bracknell.
© Crown copyright Met Office
References
• Van Genuchten, M.Th. (1980). ‘A closed form of the equation for
predicting the hydraulic conductivity of unsaturated soils. Soil Sci
Soc. Am. J., 44, pp892-898.
• Warrilow, D.A. and Buckley, E. (1989). `The impact of land surface
processes on the moisture balance of a climate model'. Annales
Geophysicae, 7, (5) pp439-450.
• Warrilow,D.A., Sangster,A.B. and Slingo,A. (1986). `Modelling of land
surface processes and their influence on European climate'.
Meteorological Office, Dynamical Climatology technical note 38,
(unpublished paper).
• Willmott, C.J., Rowe, C.M. and Mintz, Y. (1985). `Climatology of
the Terrestrial Seasonal Water Cycle', Journal of Climatology, 5,
pp589-606.
• Wilson, M.F. and Henderson-Sellers, A. (1985). `A global archive of
land cover and soils data for use in general circulation climate
models'. Journal of Climatology, 5, pp119-143.
© Crown copyright Met Office
References
• Woodward,F.I., Smith,T.M. and Emanuel,W.R. (1995). ‘A global land
primary productivity and phytogeograpy model.’ Global
Biogeochemical Cycles, 9, pp471-490.
• Woodward, S. (2001). ‘Modeling the atmospheric life cycle and
radiative impact of mineral dust in the Hadley Centre climate model.’
Journal of Geophysical Research, 106, No D16, pp18155-18166.
• Zinke,P.J., Stangenberger,A.G., Post,W.M., Emanuel,W.R. and
Olson,J.S. (1986). ‘Worldwide organic soil carbon and nitrogen data’.
NDP-018, Carbon Dioxide Information Center, Oak Ridge National
Laboratory, Oak Ridge, Tennessee, USA.
© Crown copyright Met Office