The Python Interface to the Earth
System Modeling Framework
Ryan O’Kuinghttons
Robert Oehmke
Cecelia DeLuca
Cooperative Institute for Research in Environmental Sciences
NOAA Environmental Software Infrastructure and Interoperability Project
American Meteorological Society Meeting
Austin, Texas
Jan 8, 2013
Introduction
• The Earth System Modeling Framework (ESMF) is open source software
for building modeling components, and coupling them together to
form weather prediction, climate, coastal, and other applications.
•
Currently supports a full Fortran and limited C interface
• ESMF provides a mature high performance regridding package
• Transforms data from one grid to another by generating and applying
interpolation weights
• Supports structured and unstructured, global and regional, 2D
and 3D grids, with many options
• Fully parallel and highly scalable
• The Python interface to ESMF (ESMPy) offers
access to the regridding functionality in ESMF
Why does ESMF need a
Python Interface?
• Enables ESMF regridding to be used with very little effort, in an object
oriented way:
• Regridding applied as a callable Python object
• Numpy array access to distributed data
• Some users report computation times reduced from hours to minutes
• Enables ESMF regridding to be used in other scientific packages with
Python-based workflows – current users include:
• UV-CDAT (PCMDI) – Ultrascale Visualization Climate Data Analysis Tools
• PyFerret (NOAA) – Python based interactive visualization and analysis
environment
• Community Surface Dynamics Modeling System (CSDMS) – tools for
hydrological and other surface modeling processes
ESMPy Classes
• ESMF
• Initialize and Finalize
• Logging
• Virtual Machine (parallel distribution)
• Grid
• Logically rectangular discretization object
• Mesh
• Unstructured mesh discretization object
• Field
• Grid or Mesh plus a data array and metadata
• Derived type of the Numpy Array
• Regrid
• Callable object which operates on two Fields to compute and apply
interpolation weights
A Few Words on Data
Conventions…
Following data conventions makes it easier to use data and tools.
ESMPy grid files follow standard data file formats:
-
Climate and Forecast (CF) grid conventions
• UGRID candidate CF convention for unstructured grids[1]
• GridSpec CF convention for logically rectangular grids (accepted but
not yet formally in CF) [2]
Supported Grids and Methods
• Spherical coordinates – bilinear, higher order patch [3.4], or first order
conservative regridding with:
• Global or regional 2D logically rectangular grids
• 2D unstructured meshes composed of triangles or quadrilaterals
• Cartesian (x,y) coordinates:
• Bilinear, higher order patch or first order conservative regridding between
any pair of:
• 2D meshes composed of triangles and quadrilaterals
• 2D logically rectangular grids composed of a single patch
• Bilinear and first order conservative regridding between any pair of:
• 3D meshes composed of hexahedrons
• 3D logically rectangular grids composed of a single patch
2D Unstructured Mesh
From www.ngdc.noaa.gov
FIM Unstructured Grid
Regional Grid
Code Examples
Create a Grid and access object information using Python properties:
import esmpy, numpy
grid = esmpy.Grid(maxIndex, numPeriDims=1, staggerlocs=[esmpy.StaggerLoc.CENTER])
lower_bounds = grid.lower_bounds[esmpy.StaggerLoc.CENTER]
upper_bounds = grid.upper_bounds[esmpy.StaggerLoc.CENTER]
Create a Mesh from UGRID formatted NetCDF file:
mesh = esmpy.Mesh(“mesh_netcdf_file.nc”, esmpy.FileFormat.UGRID)
Regridding object acting on two Fields:
from esmpy import Regrid
r1to2 = Regrid(grid, mesh, method=esmpy.RegridMethod.CONSERVATIVE)
destination_field = r1to2(source_field)
Write Grid and Mesh to .vtk formatted output files:
grid.write(“source_grid”)
mesh.write(“destination_mesh”)
Field is derived from Numpy Array:
exact_field = sqrt(source_field**5)
exact_slice = exact_field[:,:,5:-1]
Getting Started
Requirements:
- Python 2.6
- Numpy 1.6.1 (ctypes)
- ESMF installation
Testing:
- Regression tested nightly on 5 platforms
Supported Platforms:
- Linux, Darwin, and Cray
- Gfortran
- OpenMPI
Limitations:
- No object for collections of Fields
- No access to Mesh coordinates
- No access to Field bounds
Status and Future Work
• A prototype interface, ESMP, is currently available for download
• The beta release of a new interface, ESMPy, is expected in early 2013
• More “pythonic” API
• Later in 2013: addition of calendar and time management, and other
classes from ESMF
• Please contact esmf_support@list.woc.noaa.gov with questions or
development requests
Download
http://earthsystemcog.org/projects/esmp/releases
Questions?
References:
1. UGRID wiki:
http://publicwiki.deltares.nl/display/NETCDF/Deltares+CF+proposal+for+Unstructured+Grid+data+model
2. GridSpec wiki: https://ice.txcorp.com/trac/modave/wiki/CFProposalGridspec
3. Khoei S.A., Gharehbaghi A. R., The superconvergent patch recovery technique and data transfer operators in 3d
plasticity problems. Finite Elements in Analysis and Design, 43(8), 2007.
4. Hung K.C, Gu H., Zong Z., A modified superconvergent patch recovery method and its application to large
deformation problems. Finite Elements in Analysis and Design, 40(5-6), 2004.
ESMP
Interfacing with ctypes:
_ESMF.ESMC_GridGetCoord.restype = ctypes.POINTER(ctypes.c_void_p)
_ESMF.ESMC_GridGetCoord.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_uint,
numpy.ctypeslib.ndpointer(dtype=numpy.int32),
numpy.ctypeslib.ndpointer(dtype=numpy.int32),
ctypes.POINTER(ctypes.c_int)]
gridCoordPtr = _ESMF.ESMC_GridGetCoord(grid.struct.ptr, coordDim, staggerloc,
exclusiveLBound, exclusiveUBound,
ctypes.byref(lrc))
# adjust bounds to be 0 based
exclusiveLBound = exclusiveLBound - 1
Allocating Numpy array buffers for memory allocated in ESMF:
if grid.type == ESMP_TYPEKIND_R8:
gridbuffer = numpy.core.multiarray.int_asbuffer(
ctypes.addressof(gridCoordPtr.contents),
numpy.dtype(numpy.float64).itemsize*size)
Switching between Fortran and C array striding:
Work in progress!