A Spectral Global Atmospheric Circulation Model for
Distributed Parallel Execution
H. S. Porter, G. Cogdell, and C. Huneycutt (Department of Computer Science, Furman University)
H. G. Mayr (NASA Goddard Space Flight Center)
Abstract. We describe a numerical model of global atmospheric circulation that is designed to run efficiently in a
network-based, distributed-memory, parallel-processing environment. By formulating the model using spectral
expansions, the block diagonal structure of the coupling matrices arising in the semi-implicit time integration method
can be exploited for parallelizing the model. As a result, we are able to achieve significant speedups even when
running the model using distributed processors coupled via Ethernet. Analysis of the model running on a 10Mb/s
Ethernet LAN shows that we can achieve a parallel speedup of 9.1 when using 12 processors and the network
loading is light.
Introduction. Numerical modeling of global atmospheric circulation is one of the most computationally intensive
problems currently being studied. In fact, this problem is one of the NSF’s grand scientific challenges. Because of its
inherent high-level of computational complexity, one is forced to make compromises not only in the resolution
chosen for spatial and temporal scales, but also in the level of detail used to formulate the chemical and physical
processes simulated in the model. Because of the various types of approximations employed, many existing global
circulation models (GCMs) for the terrestrial atmosphere are not well suited either to model convective processes or
to perform comparative studies of the global circulation of planetary atmospheres. In the present paper, we describe
a parallel spectral model that is suitable for performing parametric studies of just such global scale nonlinear
dynamical processes.
Studying global scale dynamical processes such as the spinning up of a planetary atmosphere or determining
long-term tidal-oscillation periods requires long time integrations. For example, the time constant for spinning up the
atmosphere of Venus from rest until it reaches global scale dynamic equilibrium is approximately three hundred
years. Similarly, studying quasi-biannual tidal oscillations in the Earth’s atmosphere requires more than a ten year
integration. Because these integrations must be performed on systems of equations that include processes with very
short response times, such as sound waves, careful formulation of a model is imperative both to achieve numerical
stability as well as computational efficiency. Nonetheless, even with great care, the computational demands often
require more computational resources than a high performance workstation can provide.
Our main intent in this paper is to develop a model that can run efficiently on workstations distributed on an Ethernet
network. However, with a few modifications to take advantage of increased interprocessor communication speed,
this same model could also be easily transported to a distributed-memory, massively parallel computer. We have
adopted the model of Chan et al. (1994) as the starting point for our implementation because it is a spectrally based,
semi-implicit, time-marching approach that exhibits large scale granularity. Therefore a parallel implementation can
show significant speedup even for the relatively low interprocessor-communication-bandwidth found in Ethernet.
The implementation of Chan et al. (1994) was carefully optimized for a vector machine and we have been able to
retain much of that vector optimization in our distributed parallel adaptation. Mengel et al. (1995) have recently
developed an extension of the Chan et al. (1994) model which includes the Hines (1995) formulation of a
doppler-spread theory describing nonlocal momentum deposition due to atmospheric gravity wave production and
dissipation. Fortunately, we can also extend our parallel adaptation to include Hines’ theory with relatively little
effect on the parallel speedup. Because of the large additional computational demands this theory places on the
model, the enhanced speed obtained from parallel execution will be essential for the studies we have planned.
In concluding this section, we note that there has been considerable discussion in the literature contrasting the
advantages and disadvantages of spectral GCMs with grid point models. Spectral models have two main advantages
for our purposes in this study. First and perhaps most important is that semi-implicit time differencing schemes can
be easily incorporated (Robert, 1969; Simmons et al., 1978) thus allowing exploitation of the large-scale parallel
granularity inherit in the model formulation. Second, when used for low to moderate horizontal spatial resolution,
spectral models are computationally more efficient than grid point models (Manabe et al., 1979).