GEOS-CHEM: Structure,
Documentation, Platforms, and Parallelization
Bob Yantosca
Software Engineer
Atmospheric Chemistry Modeling Group
Harvard University
GEOSCHEM Users’ Meeting
02 June 2003
Topics
1. GEOS-CHEM history
2. GMAO met fields used to drive GEOS-CHEM
3. GEOS-CHEM model description
4. GEOS-CHEM standard versions and benchmarks
5. GEOS-CHEM user support and documentation
6. GEOS-CHEM platforms
7. Parallelization: OpenMP and MPI
8. Effective coding practices
9. Chemical forecasting
2 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
In the beginning …
GEOS-CHEM was originally created from:
The old Harvard/GISS-II 9-layer model (Larry, Yuhang, et al)
GEOS-CTM (Dale, Mian) – ancestor of GOCART
Pieces that came from the Harvard/GISS-II model:
Emissions (EMISSDR, BIOBURN, etc)
Dry Deposition (DRYDEP, DEPVEL, etc.)
Chemistry (SMVGEAR etc.)
Diagnostics (NDxx) and various input files
Pieces that came from the GEOS-CTM model:
Transport (TPCORE, by Lin & Rood)
BL mixing (TURBDAY) and Cloud Convection (NFCLDMX)
3 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
In the beginning …
We did not foresee all of the potential uses of GEOS-CHEM at the time we first created GEOS-CHEM.
At first, newer code was “onion-skinned” on top of older code.
This resulted in functional, albeit, convoluted code.
However, a number of recent developments (new met fields, mechanisms, etc.) has forced us to make some widespread changes to the GEOS-CHEM source code.
These problems are not unique to GEOS-CHEM, but also apply to any software product.
4 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
In the beginning …
Philosophical Digression:
“If builders built buildings the way programmers write programs, then the first woodpecker that came along would destroy civilization.” – Weinberg’s Law
“Inside every large program there is a small program struggling to get out.” – Hoare’s Law of Large Programs
Interpretation:
If you want to build a new building, you tear down the old one first and start from scratch.
With software, usually you start with an existing program and add new stuff on top of that. Your foundations may be shaky…
5 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
In the beginning …
Much of the existing GEOS-CHEM code has been recently updated, or brand-new code has been added
Biomass and biofuel emissions
Dry and wet deposition
Date/time functions
Error handling and file I/O
Sulfate chemistry
Main program
Some sections of GEOS-CHEM still need improvement
Diagnostics (especially timeseries)
Input file reading
Emissions for full chemistry (incl. Isoprene, Soil NOx)
6 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GMAO met fields
GEOS-CHEM is driven by assimilated met data from
NASA/GMAO (formerly DAO), located at GSFC.
GMAO has produced 4 major met data sets:
GEOS-1 (1985 – 1995, 20 layers to 10 hPa)
GEOS-STRAT (1995 – 1997, 46 layers to 0.1 hPa)
GEOS-3 (2000 – 2002, 48 layers to 0.01 hPa
GEOS-4/fvDAS (2003 –, 55 layers to 0.01 hPa)
At present, GEOS-CHEM can run with GEOS-1, GEOS-
STRAT, and GEOS-3 met fields.
GEOS-4 support is currently being added to GEOS-CHEM.
7 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GMAO met fields
Native horizontal resolution of GMAO met fields:
GEOS-1 2 o
GEOS-STRAT 2 o lat x 2.5
o lon lat x 2.5
o lon
GEOS-3 1 o
GEOS-4/fvDAS 1 o lat x 1 o lon lat x 1.25
o lon
We also can regrid met fields to coarser resolution in order to gain computational advantage for long simulations
GEOS-1 4
GEOS-STRAT 4 o o lat x 5 o lat x 5 o lon lon
GEOS-3 2
GEOS-4/fvDAS 2 o o lat x 2.5
o lat x 2.5
o lon AND 4 o lon AND 4 o lat x 5 o lat x 5 o lon lon
Regridding of met fields is done offline, and can be automated
8 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GMAO met fields
I-6 fields (6hr instantaneous “snapshot” quantities)
U, V, P, T, Specific humidity, albedo
A-6 fields (6-hr averaged quantities)
Cld. Frac. – random & max overlap (GEOS-1, GEOS-S)
In-cloud optical depth and total cld frac (GEOS-3)
Tendency in specific humidity (i.e. precip or evap)
Cloud mass flux and detrainment
A-3 fields (3-hr averaged quantities)
10m winds, roughness height, friction velocity
PBL height, column cloud fraction, surface temperature
Convective & total precipitation at ground
9 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GMAO met fields
A-6 fields
A-3 fields
Times of day when you need to read met fields
Z = “Zulu” = abbreviation for GMT
GEOS-1
GEOS-S
GEOS-3
A-6 fields
A-3 fields
From the fvDAS file spec on GMAO’s website
10
GEOS-4
Start of day
GEOS-CHEM: Structure, Platforms, ...
Start of next day
Monday, June 2, 2003
GMAO met fields
GEOS-1, GEOS-STRAT, GEOS-3
A-6 fields are stamped w/ center times: 00, 06, 12, 18 GMT (read them at 21, 03, 09, 15 GMT)
A3 fields are stamped w/ end times: 00, 03, 06, … 21 GMT
GEOS-4 / fvDAS
A-6 fields are stamped w/ center times: 03, 09, 15, 21 GMT (read them at 00, 06, 12, 18 GMT )
A-
3 fields are stamped w/ center times: 1:30, 4:30, 7:30, … 22:30
GEOS-4 timestamps and averaging periods differ from
GEOS-3, GEOS-1, and GEOS-STRAT!
11 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GMAO met fields
Preparing GEOS-CHEM for GEOS-4/fvDAS met fields:
1. Download, regrid, and format fvDAS met fields. (in progress)
2. Adapt code to read in fvDAS met fields w/ different timestamps and averaging intervals. (in progress)
3. Implement new TPCORE transport code for fvDAS. This contains new semi-lagrangian vertical advection. (done)
4. Fit Philip Cameron-Smith's pressure fixer to the fvDAS TPCORE transport scheme. Add additional mixing ratio adjustment for residual mass difference to ensure mass conservation. (done)
5. Implement new convection schemes, J. Hack's scheme for moist convection and Phil Rasch's scheme for convective transport. (in progress)
6. Run GEOS-CHEM with GEOS-4/fvDAS meteorology. Examine results, compare to observations and previous model simulations (yet to be done)
12 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM description
GEOSCHEM is currently 8 models in one…
4 x 5 GEOS-1
4 x 5 GEOS-STRAT
4 x 5 GEOS-3
2 x 2.5 GEOS-1
2 x 2.5 GEOS-STRAT
2 x 2.5 GEOS-3
1 x 1 GEOS-3 (nested grid over China)
1 x 1 GEOS-3 (nested grid over N. America)
… soon to be 12 models in one!
1 x 1 GEOS-3 (nested grid over Europe)
4 x 5 GEOS-4
2 x 2.5 GEOS-4
and (maybe not too soon) 1 x 1.25 GEOS-4
13 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM description
GEOS-CHEM can perform several different kinds of chemistry simulations, which are selectable at run time.
The main simulation is “full chemistry” (i.e. Ozone, etc.):
Chemistry solver: SMVGEAR package by Mark Jacobson
Photolysis code: FAST-J by O. Wild & M. Prather
300+ chemical reactions
50+ photolysis reactions
80+ chemical species
31 tracers (i.e. species which are transported)
• NOx, Ox, HC’s, DMS, SO2, SO4, MSA, NH3, NH4, Sulf. Nitrates
Heterogeneous chemistry on aerosol surfaces
Aerosol thermodynamic equlibrium (RPMARES)
14 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM description
In addition to the “full chemistry” simulation, GEOS-CHEM also can perform other chemistry simulations:
222 Rn – 210 Pb – 7 Be
Tagged CO (loss w/ archived OH)
Tagged Ox (with pre-saved P-L rates)
Methane
Ethane
Methyl Iodide
CO with parameterized OH (B. Duncan)
Offline sulfate aerosol
All of the “extra” chemistry simulations listed above come standard with GEOS-CHEM
However, some simulations may be in need of updating
Check w/ author of simulation before using it
15 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM description
Input and initialization
Dynamic loop
Read daily and monthly and various monthly quantities
Interpolate instantaneous (I-6) fields
Strat fluxes and Transport
PBL mixing and deep cloud convection
Diagnostics
Dry deposition
Emissions breakpoint
Chemistry breakpoint
Wet deposition
Save output to files and quit
16 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM description
GEOS-CHEM diagnostic output quantities include:
Tracer concentrations
OH, NO, NO2, HO2, fields
Photolysis rates (J-values)
Emissions
Cloud and aerosol optical depths
Chemical production and loss
Dry deposition fluxes and velocities
Methyl chloroform lifetime
Wet scavenging and wet deposition losses
Surface pressure, air mass, air density, etc.
GEOS-CHEM can produce average and timeseries output
You can use timeseries output to create 2-D or 3-D animations
17 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM description
Source code directory (e.g. Code.v5-04 )
Contains Makefiles and Fortran source code
Run directory (e.g. run.v5-04 )
Each user keeps run directories in his/her own disk space
Contains input files which must be modified by the user in order to select the parameters for a particular GEOS-CHEM run
GEOS-CHEM output files will be created in the run directory
Users can keep separate run directories for different simulations
Data directory (e.g. /data/ctm/GEOS_2x2.5
)
Located in a common disk space available to all users
Contains emissions, aerosol files, LAI, tropopause, etc, etc.
18 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
Std versions and benchmarks
Bob Yantosca currently maintains the standard GEOS-CHEM code at Harvard, using archive software (RCS, soon CVS?)
Procedure for adding new code into the standard model:
1.
User downloads a standard GEOS-CHEM version from Harvard
2.
User adds new code into their own local copy of GEOS-CHEM
3.
User debugs and thoroughly tests his/her local GEOS-CHEM
4.
User submits a list of his/her modifications to Bob Y.
5.
Bob Y. incorporates modifications into standard GEOS-CHEM
6.
Bob Y. benchmarks the new standard GEOS-CHEM version
7.
Daniel examines the new standard GEOS-CHEM version
8.
Upon Daniel’s blessing, Bob Y. releases GEOS-CHEM version
Single point of code modification
19 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
Std versions and benchmarks
Pros of single point of code modification :
One person is responsible for entire standard model. This prevents divergent versions from propagating among users.
Consistency is achieved in both coding style and naming files.
Documentation and revision history are consistently maintained.
Cons of single point of code modification
If point person is otherwise occupied, turnaround time for incorporating new revisions can be very long.
As the number of potential code developers grows, it is hard for one person to keep up with everyone’s submissions.
The challenge: to incorporate many revisions from multiple developers into GEOS-CHEM in a timely fashion while preventing the model from diverging.
20 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
Std versions and benchmarks
222 Rn — 210 Pb — 7 Be Benchmark
GEOS-3 meteorology w/ 48 layers @ 4x5
1 year: 1 Jan 2001 – 1 Jan 2002
Diagnostic quantities
Tracer ratios w/ previous version (surface, 500hPa)
Frequency Distribution of tracer ratios
Tracer concentrations (surface, 500hPa)
Tracer zonal means
Budgets for Rn, Pb, Be
Benchmark results are posted at Harvard and on the web site.
21 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
Std versions and benchmarks
Full chemistry benchmark
GEOS-3 meteorology w/ 30 layers @ 4x5
1 month: 1 Jul 2001 – 1 Aug 2001
Diagnostic quantities
Tracer ratios w/ previous version (surface, 500hPa)
Frequency distribution of tracer ratios
J-Value ratios w/ previous version (surface, 500hPa)
Tracer concentrations (surface, 500hPa) and zonal means
Budgets for Ox and CO; CH3CCl3 lifetime
Emissions table (comparison to previous version)
Benchmark results are posted at Harvard and on the web site.
22 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
Std versions and benchmarks
A TAR file w/ GEOS-CHEM source code and benchmark results is created for each new GEOS-CHEM version and is posted on the GEOS-CHEM website.
Benchmark results are compared to that of the previous version. We especially focus on the global budgets, emissions, tracer ratios, and the frequency distribution of tracer ratios.
Any major problems with a new GEOS-CHEM version should be evident from the benchmark simulation output.
A new GEOS-CHEM version is only released after Daniel approves the benchmark results for that version.
23 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
User support & documentation
GEOS-CHEM source code + benchmark output is available via website
GEOS-CHEM user packages
TESTRUN – script for compiling & running GEOS-CHEM jobs
(on website)
GEOS-CHEM documentation
GEOSCHEM Users’ Guide and FAQ (on website)
GEOS-CHEM Style Guide (on website)
Visualization packages
GAMAP (needs IDL; available from GAMAP website @ Harvard)
GRADS (free; see http://grads.iges.org/grads/grads.html)
Matlab (student versions available; check w/ your IT dept)
24 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
User support & documentation
How to download GEOS-CHEM source code and other related files from the web:
1.
Visit www-as.harvard.edu/chemistry/trop/geos/
2.
Click on Source Code & Data Files at the left of the page
3.
Enter the proper password (contact Bob Y.)
4.
Select the files that you wish to download
The following TAR files are available for download:
GEOS-CHEM source code w/ benchmark simulation output
2 x 2.5 GEOS-CHEM run directories
4 x 5 GEOS-CHEM run directories
2 x 2.5 GEOS-CHEM data directories
4 x 5 GEOS-CHEM data directories
25 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
User support & documentation
TESTRUN
Perl script for compiling & running GEOS-CHEM, located at: www-as.harvard.edu/chemistry/trop/geos/documentation/
What TESTRUN does
1.
Creates two scripts: one for compiling GEOS-CHEM and another for running GEOS-CHEM
2.
Submits compilation script to local batch queue system
3.
After compilation has completed, submits run script to local batch queue system. This starts the GEOS-CHEM run.
4.
Pipes GEOS-CHEM output to a timestamped log file
5.
Appends a timestamp to GEOS-CHEM output files
Use TESTRUN to make sure you don’t accidentally overwrite files that you wanted to save!
26 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
User support & documentation
GEOSCHEM Users’ Guide
Is the official GEOS-CHEM manual page, located at www-as.harvard.edu/chemistry/trop/geos/documentation/
The GEOSCHEM Users’ Guide covers the following topics:
1.
Introduction and brief overview of GEOS-CHEM
2.
Installing and compiling GEOS-CHEM
3.
Coding: practice and style
4.
Files contained in the GEOS-CHEM data directories
5.
Files contained in the GEOS-CHEM run directories
6.
Running and debugging GEOS-CHEM
7.
Vertical & horizontal resolution of GEOS-CHEM
8.
Met fields used in GEOS-CHEM
9.
GEOS-CHEM tracers, chemical species, and diagnostics
27 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
User support & documentation
GEOS-CHEM Frequently Asked Questions (FAQ)
Shortcuts to the proper section in the User’s Guide, located at www-as.harvard.edu/chemistry/trop/geos/documentation/
The GEOS-CHEM FAQ covers the following topics:
1.
What’s new in GEOS-CHEM?
2.
What kind of simulations can we run with GEOS-CHEM?
3.
What is the horizontal resolution of GEOS-CHEM?
4.
What is the vertical resolution of GEOS-CHEM?
5.
What are the met fields used by GEOS-CHEM?
6.
What diagnostic quantities are archived by GEOS-CHEM?
7.
Which chemical species are used in the GEOSCHEM “full chemistry” mechanism?
8.
Who is using GEOS-CHEM?
28 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
User support & documentation
GEOS-CHEM Style Guide
Describes GEOS-CHEM coding & documentation practices
See www-as.harvard.edu/chemistry/trop/documentation/
The GEOS-CHEM Style Guide covers the following topics:
1.
Background and brief introduction to Fortran 90
2.
Adding documentation headers, indentation, white space
3.
Numeric and character data types
4.
New language features of Fortran 90
5.
DO loops
6.
Fortran 90 modules
GEOS-CHEM Style Guide is continually being updated!
29 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
User support & documentation
GAMAP
Visualization package for GEOS-CHEM (and other models too!)
See www-as.harvard.edu/chemistry/trop/gamap/
Advantages of using GAMAP:
1.
Exploits IDL’s powerful mapping and image processing features
2.
Can read output from both GEOS-CHEM and GISS models
3.
Contains a user-friendly menu-driven interface
4.
GAMAP’s low-level routines can be called separately
5.
You can combine GAMAP’s mapping routines with IDL’s HDF,
HDF-EOS, and netCDF file I/O functions
6.
With the IDL ION package (sold separately!) you can develop an interactive web-page interface for GAMAP (cf. ITCT 2k2)
30 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
User support & documentation
We expect GEOS-CHEM developers and users to be resourceful in troubleshooting and debugging
Check input files and make sure code is debugged
Inform Bob Y. of any major bugs that you discover
Bob Y. can provide support for:
GEOS-CHEM: coding issues and problems
GAMAP: problems, bugs, and extensions
Bob Y. cannot provide support for:
General IDL or Fortran help (read the manual)
Bob Y. will “farm out” questions to other GEOS-CHEM users if he cannot answer them himself
31 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM platforms
As of June 2003, GEOS-CHEM has been ported to the following shared-memory architectures
SGI Power Challenge / Origin
Compaq Alpha
Linux (w/ Portland Group compiler)
Sun / Sparc
The standard code ships with Makefiles for each of the four supported platforms.
At this time, GEOS-CHEM is being recoded to run on distributed-memory architectures w/ help from NASA/NCCS
Hybrid “distributed-shared” memory Compaq Alpha at GSFC
32 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM platforms
Shared Memory: one “box” w/ many processors
Each processor can see all of the memory within the “box”
Array in memory System Memory
Proc 1 Proc 2 Proc 3 Proc 4
33 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM platforms
Distributed Memory: standalone “nodes” w/ network cnx
A processor can only see the memory on its own node
Mem 1 Mem 2 Mem 3
Proc 1 Proc 2
Proc 3a
Proc 3b
34 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM platforms
Hybrid “distributed-shared” memory (e.g. “Halem” at GSFC)
300+ nodes, with 4 processors and 2 GB memory per node
Halem is the 18 th fastest machine in the world (see top500.org)
2GB Memory per Node
Cnx. to other nodes
1 Node
Proc 1 Proc 2 Proc 3 Proc 4 Cnx. to other nodes
35 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM parallelization
GEOS-CHEM achieves parallelization on shared-memory machines using the OpenMP parallelization commands
OpenMP is an open standard for parallelization which is supported by many compiler vendors (see openmp.org)
SGI
Compaq / HP
Sun
IBM
Portland Group (Linux compiler)
NAG, etc.
OpenMP commands parallelize individual DO loops
36 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM parallelization
!$OMP PARALLEL DO
!$OMP+DEFAULT( SHARED )
!$OMP+PRIVATE( I, J )
DO J = 1, 1000 ! We have to do this
DO I = 1, 1000 ! a million times!
A(I,J) = A(I,J) ** 1.35 ! Exponentiation is
ENDDO ! very computationally
ENDDO ! intensive!
!$OMP END PARALLEL DO
The blue statements are OpenMP sentinels which tell the compiler that we want this DO loop to be executed in parallel
(I,J) pairs are split up among the available processors
37 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM parallelization
PRIVATE vs. SHARED
Each processor needs to keep a local copy of some variables
DO-loop indices (e.g. I, J) should be declared PRIVATE
Arrays themselves can be declared SHARED, as long as array indices (I,J) are declared PRIVATE
To optimize a parallel loop, make sure that it contains as much work as possible
Parallelize the outermost loop
Parallelize loops which require many iterations (such as looping over tracers, altitude, latitude, longitude)
Parallelize loops containing computationally expensive operations (e.g. divisions, logarithms, exponentiations, etc).
38 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM parallelization
Advantages of OpenMP
Commands are easy to use
Good scalability with increasing # of processors
Cross-platform support is guaranteed
Disadvantages of OpenMP
Assumes machine has a shared-memory architecture
Cannot use for distributed-memory machines, including:
• Beowulfs (i.e. PC’s hooked together running Linux)
•
Blade servers (IBM)
On hybrid “distributed-shared” machines (e.g. Halem), OpenMP can only be used on individual nodes
39 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM parallelization
GEOS-CHEM v5-04 fullchem, 1 day
30 levels, 4x5 grid
1 Processor
Compaq Alpha
@ GSFC
4 Processors
23.3 min / 1 day
(97% utilization)
8.5 min / 1 day
(330% utilization)
SunFire 3800
@ Dalhousie
47.2 min / 1 day
14.6 min / 1 day
(396% utilization)
Speedup 2.7 times 3.2 times
Going from 1 4 processors results in a speedup of approximately 3 times on Alpha and Sun
40 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM parallelization
OpenMP is not optimal for distributed memory machines, these require MPI (message passing interface) parallelization
GEOS-CHEM can run as-is on Halem, but only on 1 node
To use more than 1 node (4 processors) of Halem, we need to use MPI coding commands
For hybrid “distributed-shared” memory machines (e.g.
Halem), you can combine MPI and OpenMP:
Use MPI to distribute memory between nodes
Use OpenMP to operate among the processors of a single node
Tom Clune & Jack Yatteau will update us on the status of recoding GEOS-CHEM for MPI.
41 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
Effective coding strategies
Summary of good coding style
Include lots of comments (Bob Y. doesn’t know much chemistry!)
Add standard documentation headers to each routine
Use lots of white space – readability prevents bugs!!
Indent code – makes it easier for eye to follow!
Place new code into F90 modules
Use allocatable arrays (in modules) instead of COMMON blocks
Use new F90 language features whenever possible
See the GEOS-CHEM Style Guide for more details
42 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
Chemical forecasting
Past aircraft campaigns for which GEOS-CHEM was used to generate real-time forecasts
NASA TRACE-P (2001)
NOAA ITCT 2k2 (2002)
Future aircraft campaigns in which GEOS-CHEM will be used to generate real-time forecasts
NASA INTEX-NA (2004)
NOAA ITCT 2k4 (2004)
For TRACE-P and ITCT 2k2, GEOS-CHEM forecasts were generated on computers at Harvard
In the future we may run the GEOS-CHEM forecasts offsite
(maybe at GMAO???) We’re not sure yet.
43 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
Marketing strategies (?)
44
GEOS-CHEM license plates ????
(photo courtesy P. Palmer)
GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
Extra slides …
45 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GMAO met fields
Comparison of GEOS met fields up to 3 km
46 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GMAO met fields
Comparison of GEOS vertical levels to ~32 km
47 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GMAO met fields
1 x 1 nested grid: China
48 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GMAO met fields
1 x 1 nested grid: North America
49 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GMAO met fields
1 x 1 nested grid: Europe
50 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM description
New GEOS-CHEM code is written using F90 modules
Advantages of F90 modules:
Arrays and routines can be grouped together in a single “librarylike” unit. This allows for more convenient packaging.
Arrays and routines in modules can be referenced by other modules elsewhere in the code
Module arrays can be dynamically allocatable – better memory management
Internal variables and routines can be “hidden” from view to calling routines outside of the module – black box!
Modules can be used to centralize operations which may have previously been computed in several different places
Existing routines can be packaged into F90 modules as well
51 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM description
More about F90 modules
You can use PRIVATE declaration to hide module variables or routines from all other external subroutines or modules
You can select which variables or routines to import into another program via: USE MY_MODULE, ONLY : MY_SUB
If Module B references Module A, then Module A must be compiled before Module B
You can “overload” module routines with INTERFACE s so that they can accept different argument types
More references
A Programmer’s Guide to Fortran 90
GEOS-CHEM Style Guide (on website)
52 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM description
Dynamic timestep (transport, wetdep, archive diags)
10 min (1 x 1)
15 min (2 x 2.5)
30 min (4 x 5)
Convective timestep (PBL mixing, cloud convection)
Same as dynamic timestep
Emissions and dry deposition timestep
Once per hour
Photolysis and chemistry timestep
Once per hour
53 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
Std versions and benchmarks
GEOS-CHEM v5-04 is the last released standard version
Added nested-grid capability and cleaned up lots of old code
GEOS-CHEM v5-05-03 is the current development version
SMVGEAR II chemistry solver (cf. M. Jacobson)
•
Written by Mark Jacobson (then at UCLA)
• More recent version of our sparse matrix GEAR solver
New transport module (TPCORE) for GEOS-4/fvDAS winds
•
Written by S-J Lin (GSFC)
• Now with Mass Conservation! (from LLNL)
GEOS-CHEM v6-01 will be the first version to contain full support for fvDAS met fields (release date ????)
New cloud convection scheme (cf. P. Rasch) for fvDAS
54 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM description
MODULE MY_MODULE
IMPLICIT NONE ! Force declarations
REAL*8, ALLOCATABLE :: ARRAY(:,:) ! Allocatable array
CONTAINS
SUBROUTINE MY_SUB( X ) ! Subroutine example.
REAL*8, INTENT(IN) :: X ! Of course, GEOS-CHEM
ARRAY(:,:) = ARRAY(:,:) * X ! code is much more
END SUBROUTINE MY_SUB ! complex
SUBROUTINE INIT_MY_MODULE ! Subroutine to allocate
ALLOCATE( ARRAY( 100, 100 ) ) ! And initialize the array
ARRAY(:,:) = 0
END SUBROUTINE INIT_MY_MODULE
SUBROUTINE CLEANUP_MY_MODULE ! Subroutine to
IF ( ALLOCATED( ARRAY ) ) DEALLOCATE( ARRAY ) ! Deallocate the
END SUBROUTINE CLEANUP_MY_MODULE ! array
END MY_MODULE
55 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003
GEOS-CHEM description
PROGRAM MODULE_TEST
USE MY_MODULE ! References all arrays and
! routines in MY_MODULE
IMPLICIT NONE ! Force explicit declarations
CALL INIT_MY_MODULE ! Initializes and allocates
! module arrays
ARRAY(:,:) = 100.0d0 ! Assignment using array mask
CALL MY_SUB( 2.0d0 ) ! Multiplies ARRAY by 2.0
PRINT*, MINVAL( ARRAY ) ! and prints min & max
PRINT*, MAXVAL( ARRAY )
CALL CLEANUP_MY_MODULE ! Deallocates module arrays
END PROGRAM MODULE_TEST
56 GEOS-CHEM: Structure, Platforms, ...
Monday, June 2, 2003