GEOS-Chem structure, documentation, platforms, and parallelization

advertisement

GEOS-CHEM: Structure,

Documentation, Platforms, and Parallelization

Bob Yantosca

Software Engineer

Atmospheric Chemistry Modeling Group

Harvard University

GEOS-

CHEM Users’ Meeting

02 June 2003

Topics

1. GEOS-CHEM history

2. GMAO met fields used to drive GEOS-CHEM

3. GEOS-CHEM model description

4. GEOS-CHEM standard versions and benchmarks

5. GEOS-CHEM user support and documentation

6. GEOS-CHEM platforms

7. Parallelization: OpenMP and MPI

8. Effective coding practices

9. Chemical forecasting

2 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

In the beginning …

GEOS-CHEM was originally created from:

The old Harvard/GISS-II 9-layer model (Larry, Yuhang, et al)

GEOS-CTM (Dale, Mian)

– ancestor of GOCART

Pieces that came from the Harvard/GISS-II model:

Emissions (EMISSDR, BIOBURN, etc)

Dry Deposition (DRYDEP, DEPVEL, etc.)

Chemistry (SMVGEAR etc.)

Diagnostics (NDxx) and various input files

Pieces that came from the GEOS-CTM model:

Transport (TPCORE, by Lin & Rood)

BL mixing (TURBDAY) and Cloud Convection (NFCLDMX)

3 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

In the beginning …

GEOS-CHEM evolution:

We did not foresee all of the potential uses of GEOS-CHEM at the time we first created GEOS-CHEM.

At first, newer code was “onion-skinned” on top of older code.

This resulted in functional, albeit, convoluted code.

However, a number of recent developments (new met fields, mechanisms, etc.) has forced us to make some widespread changes to the GEOS-CHEM source code.

These problems are not unique to GEOS-CHEM, but also apply to any software product.

4 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

In the beginning …

Philosophical Digression:

 “If builders built buildings the way programmers write programs, then the first woodpecker that came along would destroy civilization.” – Weinberg’s Law

“Inside every large program there is a small program struggling to get out.” – Hoare’s Law of Large Programs

Interpretation:

If you want to build a new building, you tear down the old one first and start from scratch.

With software, usually you start with an existing program and add new stuff on top of that. Your foundations may be shaky…

5 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

In the beginning …

Much of the existing GEOS-CHEM code has been recently updated, or brand-new code has been added

Biomass and biofuel emissions

Dry and wet deposition

Date/time functions

Error handling and file I/O

Sulfate chemistry

Main program

Some sections of GEOS-CHEM still need improvement

Diagnostics (especially timeseries)

Input file reading

Emissions for full chemistry (incl. Isoprene, Soil NOx)

6 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GMAO met fields

GEOS-CHEM is driven by assimilated met data from

NASA/GMAO (formerly DAO), located at GSFC.

GMAO has produced 4 major met data sets:

GEOS-1

GEOS-STRAT

GEOS-3

GEOS-4/fvDAS

(1985

– 1995, 20 layers to 10 hPa)

(1995

– 1997, 46 layers to 0.1 hPa)

(2000

– 2002, 48 layers to 0.01 hPa

(2003

–, 55 layers to 0.01 hPa)

At present, GEOS-CHEM can run with GEOS-1, GEOS-

STRAT, and GEOS-3 met fields.

GEOS-4 support is currently being added to GEOS-CHEM.

7 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GMAO met fields

Native horizontal resolution of GMAO met fields:

GEOS-1

GEOS-STRAT

GEOS-3

GEOS-4/fvDAS

2 o

2 o lat x 2.5

o lon lat x 2.5

o lon

1 o

1 o lat x 1 o lon lat x 1.25

o lon

We also can regrid met fields to coarser resolution in order to gain computational advantage for long simulations

GEOS-1 4 o lat x 5 o lon

GEOS-STRAT 4 o lat x 5 o lon

GEOS-3

GEOS-4/fvDAS

2 o lat x 2.5

o

2 o lat x 2.5

o lon AND 4 o lon AND 4 o lat x 5 o lat x 5 o lon lon

Regridding of met fields is done offline, and can be automated

8 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GMAO met fields

I-6 fields (6hr instantaneous “snapshot” quantities)

U, V, P, T, Specific humidity, albedo

A-6 fields (6-hr averaged quantities)

Cld. Frac.

– random & max overlap (GEOS-1, GEOS-S)

In-cloud optical depth and total cld frac (GEOS-3)

Tendency in specific humidity (i.e. precip or evap)

Cloud mass flux and detrainment

A-3 fields (3-hr averaged quantities)

10m winds, roughness height, friction velocity

PBL height, column cloud fraction, surface temperature

Convective & total precipitation at ground

9 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GMAO met fields

A-6 fields

A-3 fields

Times of day when you need to read met fields

Z = “Zulu” = abbreviation for GMT

GEOS-1

GEOS-S

GEOS-3

A-6 fields

A-3 fields

From the fvDAS file spec on GMAO’s website

10

GEOS-4

Start of day

GEOS-CHEM: Structure, Platforms, ...

Start of next day

Monday, June 2, 2003

GMAO met fields

GEOS-1, GEOS-STRAT, GEOS-3

A-6 fields are stamped w/ center times: 00, 06, 12, 18 GMT (read them at 21, 03, 09, 15 GMT)

A-

3 fields are stamped w/ end times: 00, 03, 06, … 21 GMT

GEOS-4 / fvDAS

A-6 fields are stamped w/ center times: 03, 09, 15, 21 GMT (read them at 00, 06, 12, 18 GMT )

A-

3 fields are stamped w/ center times: 1:30, 4:30, 7:30, … 22:30

GEOS-4 timestamps and averaging periods differ from

GEOS-3, GEOS-1, and GEOS-STRAT!

11 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GMAO met fields

Preparing GEOS-CHEM for GEOS-4/fvDAS met fields:

1. Download, regrid, and format fvDAS met fields.

(in progress)

2. Adapt code to read in fvDAS met fields w/ different timestamps and averaging intervals.

(in progress)

3. Implement new TPCORE transport code for fvDAS. This contains new semi-lagrangian vertical advection.

(done)

4. Fit Philip Cameron-Smith's pressure fixer to the fvDAS TPCORE transport scheme. Add additional mixing ratio adjustment for residual mass difference to ensure mass conservation.

(done)

5. Implement new convection schemes, J. Hack's scheme for moist convection and Phil Rasch's scheme for convective transport.

(in progress)

6. Run GEOS-CHEM with GEOS-4/fvDAS meteorology. Examine results, compare to observations and previous model simulations

(yet to be done)

12 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM description

GEOS-

CHEM is currently 8 models in one…

4 x 5 GEOS-1

4 x 5 GEOS-STRAT

4 x 5 GEOS-3

2 x 2.5 GEOS-1

2 x 2.5 GEOS-STRAT

2 x 2.5 GEOS-3

1 x 1 GEOS-3 (nested grid over China)

1 x 1 GEOS-3 (nested grid over N. America)

… soon to be 12 models in one!

1 x 1 GEOS-3 (nested grid over Europe)

4 x 5 GEOS-4

2 x 2.5 GEOS-4 and (maybe not too soon) 1 x 1.25 GEOS-4

13 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM description

GEOS-CHEM can perform several different kinds of chemistry simulations, which are selectable at run time.

The main simulation is “full chemistry” (i.e. Ozone, etc.):

Chemistry solver: SMVGEAR package by Mark Jacobson

Photolysis code: FAST-J by O. Wild & M. Prather

300+ chemical reactions

50+ photolysis reactions

80+ chemical species

31 tracers (i.e. species which are transported)

• NOx, Ox, HC’s, DMS, SO2, SO4, MSA, NH3, NH4, Sulf. Nitrates

Heterogeneous chemistry on aerosol surfaces

Aerosol thermodynamic equlibrium (RPMARES)

14 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM description

In addition to the “full chemistry” simulation, GEOS-CHEM also can perform other chemistry simulations:

222

Rn

210

Pb

7

Be

Tagged CO (loss w/ archived OH)

Tagged Ox (with pre-saved P-L rates)

Methane

Ethane

Methyl Iodide

CO with parameterized OH (B. Duncan)

Offline sulfate aerosol

All of the “extra” chemistry simulations listed above come standard with GEOS-CHEM

However, some simulations may be in need of updating

Check w/ author of simulation before using it

15 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM description

GEOS-CHEM source code structure

Input and initialization

Dynamic loop

Read daily and monthly and various monthly quantities

Interpolate instantaneous (I-6) fields

Strat fluxes and Transport

PBL mixing and deep cloud convection

Diagnostics

Dry deposition

Emissions breakpoint

Chemistry breakpoint

Wet deposition

Save output to files and quit

16 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM description

GEOS-CHEM diagnostic output quantities include:

Tracer concentrations

OH, NO, NO2, HO2, fields

Photolysis rates (J-values)

Emissions

Cloud and aerosol optical depths

Chemical production and loss

Dry deposition fluxes and velocities

Methyl chloroform lifetime

Wet scavenging and wet deposition losses

Surface pressure, air mass, air density, etc.

GEOS-CHEM can produce average and timeseries output

You can use timeseries output to create 2-D or 3-D animations

17 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM description

GEOS-CHEM directory structure

Source code directory (e.g.

Code.v5-04

)

Contains Makefiles and Fortran source code

Run directory (e.g.

run.v5-04

)

Each user keeps run directories in his/her own disk space

Contains input files which must be modified by the user in order to select the parameters for a particular GEOS-CHEM run

GEOS-CHEM output files will be created in the run directory

Users can keep separate run directories for different simulations

Data directory (e.g.

/data/ctm/GEOS_2x2.5

)

Located in a common disk space available to all users

Contains emissions, aerosol files, LAI, tropopause, etc, etc.

18 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

Std versions and benchmarks

Bob Yantosca currently maintains the standard GEOS-CHEM code at Harvard, using archive software (RCS, soon CVS?)

1.

2.

3.

4.

5.

6.

7.

8.

Procedure for adding new code into the standard model:

User downloads a standard GEOS-CHEM version from Harvard

User adds new code into their own local copy of GEOS-CHEM

User debugs and thoroughly tests his/her local GEOS-CHEM

User submits a list of his/her modifications to Bob Y.

Bob Y. incorporates modifications into standard GEOS-CHEM

Bob Y. benchmarks the new standard GEOS-CHEM version

Daniel examines the new standard GEOS-CHEM version

Upon Daniel’s blessing, Bob Y. releases GEOS-CHEM version

Single point of code modification

19 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

Std versions and benchmarks

Pros of single point of code modification :

One person is responsible for entire standard model. This prevents divergent versions from propagating among users.

Consistency is achieved in both coding style and naming files.

Documentation and revision history are consistently maintained.

Cons of single point of code modification

If point person is otherwise occupied, turnaround time for incorporating new revisions can be very long.

As the number of potential code developers grows, it is hard for one person to keep up with everyone’s submissions.

The challenge: to incorporate many revisions from multiple developers into GEOS-CHEM in a timely fashion while preventing the model from diverging.

20 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

Std versions and benchmarks

222

Rn

210

Pb

7

Be Benchmark

GEOS-3 meteorology w/ 48 layers @ 4x5

1 year: 1 Jan 2001

– 1 Jan 2002

Diagnostic quantities

Tracer ratios w/ previous version (surface, 500hPa)

Frequency Distribution of tracer ratios

Tracer concentrations (surface, 500hPa)

Tracer zonal means

Budgets for Rn, Pb, Be

Benchmark results are posted at Harvard and on the web site.

21 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

Std versions and benchmarks

Full chemistry benchmark

GEOS-3 meteorology w/ 30 layers @ 4x5

1 month: 1 Jul 2001

– 1 Aug 2001

Diagnostic quantities

Tracer ratios w/ previous version (surface, 500hPa)

Frequency distribution of tracer ratios

J-Value ratios w/ previous version (surface, 500hPa)

Tracer concentrations (surface, 500hPa) and zonal means

Budgets for Ox and CO; CH3CCl3 lifetime

Emissions table (comparison to previous version)

Benchmark results are posted at Harvard and on the web site.

22 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

Std versions and benchmarks

A TAR file w/ GEOS-CHEM source code and benchmark results is created for each new GEOS-CHEM version and is posted on the GEOS-CHEM website.

Benchmark results are compared to that of the previous version. We especially focus on the global budgets, emissions, tracer ratios, and the frequency distribution of tracer ratios.

Any major problems with a new GEOS-CHEM version should be evident from the benchmark simulation output.

A new GEOS-CHEM version is only released after Daniel approves the benchmark results for that version.

23 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

User support & documentation

GEOS-CHEM source code + benchmark output is available via website

GEOS-CHEM user packages

TESTRUN

– script for compiling & running GEOS-CHEM jobs

(on website)

GEOS-CHEM documentation

GEOS-

CHEM Users’ Guide and FAQ (on website)

GEOS-CHEM Style Guide (on website)

Visualization packages

GAMAP (needs IDL; available from GAMAP website @ Harvard)

GRADS (free; see http://grads.iges.org/grads/grads.html)

Matlab (student versions available; check w/ your IT dept)

24 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

User support & documentation

How to download GEOS-CHEM source code and other related files from the web:

1.

Visit

www-as.harvard.edu/chemistry/trop/geos/

2.

3.

4.

Click on

Source Code & Data Files

at the left of the page

Enter the proper password (contact Bob Y.)

Select the files that you wish to download

The following TAR files are available for download:

GEOS-CHEM source code w/ benchmark simulation output

2 x 2.5 GEOS-CHEM run directories

4 x 5 GEOS-CHEM run directories

2 x 2.5 GEOS-CHEM data directories

4 x 5 GEOS-CHEM data directories

25 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

User support & documentation

TESTRUN

Perl script for compiling & running GEOS-CHEM, located at:

www-as.harvard.edu/chemistry/trop/geos/documentation/

What TESTRUN does

1.

2.

3.

4.

5.

Creates two scripts: one for compiling GEOS-CHEM and another for running GEOS-CHEM

Submits compilation script to local batch queue system

After compilation has completed, submits run script to local batch queue system. This starts the GEOS-CHEM run.

Pipes GEOS-CHEM output to a timestamped log file

Appends a timestamp to GEOS-CHEM output files

Use TESTRUN to make sure you don’t accidentally overwrite files that you wanted to save!

26 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

User support & documentation

GEOS-

CHEM Users’ Guide

Is the official GEOS-CHEM manual page, located at

www-as.harvard.edu/chemistry/trop/geos/documentation/

The GEOS-

CHEM Users’ Guide covers the following topics:

1.

Introduction and brief overview of GEOS-CHEM

2.

3.

4.

5.

6.

7.

8.

9.

Installing and compiling GEOS-CHEM

Coding: practice and style

Files contained in the GEOS-CHEM data directories

Files contained in the GEOS-CHEM run directories

Running and debugging GEOS-CHEM

Vertical & horizontal resolution of GEOS-CHEM

Met fields used in GEOS-CHEM

GEOS-CHEM tracers, chemical species, and diagnostics

27 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

User support & documentation

GEOS-CHEM Frequently Asked Questions (FAQ)

 Shortcuts to the proper section in the User’s Guide, located at

www-as.harvard.edu/chemistry/trop/geos/documentation/

The GEOS-CHEM FAQ covers the following topics:

1.

What’s new in GEOS-CHEM?

2.

3.

4.

5.

6.

7.

8.

What kind of simulations can we run with GEOS-CHEM?

What is the horizontal resolution of GEOS-CHEM?

What is the vertical resolution of GEOS-CHEM?

What are the met fields used by GEOS-CHEM?

What diagnostic quantities are archived by GEOS-CHEM?

Which chemical species are used in the GEOS-

CHEM “full chemistry” mechanism?

Who is using GEOS-CHEM?

28 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

User support & documentation

GEOS-CHEM Style Guide

Describes GEOS-CHEM coding & documentation practices

See

www-as.harvard.edu/chemistry/trop/documentation/

The GEOS-CHEM Style Guide covers the following topics:

1.

Background and brief introduction to Fortran 90

2.

3.

4.

5.

6.

Adding documentation headers, indentation, white space

Numeric and character data types

New language features of Fortran 90

DO loops

Fortran 90 modules

GEOS-CHEM Style Guide is continually being updated!

29 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

User support & documentation

GAMAP

Visualization package for GEOS-CHEM (and other models too!)

See

www-as.harvard.edu/chemistry/trop/gamap/

2.

3.

4.

5.

Advantages of using GAMAP:

1.

Exploits IDL’s powerful mapping and image processing features

6.

Can read output from both GEOS-CHEM and GISS models

Contains a user-friendly menu-driven interface

GAMAP’s low-level routines can be called separately

You can combine GAMAP’s mapping routines with IDL’s HDF,

HDF-EOS, and netCDF file I/O functions

With the IDL ION package (sold separately!) you can develop an interactive web-page interface for GAMAP (cf. ITCT 2k2)

30 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

User support & documentation

We expect GEOS-CHEM developers and users to be resourceful in troubleshooting and debugging

Check input files and make sure code is debugged

Inform Bob Y. of any major bugs that you discover

Bob Y. can provide support for:

GEOS-CHEM: coding issues and problems

GAMAP: problems, bugs, and extensions

Bob Y. cannot provide support for:

General IDL or Fortran help (read the manual)

Bob Y. will “farm out” questions to other GEOS-CHEM users if he cannot answer them himself

31 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM platforms

As of June 2003, GEOS-CHEM has been ported to the following shared-memory architectures

SGI Power Challenge / Origin

Compaq Alpha

Linux (w/ Portland Group compiler)

Sun / Sparc

The standard code ships with Makefiles for each of the four supported platforms.

At this time, GEOS-CHEM is being recoded to run on distributed-memory architectures w/ help from NASA/NCCS

 Hybrid “distributed-shared” memory Compaq Alpha at GSFC

32 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM platforms

Shared Memory: one “box” w/ many processors

 Each processor can see all of the memory within the “box”

Array in memory System Memory

Proc 1 Proc 2 Proc 3 Proc 4

33 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM platforms

Distributed Memory: standalone “nodes” w/ network cnx

A processor can only see the memory on its own node

Mem 1 Mem 2 Mem 3

Proc 1 Proc 2

Proc 3a

Proc 3b

34 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM platforms

Hybrid “distributed-shared” memory (e.g. “Halem” at GSFC)

300+ nodes, with 4 processors and 2 GB memory per node

Halem is the 18 th fastest machine in the world (see top500.org)

2GB Memory per Node

Cnx. to other nodes

1 Node

Proc 1 Proc 2 Proc 3 Proc 4

Cnx. to other nodes

35 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM parallelization

GEOS-CHEM achieves parallelization on shared-memory machines using the OpenMP parallelization commands

OpenMP is an open standard for parallelization which is supported by many compiler vendors (see openmp.org)

SGI

Compaq / HP

Sun

IBM

Portland Group (Linux compiler)

NAG, etc.

OpenMP commands parallelize individual DO loops

36 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM parallelization

!$OMP PARALLEL DO

!$OMP+DEFAULT( SHARED )

!$OMP+PRIVATE( I, J )

DO J = 1, 1000 ! We have to do this

DO I = 1, 1000 ! a million times!

A(I,J) = A(I,J) ** 1.35 ! Exponentiation is

ENDDO ! very computationally

ENDDO ! intensive!

!$OMP END PARALLEL DO

The blue statements are OpenMP sentinels which tell the compiler that we want this DO loop to be executed in parallel

(I,J) pairs are split up among the available processors

37 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM parallelization

PRIVATE vs. SHARED

Each processor needs to keep a local copy of some variables

DO-loop indices (e.g. I, J) should be declared PRIVATE

Arrays themselves can be declared SHARED, as long as array indices (I,J) are declared PRIVATE

To optimize a parallel loop, make sure that it contains as much work as possible

Parallelize the outermost loop

Parallelize loops which require many iterations (such as looping over tracers, altitude, latitude, longitude)

Parallelize loops containing computationally expensive operations (e.g. divisions, logarithms, exponentiations, etc).

38 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM parallelization

Advantages of OpenMP

Commands are easy to use

Good scalability with increasing # of processors

Cross-platform support is guaranteed

Disadvantages of OpenMP

Assumes machine has a shared-memory architecture

Cannot use for distributed-memory machines, including:

• Beowulfs (i.e. PC’s hooked together running Linux)

Blade servers (IBM)

On hybrid “distributed-shared” machines (e.g. Halem), OpenMP can only be used on individual nodes

39 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM parallelization

GEOS-CHEM v5-04 fullchem, 1 day

30 levels, 4x5 grid

1 Processor

Compaq Alpha

@ GSFC

4 Processors

23.3 min / 1 day

(97% utilization)

8.5 min / 1 day

(330% utilization)

SunFire 3800

@ Dalhousie

47.2 min / 1 day

14.6 min / 1 day

(396% utilization)

Speedup 2.7 times 3.2 times

Going from 1

4 processors results in a speedup of approximately 3 times on Alpha and Sun

40 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM parallelization

OpenMP is not optimal for distributed memory machines, these require MPI (message passing interface) parallelization

GEOS-CHEM can run as-is on Halem, but only on 1 node

To use more than 1 node (4 processors) of Halem, we need to use MPI coding commands

For hybrid “distributed-shared” memory machines (e.g.

Halem), you can combine MPI and OpenMP:

Use MPI to distribute memory between nodes

Use OpenMP to operate among the processors of a single node

Tom Clune & Jack Yatteau will update us on the status of recoding GEOS-CHEM for MPI.

41 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

Effective coding strategies

Summary of good coding style

Include lots of comments (Bob Y. doesn’t know much chemistry!)

Add standard documentation headers to each routine

Use lots of white space

– readability prevents bugs!!

Indent code

– makes it easier for eye to follow!

Place new code into F90 modules

Use allocatable arrays (in modules) instead of COMMON blocks

Use new F90 language features whenever possible

See the GEOS-CHEM Style Guide for more details

42 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

Chemical forecasting

Past aircraft campaigns for which GEOS-CHEM was used to generate real-time forecasts

NASA TRACE-P (2001)

NOAA ITCT 2k2 (2002)

Future aircraft campaigns in which GEOS-CHEM will be used to generate real-time forecasts

NASA INTEX-NA (2004)

NOAA ITCT 2k4 (2004)

For TRACE-P and ITCT 2k2, GEOS-CHEM forecasts were generated on computers at Harvard

In the future we may run the GEOS-CHEM forecasts offsite

(maybe at GMAO???) We’re not sure yet.

43 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

Marketing strategies (?)

44

GEOS-CHEM license plates ????

(photo courtesy P. Palmer)

GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

Extra slides …

45 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GMAO met fields

Comparison of GEOS met fields up to 3 km

46 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GMAO met fields

Comparison of GEOS vertical levels to ~32 km

47 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GMAO met fields

1 x 1 nested grid: China

48 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GMAO met fields

1 x 1 nested grid: North America

49 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GMAO met fields

1 x 1 nested grid: Europe

50 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM description

New GEOS-CHEM code is written using F90 modules

Advantages of F90 modules:

 Arrays and routines can be grouped together in a single “librarylike” unit. This allows for more convenient packaging.

Arrays and routines in modules can be referenced by other modules elsewhere in the code

Module arrays can be dynamically allocatable

– better memory management

Internal variables and routines can be “hidden” from view to calling routines outside of the module

– black box!

Modules can be used to centralize operations which may have previously been computed in several different places

Existing routines can be packaged into F90 modules as well

51 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM description

More about F90 modules

You can use

PRIVATE

declaration to hide module variables or routines from all other external subroutines or modules

You can select which variables or routines to import into another program via:

USE MY_MODULE, ONLY : MY_SUB

If Module B references Module A, then Module A must be compiled before Module B

You can “overload” module routines with

INTERFACE

s so that they can accept different argument types

More references

A Programmer’s Guide to Fortran 90

GEOS-CHEM Style Guide (on website)

52 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM description

GEOS-CHEM timesteps

Dynamic timestep (transport, wetdep, archive diags)

10 min (1 x 1)

15 min (2 x 2.5)

30 min (4 x 5)

Convective timestep (PBL mixing, cloud convection)

Same as dynamic timestep

Emissions and dry deposition timestep

Once per hour

Photolysis and chemistry timestep

Once per hour

53 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

Std versions and benchmarks

GEOS-CHEM v5-04 is the last released standard version

Added nested-grid capability and cleaned up lots of old code

GEOS-CHEM v5-05-03 is the current development version

SMVGEAR II chemistry solver (cf. M. Jacobson)

Written by Mark Jacobson (then at UCLA)

More recent version of our sparse matrix GEAR solver

New transport module (TPCORE) for GEOS-4/fvDAS winds

Written by S-J Lin (GSFC)

Now with Mass Conservation! (from LLNL)

GEOS-CHEM v6-01 will be the first version to contain full support for fvDAS met fields (release date ????)

New cloud convection scheme (cf. P. Rasch) for fvDAS

54 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM description

MODULE MY_MODULE

IMPLICIT NONE ! Force declarations

REAL*8, ALLOCATABLE :: ARRAY(:,:) ! Allocatable array

CONTAINS

SUBROUTINE MY_SUB( X ) ! Subroutine example.

REAL*8, INTENT(IN) :: X ! Of course, GEOS-CHEM

ARRAY(:,:) = ARRAY(:,:) * X ! code is much more

END SUBROUTINE MY_SUB ! complex

SUBROUTINE INIT_MY_MODULE ! Subroutine to allocate

ALLOCATE( ARRAY( 100, 100 ) ) ! And initialize the array

ARRAY(:,:) = 0

END SUBROUTINE INIT_MY_MODULE

SUBROUTINE CLEANUP_MY_MODULE ! Subroutine to

IF ( ALLOCATED( ARRAY ) ) DEALLOCATE( ARRAY ) ! Deallocate the

END SUBROUTINE CLEANUP_MY_MODULE ! array

END MY_MODULE

55 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

GEOS-CHEM description

PROGRAM MODULE_TEST

USE MY_MODULE ! References all arrays and

! routines in MY_MODULE

IMPLICIT NONE ! Force explicit declarations

CALL INIT_MY_MODULE ! Initializes and allocates

! module arrays

ARRAY(:,:) = 100.0d0 ! Assignment using array mask

CALL MY_SUB( 2.0d0 ) ! Multiplies ARRAY by 2.0

PRINT*, MINVAL( ARRAY ) ! and prints min & max

PRINT*, MAXVAL( ARRAY )

CALL CLEANUP_MY_MODULE ! Deallocates module arrays

END PROGRAM MODULE_TEST

56 GEOS-CHEM: Structure, Platforms, ...

Monday, June 2, 2003

Download