Research Tools Presentation
Matthew Janiga
10/30/2012

http://www.ncl.ucar.edu/index.shtml

http://www.unidata.ucar.edu/software/netcdf/

• NCL is a complete language with types, variables, operators, expressions, conditional statements, loops, and functions and procedures.
• NCL can read many data formats: NetCDF, HDF, GRIB, ascii and csv, binary, WRF output, and more.
– With some additional work it can read GEMPAK, McIDAS
AREA, and others.
• NCL like MATLAB is an array based language. Arrays can be sent to functions (100s and growing) which perform computations, plotting functions (dozens and growing), or files (typically NetCDF but also ascii and binary).

• NetCDF is a data format for storing arrays. Unlike ascii it also stores information describing the arrays.
– Attributes: strings or numbers storing: units, source locations, references, methodology, etc….
– Dimension coordinates: Arrays which describe the dimensions: lat, lon, time, level, etc. Dimensions coordinates can have their own attributes.
– NetCDF files are EASILY examined using GUIs (NCVIEW, IDV, Panopoly, etc.) or command line programs (ncdump, ncl_filedump).
– NetCDF files can also be created, read, or manipulated by functions in libraries written for Fortran, C, Java, and more!
• How difficult is say… compiling a Fortran program that uses NetCDF?
– Just type "nc-config --libs“ on the command line and all flags will be given to you!
• NetCDF is also fairly easy to read into programs other than NCL: R,
MATLAB, etc.
– http://www.r-project.org/

The NetCDF Java is a jar file opened from the web that can convert many formats to NetCDF.
http://www.unidata.ucar.edu/downloads/netcdf/netcdf-java-4/index.jsp
An efficient command line program for merging, splicing, and otherwise editing NetCDF files. I’ve found NCL to be much easier to use and powerful but this is another option.
http://nco.sourceforge.net
/

NCVIEW is one of MANY programs that do this.
http://www.unidata.ucar.edu/softwar e/netcdf/software.html
NCVIEW is a simple GUI that lets you explore a NetCDF file: examine slices, time series, animate, and more. Look at the data you’ve generated before spending time to make a plot!

• Free, stable, and extremely well documented.
• NCL is a fully fledged programming language not calls to compiled programs.
Code yourself don’t “trick”.
• NCL allows you to read in large amounts of data into memory (64-bit vars >
2GB) and perform computations there instead of constantly reading and writing to disk.
• Creating your own procedures to call externally is exactly the same as writing a script (no Fortran or shell scripting necessary).
• Handles many vertical and horizontal projections and conversions from one to the other.
• NCL is very flexible, it can analyze any array of any dimension, type, with any coordinate description.
– Store and process: trajectories, tracks, hovmollers, time series, polar orbit swaths and other non-uniform projections, radar data, geostationary satellite data, surface obs, profiler data, radiosondes, and pretty much anything you can think of.

• NCL has a bit of a learning curve. Learning how to fully utilize it and code using arrays takes time.
• For some plots (plan plots and cross-sections) GEMPAK may be able to produce a similar plot with less code (assuming data is already in
GEMPAK format).
• GEMPAK is still heavily used in operations.
• While there are lots of mathematical functions and procedures
MATLAB has more types and variants.
• Similarly R has more statistical tools.

And more…..

NCL for EMACS
• OPeNDAP: The ability to grab what you want out of datasets directly from a server.
Makes some local storage of data obsolete.
– Available for many datasets and reanalyses.
• system and systemfunc
– avail_files = systemfunc("ls -1")
– time = systemfunc("date +'%s'")
– system(“convert img.ps img.png &”)
• Use WRAPIT to send information to and from pre-compiled user-created or commercial Fortran or C programs
– http://www.ncl.ucar.edu/Document/To ols/WRAPIT.shtml
http://www.ncl.ucar.edu/Applications/editor.shtml

• ParNCL - Parallel version of NCL for analyzing large datasets.
• Earth System Modeling Framework (ESMF) regridding software.
• NetCDF 4 – Ability to store and read groups
“object-orientated variables”.
• MJO CLIVAR Diagnostics – MATLAB still more flexible.

Documentation! The NCL webpage has 100s of examples each containing a picture of the plot and the code required to make it.
http://www.ncl.ucar.edu/Applications/

Every function has an exhaustive documentation page showing how to use it, describing how the calculation is performed, and providing examples.

Perform the calculation
Libraries: NCLs and your own.
Read data