NEMSIO LIBRARY
NMM tutorial 2015
1
Jun Wang
Hang Lei
Mark Iredell
INTRODUCTION
 nemsio
NMM tutorial 2015
is a package to support input/output in
NOAA Environmental Modeling System
(NEMS)
 Current
NEMSIO can handle binary data and
GRIB-1 data
 NEMSIO
has serial version and MPI version
2
DATA STRUCTURE


NMM tutorial 2015

A NEMSIO file contains two parts: meta data and
data fields.
Meta data:
 Standard model meta data
 User defined meta data (optional)
Data fields:
 Same dimension for all the fields
 Real kind (4 or 8 bytes real)
 Same format for all the fields
3
STANDARD META DATA



Total 12 meta data records, can be reduced by users
NMM tutorial 2015

required by NEMS model applications such as NMMB and GFS and
NCEP GSI
First 2 meta data are required in all cases:
 Gtype, gdatatype, modelname,version, nmeta
 Nrec, idate, forecast time, dimention, nframe, nsoil,
ntrac,jcap,ncldt,idsl,idvc,idvm,idrt,rlon_min,
rlon_max,rlat_min,rlat_max
10 other meta data records: recname, reclevtyp and reclev (meta
data 3:5), vcoord,lat,lon,dx,dy,Cpi,Ri
4
STANDARD META DATA 1
Name
Data type /value
gtype
Nemsio file flag
Character(8)
“nemsio
gdatatype
Data field type
Character(8)
“bin4”, “bin8”,”grib”
Modelname
Model name
character(8),
”GFS”,”NMMB”
version
Nemsio version
number
Integer.
“201010”
nmeta
Number of standard
meta
data records
Integer
<=12
NMM tutorial 2015
meaning
5
STANDARD META DATA 2
meaning
Data type
nrec
Number of data fields integer
idate
Initialization Date
integer
nfday,nfhour,
nfminute,
nsecondn,
nsecondd
Forecast day,
hour,minutes,
seconds
integer
dimx/dimy/dimz
Dimension in x/y/z
direction
integer
nframe
Halo Dimention
integer
Nsoil
number of soil layers
integer
NMM tutorial 2015
name
6
STANDARD META DATA 2 (CONTINUED)
meaning
Data type
ntrac
Number of tracers
integer
jcap
Spectral truncation
integer
ncld
number of cloud
types
integer
idsl
semi-lagrangian id
integer
idvc
vertical coordinate id
integer
idvm
Mass variable id
integer
idrt
grid identifier
integer
rlon_min,rlon_max,
rlat_min,rlat_max
min/max of domain
longitude /latitude
Real(4)
NMM tutorial 2015
name
7
USER DEFINED META DATA

NMM tutorial 2015

extrameta in second standard meta data indicates if there
are any user defined meta data
User defined mata data can have 5 data type:
integer, real(4), real(8), logical and character(16)

if extrameta is true, meta data record exists:
Nmetavari,nmetavarr,nmetavarl,nmetavarc,nmetavarr8
 Nmetaaryi,nmetaaryr,nmetaaryl,nmetaaryc,nmetaaryr8


User defined meta data records holds each meta data’s
name, length (for array) and value
8
USER DEFINED META DATA (CONT.)

User defined meta data
if(nmetavari>0)
NMM tutorial 2015
variname(nmetavari)
varival(nmetavari)
endif
If(nmetaaryr>0)
aryrname(nmetaaryr)
aryrlen(nmetaaryr)
do i=1,nmetaaryr
aryrval(1:aryrlen(i),i)
enddo
Endif
9

Every 1D array will only output its own length of data to the file
NEMSIO INTERFACES
–SERIAL VERSION
nemsio_init(iret)

nemsio_finalize(iret)

nemsio_open(gfile,gname,'read|write|rdwr',optarg,iret)

nemsio_close(gfile, iret)

nemsio_getfilehead(gfile,optargs,iret)

nemsio_getheadvar(gfile,argname,argval,iret)

nemsio_getrechead(gfile,jrec,vname,vlevtyp,vlev,iret)

nemsio_setfilehead(gfile,optarg,iret)

nemsio_readrec(gfile,jrec,data,nframe,iret)

nemsio_readrecv(gfile,name,levtyp,lev,data,nframe,iret)

nemsio_writerec(gfile,jrec,itr,zhour,data,iret)

nemsio_writerecv(gfile,name,levtyp,lev,itr,zhour,data,iret)
NMM tutorial 2015

10
NEMSIO PARALLEL VERSION
o Using MPI 2
o More efficient for file size ~GB
NMM tutorial 2015
11
NEMSIO INTERFACE
– PARALLEL VERSION
 nemsio_open(gfile,gfname,gaction,mpi_co
 nemsio_denseread(gfile,ista,iend,jsta,jend,
data,iret)
NMM tutorial 2015
mm,optargs,iret)
 nemsio_densewrite(gfile,ista,iend,jsta,jen
d,data,jrecs,jrece,iret)
12
EXAMPLE: NEMSIO PARALLEL READ
call nemsio_open(gfile,infile,'read',mpi_comm_comp,iret=ierr)
call nemsio_getfilehead(gfile,nrec=nrec,iret=ierr)
fldsize=(jte-jts+1)*(ite-its+1)
allocate(tmp((ite-its+1)*(jte-jts+1)*nrec),stat=i)
NMM tutorial 2015
call nemsio_init()
!
call nemsio_denseread(gfile,its,ite,jts,jte,tmp,iret=ierr)
!
call nemsio_close(gfile,iret=ierr)
13
UTILITIES FOR NEMSIO FILES



nemsio_get: get value of a specific meta data variable or a data
field
mknmmbctl: create .ctl file from a nemsio file for viewing the data
in Grads
NMM tutorial 2015

nemsio_read: read a nemsio file, list all the meta data information
and maxium/minium of each data field
nemsio_cvt: convert a nemsio file from big/little endian to
little/big endian
14
NEMSIO UTIL USUAGE
Nemsio_read file_name

Mknmmbctl file_name


file_name.ctl will be generated
Nemsio_get file_name meta_data_name|data
field name


Eg:
nemsio_get nmmb_hst_01_nio_0048h_00m_00.00s jm
jm=

NMM tutorial 2015

201
nemsio_get nmmb_hst_01_nio_0048h_00m_00.00s "spfh" "mid
layer" 1 |more
15
LOCATIONS FOR NEMSIO LIBRARY AND
UTILITIES

Nemsio repository

Wcoss:
/usrx/local/nceplibs/libnemsio_v2.2.1.a
 /global/save/emc.glopara/bin (?)


NMM tutorial 2015

https://svnemc.ncep.noaa.gov/projects/nceplibs/nemsio
Zeus:


/contrib/nceplibs/nwprod/lib
/contrib/nceplibs/nwprod/util
16
FUTURE WORK
With model resolution increasing, I/O could become
a bottleneck for numerical models.

I/O package needs to be scalable and efficient in
the petascale computing resources (BIG IO)



Parallel
Compression
NMM tutorial 2015

Nemsio near term plan:




Implement nemsio parallel version in the model
NetCDF
GRIB2
HDF
17
DOCUMENTATION
sio.php
NMM tutorial 2015
 http://www.emc.ncep.noaa.gov/NEMS/nem
18
NMM tutorial 2015
THANK YOU !
19