The Next Generation McIDAS
A Look Towards the Future
21st IIPS 8.2 San Diego, CA - 11 January 2005
Dave Santek
Tom Whittaker
Thomas Achtor
Space Science &
Engineering Center
(SSEC) ,
University of
Wisconsin - Madison
Outline
• Basic functionality of McIDAS-X
• History of McIDAS and current user
base
• The next generation: McIDAS-V
– VisAD
– IDV (Integrated Data Viewer)
• Data servers: ADDE and OpenADDE
What is McIDAS ?
(Man computer Interactive
Data Access System)
• Collection of user programs and libraries
for visualizing and analyzing geophysical data (focus on
environmental satellites)
– UNIX, PC & Mac capable
• A synergistic tool that
integrates numerous data
types into one system
• First developed in the
early 1970s
• Still in use world-wide at
research, operational,
educational, and
commercial sites
Key McIDAS Attributes
• Access to extensive geophysical database
• Core package plus user-written
applications
• Diverse functionality through software
(1 million + lines of code)
• Extensive visualization capabilities
• Satellite and NOAAPORT data ingest
McIDAS Functionality
• Digital Image Processing
• GIS Applications
• Weather and Climate Data Analysis and
Applications
• Graphical Displays of Data & Information
• Gridded Data Processing and Analysis
Tools
• Display & Process Control Utilities
1972
First Generation of
McIDAS
Runs on Harris /5 with
96 KB of programmable
memory and 2-5 MB
hard drives
1970
1991
McIDAS runs on
1997
UNIX
McIDAS Users’
workstations
1983
Group sunsets
People’s Republic of China
support for
funds port of McIDAS
mainframe
software to IBM mainframe
McIDAS
1980
1976
GOES ingest
system added to
McIDAS
1968
WINDCO
generates wind
vectors from ATS
images
1990
1985
McIDAS runs on
PCs under DOS
operating system
1979
Wichita, OK tornado
disaster: Congress directs
Operational McIDAS
system to be installed at
the National Severe Storms
Forecast Center
2000
1994
Satellite and conventional
data is served from UNIX
workstations, beginning the
use of ADDE (Abstract Data
Distribution Environment)
1987
McIDAS runs on
PCs under OS/2
operating system
and McIDAS Users’
Group is formed
McIDAS Users
•
•
•
•
NOAA – NESDIS, AWC, SPC, TPC, etc.
NASA – STS, LaRC, MSFC, JPL
Unidata – 130 universities and colleges
International – EUMETSAT, ECMWF,
Spain, Greece, Mexico, Australia
• Industry – Boeing, Honeywell,
Weathernews, Universal Weather,
Meteorlogix, etc.
Why Change?
• McIDAS software (written in Fortran 77
and C) has a 30 year heritage resulting in
limited extensibility potential
• New visualization concepts cannot be
incorporated (4D)
• Forthcoming environmental satellite data
cannot be utilized in the best way
(GOES-R & NPOESS operational
systems)
ABS/HES
(operational)
(~1600) GIFTS (experimental)
The road to the next
generation Geostationary
Sounders
(~1600)
CrIS (operational)
(# of channels)
IASI (operational)
(~2400)
(18)
AIRS (operational)
GOES Sounder (operational)
Aircraft data (experimental), IMG
(12) VAS (experimental)
The ABS/HES is the next
step from experimental or
Ground-based (exp)
low spectral resolution to
HIRS (operational)
operational high-spectral
resolution, high temporal
IRIS (experimental)
sounders.
Pioneering work (theory)
time
Hyperspectral Altitude Resolved Water Vapor Winds
GOES R - IHOP simulation 1830z 12 June 02
GOES-8 winds 1655z 12 June 02
Simulated GOES-R winds (left) versus GOES current operational winds (right)
McIDAS-V:
Looking to the Future
We seek to advance the functionality of
McIDAS to meet the challenges of the future
while retaining its current capabilities,
including user-developed code.
We are investigating a means to transition
existing McIDAS applications to operate in a
new environment that does not have the
limitations of the current McIDAS system.
IDV: The Candidate
Transition System
• IDV (Integrated Data Viewer)
– Based on SSEC’s VisAD package
– Has a data model that is inherently capable
of working with data types, both now and in
the future
– Developed for the education community by
Unidata
– Designed to replace McIDAS-X and
GEMPAK at universities
What is VisAD?
• Open-source, Java library for building
interactive and collaborative visualization and
analysis tools
• Features include:
– Powerful mathematical data model that embraces
virtually any numerical data set
– General display model that supports 2- and 3-D
displays, multiple data views, direct manipulation
– Adapters for multiple data formats (netCDF,
HDF-5, FITS, HDF-EOS, McIDAS, Vis5D, etc.)
and access to remote data servers through HTTP,
FTP, DODS/OPeNDAP, and ADDE protocols
VisAD’s Data Model
• Designed to support virtually any
numerical data
• Metadata can be integrated into
each data object (units, errors,
coordinates, etc.) – carried along
through mathematical operations
• Supports mathematical operations
as well as evaluation and
resampling of data
• Supports spatial and temporal colocation of data
• Supports data sharing among
different users, different data
sources and different scientific
disciplines
Gully erosion/deposition
from VisAD-based
DEMViewer
What is the IDV?
• Unidata developed scientific
analysis and visualization tool
• Freely available Java™
framework and reference
application
• Provides 2- and 3-D displays of
geo-scientific data (plus, of
course, animations)
• Stand-alone or networked
application
http://www.unidata.ucar.edu/idv
IDV Features
• Integrated displays of a
variety of data types
• Support for a variety data
access methods
• Interactive probes
• User defined formulas and
functions
• Bundling of user preferences
• Integrated HTML viewer
• Easy configuration
• Integrated documentation
Supported Data Formats

The IDV can integrate displays of a variety
of geo-scientific and other data including:
– Scientific
• Geo-referenced
netCDF data (local or
remote)
• McIDAS/ADDE
– Societal
• GIS - ESRI shapefiles
(polygons)
– Educational
• HTML
• Quicktime
NO2 concentrations over
Berlin, Germany – data from
netCDF, maps from shapefile
Surface observations with Jet Stream winds
EOS AIRS Hyperspectral Data Analysis Applications
Current Efforts at SSEC
• A mechanism to transform McIDAS graphical
displays into the geo-referenced VisAD / IDV
display
• A unified user interface to access image data via
ADDE and OPeNDAP
• The capability to run IDV processes in a
background mode
• A mode to run heritage McIDAS applications
• OpenADDE: ADDE data servers available as
Open Source
What is ADDE?
• Abstract Data Distribution Environment
• Key data access method for McIDAS-V
• Data transfer and reformatting
– All data returned in common format: not as stored
• Sectorizing by geographic coverage and spectral
bands
• TCP/IP remote access through registered port
112
• Compressed transmission
• Variety of clients can access the servers
• In use for 10 years
What servers are available?
•
•
•
•
•
•
•
•
•
McIDAS format: Image, grid, point, text
netCDF as grid and point
HDF as point
MODIS Level 1b and products as images
AIRS Level 1b
AMSR-E Level 1b
Level III NEXRAD
AWIPS image
GRIB
What clients can be used?
•
•
•
•
•
McIDAS-X
McIDAS-Lite
IDV
Matlab
IDL
What is OpenADDE?
• Open Source version of the ADDE servers
and required McIDAS-X library routines
• Another method for data providers to
serve data
– McIDAS sites & non-McIDAS sites
– Subsecting
• Encourage data providers to write servers
for new data types
• Code provided under the GNU Lesser
General Public License
What is OpenADDE?
• Initial release are only the image read
servers:
– McIDAS Area, MODIS & AIRS HDF,
NEXRAD, GINI, AWIPS (?)
– McIDAS library routines
• Navigation and calibration modules, utilities
• 30% of the McIDAS-X library code
• Available at
http://www.ssec.wisc.edu/openadde
– Source code, installation and configuration
scripts, sample data
Where’s the data?
•
•
•
•
•
Servers at NOAA, NASA, SSEC, Unidata
Realtime global GOES
Realtime global MODIS Level 1b
GOES product files
Polar composites of satellite data
The Next Generation McIDAS
A Look Towards the Future
21st IIPS 8.2 San Diego, CA - 11 January 2005
Questions about
McIDAS-V or
OpenADDE?
Stop down to the
University of
Wisconsin booth:
406