NIDAS
NCAR In-Situ Data Acquisition Software
Gordon Maclean
Software Engineer
NCAR/EOL
Background





Developed at the NCAR Earth Observing Laboratory (EOL), starting
in 2004.
Targeted as the primary data system for the NSF HIAPER, NCAR's
Gulfstream V research aircraft.
Developers included software engineers with experience at EOL's
Research Aviation Facility (RAF) and In-Situ Sensing Facility (ISF)
Design was expanded to include the acquisition of ground-based
meteorological measurements.
Built on the past experience of EOL engineers:


Aircraft Data System: ( ADS, ADS-2), 1970s – 2009
Surface systems: Portable Automated Mesonet (PAM I,II,III), 19732004 and the Atmosphere-Surface Turbulence Exchange Facility
(ASTER), 1988-2002
Targeted Research
Diverse topics, primarily NSF funded

Aircraft




Boundary layer to 50K ft
Cloud physics, convection, severe weather, chemistry and
pollution, remote sensing, sensor validation
20 to 200 flight hours per research project
Surface

PBL surface boundary conditions, energy balance, turbulence

Fluxes of momentum, heat, moisture and trace gases

Horizontal and vertical (0-50 meter) gradients

24x7 operations, often unattended

Short term, typically 1 to 3 months per project
Airborne Instrumentation
Navigation
GPS, inertial reference systems, radar altimeter
State parameters
Pressure, temperature, humidity, dew point (various)
Winds
Pitot tubes
Laser Air Motion Sensor: LAMS (under development)
Laser absorption hygrometers
Vertical Cavity Surface Emitting Laser Hygrometer: VCSEL
NCAR TDL Water Vapor
Cloud particles, aerosols
2D particle probes: PMS 2D-C, 2D-P, Fast2DC
Cloud Droplet probe
Forward Scattering Spectrometer Probes: FSSP 100,200,300
Ultra-high Sensitivity Aerosol Spectrometer: UHSAS
Condensation Nuclei counters
PMS King Hot Wire Liquid Water Probe
Cloud Particle Imager: 3V-CPI
Ice detectors
Rosemount Icing Detector
Small Ice Detector: SID-II
Inlets
Counter-Flow Virtual Impactor: CVI
HIML
Chemistry
Chemiluminescense: NCAR CO, O3
Gas Chromatographs: Panther, UCATS: H2O, O3, N2O,CH4
Soot photometer: DMT SP2
Picarro: CO2,Methane,O18,H2O,Deuterium
Radiation
Pyranometers, pyrgeometers, UV, IR surface temp
Temperature profiles
Microwave Temperature Profiler: MTP*
Engineering
Accelerometers, CMIGITS
Air Pressure Scanner: PSI-9116
Aircraft Deployments
Surface Instrumentation
State parameters
Time, location
2d anemometers
Barometer, T, RH: (various)
GPS with pulse-per-second
RMYoung Propvane, 85004, Gill WindSonic, WindObserver,
MetOne, Vaisala WMT
3d anemometers
Campbell CSAT3
Applied Technologies: ATIK
Water Vapor, CO2
Campbell Krypton Hygrometer
Other trace gases
LI-COR 7000, 7500, Campbell EC150
Radiation
Pyranometers, Pyrgeometers: K&Z, Eppley, Hukseflux NR01
Net: Rebs Q7, Hukseflux NR02, KZ NRLite2
Solar and Diffuse: SPN1, LI-COR, Apogee
Surface Temp: Everest
Photosynthetic Active Radiation: LI-COR 190SA
Soil
Heat flux, temp, moisture, thermal prop: REBS, Decagon,
Hukesflux
Mesh sensor networks
Digi Zigbee, EOL Wisard
Weather stations, loggers Vaisala WXT, CR10
Precipitation
ETI, MRI rain gauges, Decagon Leaf Wetness
Fast pressure
Paroscientific 216B
Hot film anemometers
Dantek 1D/3D
Engineering
Power monitor, battery and charging system status
Surface Deployments
Required Data System Interfaces
Type
RS 232/422/485 serial
Implementation
Diamond Systems 8 port (with digio)
WinSystems PCM-COM8
Analog to Digital, 16 bit
NCAR 8 channel, differential, sigmadelta
Diamond MM-16, MM-32: 16 channel
diff, multiplexed, SAR
Processor motherboard
Processor motherboard
Mesa Anything I/O FPGA
Diamond GPIO-MM
Arcom Viper CPU
Diamond MM, IR104
Condor Engineering CEI-420A
Brandywine PC104-SG
USB/Bluetooth adapters
Serial interface to EOL Wisard base
motes
USB 1,2
Ethernet
Pulse counters, frequency
Digital I/O, analog out, relays
ARINC 429 navigation
IRIG-B timing protocol
Bluetooth RFCOMM serial
Mesh Networks, sensor arrays
Initial Design Choices

















Operating System: Linux
Open source, near real-time, wide choice of system vendor
Same OS on embedded systems and processing and display servers
Extensive network services
Multithreaded, dynamic (run-time) object loading
Data Sampling Modules: ARM Processors and PC104 expansion bus
Low power (~2 Watt) low heat generation
Extensive vendor support for PC104 interface cards
Languages:
C++: object oriented, compiled
C: low level interface driver modules
Configuration:
XML: flexible, tree structured, widely used, field editable (by NIDAS geek)
Software Tools:
Version control: subversion
Builds: scons
Packaging: RPM, Debian (embedded)
Server Systems
Networks
Data Sampling Modules, DSMs
NIDAS Sample Flow
Time-tagged, converted
Time-tagged, raw
NIDAS
Processors
Processing Pipeline
Statistics
NetCDF
files
Sorter
Sensor
Objects
network
SensorHandler
Sorter
Sensor Objects
Low level driver modules
ncar_a2d
serial
Data System Hardware Interfaces
PC104 A2D card Serial Port
Calibration
data
network
Instrumentation
Ethernet
Sensor Object Hierarchy
Highlights

Supports diverse instrumentation

Wide range of sampling rates: ~0 to ~1000 samples/sec

Highly configurable, distributed processing

Runnable on low power (~2 Watt) embedded processors

Networked,
supporting remote login, serial sensor interaction, power
control

Open source, including all low level driver code

Acquired data is archived in raw sensor format

Archived time tags have microsecond precision

System clocks have ~20 microsecond accuracy

Many deployments

Revision controlled, automated tests

GUI configuration editor for aircraft deployments
Future Plans, More Info
•
Kernel upgrades on DSMs, using newer real-time capabilities of Linux.
•
Reduce serial sampling latency (line discipline kernel module)
•
Improve network service discovery mechanism
•
Enhancements to real-time displays, diagnostics
•
Framework for instrumentation control
•
Calibration database
•
Other server platforms: MacOSX
•
•
Other embedded hardware platforms: Eurotech Titan, gumstix, new
ARM/Intel processors, pc104+
Support and collaboration with outside users.
http://www.eol.ucar.edu/data/software/nidas (under development)
maclean@ucar..edu
Questions/Answers