Building An End-to-end System
For Long Term Soil Monitoring
Katalin Szlávecz, Andreas Terzis,
Razvan Musaloiu-E., Alex Szalay,
Josh Cogan, Randal Burns
The Johns Hopkins University
Jim Gray, Stuart Ozer
Microsoft Research
The Baltimore Ecosystem Study
Established 1998
Part of LTER (Long
Term Ecological
Research) Network
Focus: Urban
environments as
ecological systems
Collaborative effort
IES, USFS, UNC,
USGS, UMBC, UMD,
JHU, PPF, Yale
http://www.beslter.org/
BES Main Questions
Structure
What is the spatial and temporal patch structure
of socio-economic, ecological and physical factors
in the urban ecosystem?
Function
What are the fluxes of energy, matter, capital
and population in the urban ecosystem
People
How can people develop and use an understanding
of the city as an ecological system to improve
the quality of their lives and their environment?
Ongoing BES Data Collection
precipitation depth
(point gages)
NEXRAD precipitation
precipitation
chemistry
air temperature
humidity
atmospheric pressure
wind speed
wind direction
total solar irradiance
photosynthetically
active radiation
net radiation
CO2 flux
CO2 profile
leaf wetness profile
dewpoint temperature
soil permanent plots stream chemistry
water temperature
temperature
air temperature
moisture content
pH
trace gas fluxes
dissolved oxygen
(N2O, CO2, CH4)
specific
nutrients (total N,
+
conductance
NO3 , NH4 , PO4 )
anions (NO3-, CI -,
matric potential
SO42-, PO4-)
heat flux
cations (Na+, K+,
fauna
Mg 2+, Ca2+)
toxics
total N and P
soil survey data
total suspended
surficial geology
solids
well logs
turbidity
rock core records
fecal coliform
groundwater levels
toxics
pathogens
stream flow
biota
stage
velocity,discharge
Welty and McGuire 2006
storm sewer flows
storm sewer toxics
storm sewer
nutrients
vegetation inventory
vegetation remote
sensing
digital elevation
model
LIDAR elevation
data
sediment surveys
channel profiles
channel stability
data
land use/land cover
demogr aphic data
economic data
Current Methods Of Data Collection
1.
Manual data collection in the field, some requiring laboratory
analysis
e.g., stream chemistry, including N species
UPS
Sample Bottle
Field Tech
Site Visit
Site Observation
2.
Turbidity
TSS
Institute of
Total N and P
Ecosystem
Studies, NY
Anions
Cations
UMBC
Fecal Coliform
Temperature
pH
Dissolved Oxygen
Data automatically collected and periodically downloaded in field
e.g., stream temperature, well water levels -- data loggers
3.
Data automatically collected and downloaded via cellular modem
e.g., USGS stream flow (stage); meteorological data
Welty and McGuire 2006
SQL Server
Database
Soils
The Final Frontier
Mechanical support
Habitat for organisms
Water storage
Storage of organic
matter (carbon)
Element release
(greenhouse gases:
CO2, CH4, NOx)
Soil
“The Poor Man’s Rainforest”
Biodiversity
Biomass
“..we need invertebrates
but they don’t need us”
E. O. Wilson: In Search of Nature
Density Of Soil Organisms
Heterogeneity
The Greatest Challenge
Vertical Heterogeneity
Dung beetles
Nardi: A Guide to life in the Soil
Earthworms
Harvester ants
Landscape Heterogeneity
Remnant fragments
Parks
Baltimore,
1752
Lawns, gardens
Greenhouses
Increasing
human impact
Buildings
Agricultural Landscape
USDA Sustainable Agricultural Systems Lab, Beltsville
E. Venteris, unpubl.
http://www.ars.usda.gov/SP2UserFile
s/Place/12650400/2.FSP10YearReviewSummaryBook103105.pd
f
Crop yields,
Weed dynamics
Soil erosion
Nitrogen balance
Greenhouse gas
fluxes,
Soil invertebrate
communities
Economic
performance
70 % of US N2O
emissions comes
from agriculture
Heterogeneity
Sampling problem
Scaling problem
Motivation For A Sensor Network
Monitoring: Background data, trends =>
Soil animal activity/metabolic processes depend on
moisture, temperature
Frequent visits disturb the sites
Soil respiration, trace gas fluxes
Better input for terrestrial hydrology models
CS: Build and learn from a deployed system
Understand and connect local, regional and global
biogeochemical and cycles
Identify sources and sinks, measure fluxes
Hardware And Architecture
Deployment Site
Forest next to JHU’s Homewood Campus
Accessible environment
Diverse environmental conditions
Network Design
Ten mote network
Each mote
samples every min
data stored in FLASH
status every 2 min,
wait for data request
Single hop network
Gateway connected
to campus network
2m
8m
2m
From Raw Data To Useful Quantities
Current Status Olin Deployment
Operating since Sep 2005
Over 8M data points
Winding down
Online Data Access
a
b
c
Database/Datacube
SQL Server 2005 database
Rich metadata stored in DB
Adopted from astronomy: SkyServer
Data access through web services
Graphical interface
DataCube under construction
(multidimensional summary of data)
Sensor Datacube Dimension Model
Sensor Dimension
depth
type
Measurement
Type
Dimension
Time Dimension
all
all
year
site
week
wk. of year
patch
day
day of year
node
hour
hour of day
sensor
measurement
type
tenMinute
make/model
Measures
(sum, count, min,
max, median, std deviation)
Lessons Learned
Transmission range is considerably
shorter than in lab due to foliage
Relay node helps 
Low level programming
is still required 
Importance of sensor uniformity
is essential
Switch to Echo sensors 
Nodes continue operate
despite large environmental
fluctuations 
Waterproofing is still an issue
Lessons Learned
Data systems
We got real data, end-to-end ! 
Sensors respond to
environmental changes 
Database from off-the-shelf components 
Getting high level summaries: DataCube 
We need an automated pipeline:
The current two manual steps are still
too labor intensive 
Integration Of Sensor Data Into
Baltimore Ecosystem Study Projects
CO2 Flux tower
Many land use/land
management types
Diverse soil
conditions, biota
Plan: To deploy
200 motes
Better understanding
of coupled water
and C cycle
Map by E. Ellis and D. Cilento,
Dept. of Geography, UMBC
Integration
Urban rural gradient
Oregon Ridge
F,W
Gwynnbrook
McDonogh
W
A,G
F
G
W
A
Hillsdale
F
Leakin Park
F,W
UMBC
G
Map Source: MD Office of Planning, Revitalizing
Forest
Grass
Wetland
Agriculture
Adding New Sensors
Trace gases
B, Oriola, F. Farahi, and M.
Cavigelli, USDA
P. dilatatus
2500
A. nasatum
2000
1500
[CO2] in PPM
Soil
1000
500
0
0
5000
10000
15000
20000
25000
Time in seconds since seal
30000
35000
40000
45000
Ongoing BES Data Collection
precipitation depth
(point gages)
NEXRAD precipitation
precipitation
chemistry
air temperature
humidity
atmospheric pressure
wind speed
wind direction
total solar irradiance
photosynthetically
active radiation
net radiation
CO2 flux
CO2 profile
leaf wetness profile
dewpoint temperature
soil permanent plots stream chemistry
temperature
water temperature
moisture content
air temperature
trace gas fluxes
pH
(N2O, CO2, CH4)
dissolved oxygen
nutrients (total N,
specific
+
NO3 , NH4 , PO4 )
conductance
matric potential
anions (NO3-, CI -,
heat flux
SO42-, PO4-)
fauna
cations (Na+, K+,
toxics
Mg 2+, Ca2+)
total N and P
soil survey data
total suspended
surficial geology
solids
well logs
turbidity
rock core records
fecal coliform
groundwater levels
toxics
pathogens
stream flow
biota
stage
velocity,discharge
Welty and McGuire 2006
storm sewer flows
storm sewer toxics
storm sewer
nutrients
vegetation inventory
vegetation remote
sensing
digital elevation
model
LIDAR elevation
data
sediment surveys
channel profiles
channel stability
data
land use/land cover
demogr aphic data
economic data
Impact And Importance
Use of sensor data enables new science
Long term stable monitoring is essential
Environmental science combines many different
kinds of data:
Sensors+animals+hydrology…
One person’s background is the other’s signal
We need to move from Excel to DB
Data must be linked to GIS systems
Advanced visualization is important
High level view as well as all the details
Acknowledgements
Microsoft Research
The Gordon and Betty Moore Foundation
Seaver Foundation
Gordon Bell
JHU Provost Undergraduate
Research Fund
Allison Smykel, Claire Welty
© 2006 Microsoft Corporation. All rights reserved.
Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation.
Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft,
and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.