Making Usable Data Available and Available Data
Usable: The Application of Geospatial Standards in
Near Real-Time Sensor Networks
Daniel Getman
Aaron Myers
Geographic Information Science and Technology Group
Oak Ridge National Laboratory
April 20, 2007
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
What is SensorNet
 Much more than what is presented here
 Stand Alone Sensor Networks
• Mobile Sensor Systems
• Weigh Station Systems
• Port Systems
• Military Base Systems
 Distributed Sensor Networks
• Connecting stand alone systems in a larger network
 Overarching Concepts
• Interoperability/Standards
• Real Time Data Distribution and Alerting
• Ubiquitous Data Access
• Development of Global Scale Sensor Networks
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Describing the Stand Alone Systems
 Multiple Sensors
 Chemical – Radiation – Biological – Weather
 Video with real time analysis capabilities
 Multiple Viewers
 ArcMap
 Google Earth
 Browser based viewers
 Integration with Models
 HPAC
 Data Distribution Capabilities
 Mobile systems communicating with operations centers and
other systems
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Defining The Goals
 Minimum System Goal
 Provision of near real-time data within the stand alone
system while also providing access to data to external users
including Operation Centers and other emergency response
personnel
 Maximum System Goal
 Develop an open standards based system within which a
variety of sensors, viewers, and complete stand alone systems
can plug-and-play can use and share near real-time sensor
data
 Stretch Goal
 Make the openness of the system transparent to all users
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Standards Used in the Stand Alone Systems
 Open Geospatial Consortium (OGC)
 Sensor Web Enablement (SWE)
• Includes Sensor Observation Service (SOS), Sensor Alert
Services (SAS), and Sensor Planning Service (SPS)
 Web Feature Service (WFS)
 Geography Markup Language (GML)
 U.S. Department of Homeland Security (DHS)
 ANSI N42.42 (Radiation Data Representation)
 Organization for the Advancement of Structured Information
Standards (OASIS)
 Common Alerting Protocol (CAP)
 National Institute of Standards and Technology (NIST)
 IEEE 1451 (Sensor Plug-n-play)
Courtesy: David Resseguie, ORNL
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Why Standards are Great
 Standards are great once they are adopted
 Data sharing is not only easier but can be easily automated
 Everything is less expensive…eventually
 Creativity is stimulated in application development
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Why Standards Can Be Difficult to Use
 Querying differences
 Data Size Differences
 Data Structure Differences
Traditional Database Query
WFS Database Query
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
WFS and SQL Queries
 Traditional SQL Query
SELECT Col1, Att1, Val1
FROM TABLE1
WHERE Val2 = “SomeValue”
 WFS Query
<wfs:GetFeature service="WFS" version="1.0.0" xmlns:gml="http://www.opengis.net/gml"
xmlns:ogc="http://www.opengis.net/ogc" xmlns:snet="http://www.sensornet.gov/snet"
xmlns:wfs="http://www.opengis.net/wfs">
<wfs:Query typeName=“snet:Table1">
<ogc:Filter xmlns:ogc="http://www.opengis.net/ogc">
<ogc:PropertyIsLike>
<ogc:PropertyName>//snet:Val2</ogc:PropertyName>
<ogc:Literal>SomeValue</Literal>
<ogc:PropertyIsLike>
</ogc:Filter>
</wfs:Query>
</wfs:GetFeature>
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Simple WFS Performance Tests
Xhive and Oracle bassed WFS and MySQL DB
Processing Time For n Concurrent Queries While Processing Ten Inserters at Ten Features Per Insert
25000
XHive based WFS
Oracle based WFS
MySQL DB
Average Response Time (ms)
20000
15000
10000
5000
0
1
5
10
15
20
25
30
Concurrent Requesters
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
35
40
45
50
100
Practical Difficulties
 Clients can have conflicting expectations
 Things that are difficult with Standards Based Systems
• Legacy Software and Data
– Few software packages will natively access standards based data
sources
• Familiar/Robust/Highly Functional Interface
– It has to be as good as the proprietary system
• Near Real-Time Dynamic Data
– Lots of queries, some very specific (alerting)
• Database Design Limitations
– Designed for support of standard rather than support of user
activities
– Large Data Capacity can be problematic
» Keeping all of the data without impeding system speed
• Data Integration
– Leveraging of existing proprietary data
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Idealistic SensorNet System
Everyone Else
Standard Query Format
Standard Output Format
WFS
Sensor
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Standard Output Format
Standards
Based Viewer
Where does this leave the ISS?
Proprietary Output
Format 1
Sensor
?
?
Standard Output
Format
Standard Query Format
WFS
Sensor
Proprietary Output
Format 2
Sensor
?
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
?
Everyone Else
Standard Output Format
?
Proprietary
Viewer 1
?
Standards
Based Viewer
Proprietary
Viewer 2
The Duct Tape Approach
 Implement standards where the technology and client
requirements allow
 Implement component based solutions that handle the places in
the system where standards either do not exist or are not fully
implemented
 Plan to remove the duct tape when the standards are
implemented and adopted
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Slightly More Realistic SensorNet System
Everyone Else
Proprietary Output
Format 1
Custom
Conversion
Utility
Sensor
Proprietary
Data Source
Proprietary
Viewer 1
Proprietary Output
Format 2
Sensor
?
Standard Output
Format
Custom
Conversion
Utility
Sensor
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Proprietary
Viewer 2
?
WFS
Standards
Based Viewer
Tale of Two Deployments
 JFHQ SNAPS in Washington DC
 Mobile SensorNet System (Actually in a trailer)
 JOC System designed to receive data from Mobile and Base
systems
 GPS enabled Radiation, Chemical, Weather, Video
 HPAC Integration
 SRRPP at the Port of Charleston
 Land and Marine based SensorNet Systems
 Vehicle and Officer GPS Tracking
 GPS enabled Radiation, Video
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
JFHQ SNAPS Deployment
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
SRRPP Deployment
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Lessons Learned with the SMRSS
 Intrinsic problems include
 Data is duplicated
 Two conversion processes to maintain
 Performance Issues
 Both WFS and ArcSDE are bottlenecks compared to direct database
interaction. Together, they can really hinder performance
 The conversion to ArcSDE caused a large number of reads on the WFS.
This had a negative impact on the overall performance of the system. More
reads per second means less writes per second
 Inserting data into ArcSDE is significantly slower than inserting data
directly into a database
• We found that ArcObjects inserts were faster than SDE API inserts
• Querying against the data tables directly in Oracle to gather data used
in the conversion process, rather than going through SDE, is a good
shortcut
 Conversion process can be used for other tasking that would otherwise
require an additional WFS call
• Alerting and interfacing with other systems (cameras for example)
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Realistic Idealistic SensorNet System
Everyone Gets What They Want
Everyone Else
Single Conversion Utility
Proprietary Output
Format 1
Proprietary
Data Source
Sensor
Proprietary
Viewer 1
Proprietary Output
Format 2
Sensor
Standard Output
Format
Proprietary
Viewer 2
Custom
Conversion
Utility
WFS
Sensor
No pounding the WFS with SDE Inserts
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Standards
Based Viewer
Evolution of the RISS
Times They Are A-Changin'
Everyone Else
Proprietary Output
Format 1
Proprietary
Data Source
Sensor
Proprietary
Viewer 1
Proprietary Output
Format 2
Sensor
Standard Output
Format
Proprietary
Viewer 2
Custom
Conversion
Utility
WFS
Sensor
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Standards
Based Viewer
Evolution of the RISS
Everyone Else
Proprietary Output
Format 1
Proprietary
Viewer 1
Sensor
Many, Many
Data Services
Proprietary Output
Format 2
Sensor
Standard Output
Format
Proprietary
Viewer 2
Custom
Conversion
Utility
DB
Sensor
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
Standards
Based Viewer
New & Improved Idealistic SensorNet System
Everyone Else
Standard Output
Format
Many, Many
Data Services
Sensor
Proprietary
Viewer 1
Standard Output
Format
Proprietary
Viewer 2
Sensor
Standard Output
Format
Sensor
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
DB
Standards
Based Viewer
- Questions and Comments -
Daniel Getman
getmandj@ornl.gov
865-241-1745
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY