Information, Cyberinfrastructure
and Databasing to Support
Ecological Modeling
Group focus
• Support group activities in:
– Communications
– Computing
– Storage
• Key Players
–
–
–
–
–
–
–
V. Frost-KU: Networking/Communications
Erik Perrins-KU: Communications
Dan Andresen-KSU: high performance scientific computing
Shiva Mohandass-KSU: Geodatabases
Rick Chubb-KSU: GIS Applications
Eric Bernard –KSU: GIS
GRAs
• Joshua Campbell - KU: GIS and Remote Sensing specialist
• Sayak Bose - KU: Communications and Networks
Communications Infrastructure
Vision for terrestrial communications infrastructure to
measure ecological properties
Flux tower to
central plains data repository
NESA HQ – Flux Tower Network
NESA Wireless Data Link
Flux Tower
N 39.05690
W 95.19061
Tree Interference
Hedgerow Tree Interference
1.03 km
NESA HQ
N 39.04793
W 95.19307
NESA cont’d…
Internet
Sa
802.11 link
te
lli
te
lin
k
Environmental study site
Nelson Environmental Study Area
Server
KU or KSU
Communication Hardware
at the Remote Flux Tower
•
•
•
•
Directional 802.11 antenna provides
enough gain to make 1.03 km link possible,
without line-of-sight and through vegetation
Commercial 802.11 equipment is versatile
and inexpensive
Current status: active two-way link between
tower sensors and campus office
Future work: 802.11 allows a
straightforward migration path to “Phase
Two” system, where there are multiple
sensors in the general vicinity of a remote
hub, e.g. a flux tower. Data from these
sensors are routed to the hub in a multihop wireless network; when the data
reaches the hub, it is transmitted back to
the research office via the main link.
NESA HQ – Flux Tower Datalogger Status Via Satellite & Wireless Link
Status of the Flux Tower datalogger
was checked across the satellite link
from ITTC offices. Microsoft Remote
Desktop was used to access the
data server PC located in the NESA
HQ building, then the datalogger
was queried via the 802.11 wireless
link to the Flux Tower. Flux Tower
data will typically be delivered via
FTP, rather than Remote Access
Satellite Link
Capabilities
•
•
•
•
•
Can provide field internet access anywhere in the contiguous US
Specified data rates
– Upload: up to 500 kbps
– Download: up to 2 Mbps
Measured data rates
– Upload: 477 kbps (large 10.7MB ZIP file, speed test from
www.dslreports.com)
Measured latency
– avg. delay ~800 ms, max delay 1060 ms, min delay 647 ms
Cost
– one-time cost $600, monthly cost $200
Satellite Link Bandwidth Tests
Microsoft Remote Desktop was used to access the data server PC located in the NESA HQ building, then
internet access bandwidth tests were initiated. Remote Desktop has a significant impact on the available
bandwidth reported. Typical numbers directly from the NESA site are 1440 kbps up / 700 kbps down.
• Conference paper publication:
–
E. Perrins, B. Kumaraswamy, and S. Bose, “Telemetry System for a
Remote Biological Field Station,” to appear in Proceedings of the
International Telemetering Conference, San Diego, CA, October 27--30,
2008.
K-State Eco-forecasting
Dataflow
K-State Museum Data Infrastructure
Specify
Databases
Herbarium
Collection
Records
Insect
Collection
Records
KSCData
MEPAR
Fish Ecology
SQL Server 2000
dandelion.konza.ksu.edu
ArcSDE 9.2
Geodatabases
Herb
Herb
Bugs
Fish
Web-Mapping
Applications
ArcGIS Server
(Herb)
ArcGIS Server
(Bugs)
ArcGIS Server
(Fish)
Data
Download
Site
Data
Download
Site
Herbarium
Collection
Records
K-State Herbarium collection consists of ca. 200,000 plant specimen
records, dating as far back as 1800s
Herbarium collection records are georeferenced and manually entered into
a Specify database called KSCData by student workers, supervised by Drs.
Mayfield and Ferguson.
KSCData
KSCData holds thousands of records to various geographic resolutions (X,Y
coordinates, County etc).
Georeferenced data from KSCData along with other vital collection
information are exported to an ArcSDE database (herb) when updated data
becomes available.
Herb
ArcGIS Server
herbmap
Herb, an ArcSDE 9.2 Geodatabase contains feature class datasets that are
ready for map display.
An interactive web-mapping application that displays County level plant
collection data along with other relevant GIS data in the background. This
application will blend with the already existing K-State herbarium web
pages (http://www.k-state.edu/herbarium).
Insect
Collection
Records
K-State insect collection consists of 824,000 insect specimen records
dating as far back as 1879.
Insect collection records are georeferenced and manually entered into a
specify database called MEPAR by a student worker, supervised by Dr.
Zolnerowich.
MEPAR
MEPAR holds hundreds of records to various geographic resolutions (X,Y
coordinates, County etc). Such georeferenced data along with other vital
mapping information are exported to an ArcSDE database (Bugs)
Georeferenced data along with other vital collection information are
exported to ArcSDE database (bugs) when updated data becomes
available.
Bugs
ArcGIS Server
insect_museum
Bugs, an ArcSDE 9.2 Geodatabase contains feature class datasets that are
ready for map display.
An interactive web-mapping application that displays County level plant
collection data. This application will blend in the already existing K-State
entomology web pages (http://www.oznet.ksu.edu/entomology).
Fish Ecology
Research
Fish
ArcGIS Server
fish_ecology
Data
Download
Site
Changes in fish habitat structure due to climate change and studies on fish
assemblage structure in adventitious streams involve GIS. Datasets
originating from National Hydrography Dataset (NHD) and National
Cooperative Soil Survey (NRCS) dataset are used in the K-State Fish
Ecology Lab
NHD, NRCS and other stream network datasets in that are analyzed at the
Fish ecology lab are stored as geodatabase files in an ArcSDE geodatabase
called Fish.
An interactive web-mapping application that displays County level plant
collection data. This application will blend in the already existing K-State
fish ecology web pages.
A query based data download site pertaining to the Kansas aquatic
research will enable users to download non-spatial data.
Status
•
•
All ArcSDE 9.2 Geodatabases (Herb, Bugs, Fish) are access-ready for
ecoforecast researchers.
All ArcGIS Server interactive Web Mapping applications are functional
–
–
–
http://www.konza.ksu.edu/herbmap
http://www.konza.ksu.edu/insect_musuem
http://www.konza.ksu.edu/fish_ecology
http://www.konza.ksu.edu/herbmap
http://www.konza.ksu.edu/insect_museum
http://www.konza.ksu.edu/fish_ecology
Kansas Biological Survey
Cyberinfrastructure (CI)
KBS: CI
• In Year 2, CI activities at KBS focused on:
–
–
–
–
continued organization of KBS databases
development of capabilities for data sharing and collaboration
personnel training in web-GIS technologies
strategic planning for CI hardware acquisition
• Relates to the NSF CI focuses of:
– Data, data analysis and visualization
– Virtual organizations for distributed communities
– Learning and workforce development
KBS: CI
• Significant effort has been focused on:
– Developing the digital database(s) required for
advanced CI applications
– Prototyping of web applications for the dissemination
of data and tools
– Personnel training in the underlying technologies
required to take geospatial data to the web
KBS: CI
Goals for Year 3
• Significant upgrade of hardware infrastructure
• Development of a metadata portal
– Utilize open source GeoNetwork software
• Organization and dissemination of KARS image
resources
• Continued development of web mapping
applications
Goals for Year 3 cont…
• Provide open access to the key database(s) housed at
the KBS
– Data will be searchable and accessible via the metadata catalog
– Data will be viewable through a range of applications,
• ArcGIS Server: ESRI GIS users
• FOSS: Open standards and non-ESRI users
• KML: Communication with the general public
• Begin development of web accessible tools to analyze
and visualize KBS data
Geodatabases and Models
Ecoforecasting Study Area
NED 10m Watersheds In/Out
of KS 1x1 Degree Tiles
+ Slope, Aspect, Hillshade
New 28 TB Storage Device
Spatial-Temporal Challenges
•
•
•
•
•
How does a database mimic natural
& social phenomena interwoven in
agricultural systems through time
and across scales?
Accuracy & Precision
Climatology, Elevation…
Scalability
Temporal Record
Distributed Sensor Networks
Data Storage, Access,
Computation, and Visualization
Progress
Enterprise geodatabases- New 28TB Sun Storage Device
Internet Mapping, Data Sharing, Web Services Developing
Development and integration of disparate data models continues
EPIC code modified and working on high performance computing cluster (HPCC)
for crop modeling
Geodatabases capable of storing spatial temporal variables and supplying
variables to hydrologic, economic and crop models
Developed KS Statewide SSURGO geodatabase (Spring 08)
~650,000 polygons in Feature Class (>1GB)
Improving methods for storage and serving detailed land surface for populating
models: 10m NED and LiDAR 2m (>1 TB)
Optimization of EPIC on HPCC nearly complete
132 CPU Years initially down to 2 weeks on 50 cores
Developing code to run all models on the HPCC
ArcGIS 9.2 is not multi-threaded but 9.3 is coming
Initial EPIC results of 3.5 billion runs are producing parameter estimates of water
use and crop yield very similar to those reported in the WIMAS and NASS for
one study area in KS
Complex Systems Linking Data Models
•
•
Economic Model yields land use
Production costs, Prices, etc.
Hydrologic Model yields groundwater
elevation
Saturated Thickness, Conductivity
Boundaries, Pumping Rate, etc.
Data Models & Web
Served Geodatabase
Groundwater Economic Model Variables
Object/Attribute Value Data Source
(
http://gis.ksu.edu/ogallala).
Groundwater
BaseElevation (slope) Macfarlane and Wilson [2006]
Depth (predeveloment saturated Gutentag et al. [1988]
thickness)
HydraulicConductivity Cederstrand and Becker [1998b]
SpecificYield Cederstrand and Becker [1998b]
Recharge Hansen [1991]
PumpingRate (historical) Wilson [1998]
GroundwaterElevation
Hausberger et al. [1998]
(historical)
Economic
LandUse (historical) Wilson [1998]
WaterUse (historical) Wilson [1998]
Commodity Research Bureau [2006]; InputPrice
USDA [2008c]
Commodity Research Bureau [2006]; OutptPrice
USDA [2008c]
Incentive USDA [2008a]
Regulation GMD4 [2008]
Agriculture
IrrigatedAcres Wilson [1998]
IrrigationType Wilson [1998]
SoilsDetailed USDA [1994, 2006]
IrrigationRequirement …
Parcel
AdministrativeRegulatedUse GMD4 [2008]
SurveyFirstDivision USDA [2008b]
Atmosphere
WeatherPointMeasurements
Kansas Weather Data Library [2008]
Flint Hills Ecoregion
Soil
~650,000 SURGO Soil Polygons in
Geodatabase
1 GB+ Feature Class
Flint Hills Ecoregion
Geology
Flint Hills Ecoregion
Geology
Soil
Flint Hills Ecoregion
Temperature
Precipitation
Flint Hills Ecoregion: Konza
Landform: LiDAR 2m Hillshade
Konza Headquarters
LiDAR Elevation Data
Channel Profiles
Kansas River
Manhattan, KS
NED 30m
NED 10 m
LiDAR
•
3D River-Stream Channel and Floodplain
Flint Hills Ecoregion: Konza
Landform: LiDAR 2m Hillshade
NHD Scale Limitations For Site or Management Scale
Flint Hills Ecoregion: Konza
Landform: NAIP 1m Color draped on 3D LiDAR Surface- Konza
headquarters
Flint Hills Ecoregion: Konza
3D Landform: Challenges in Spatial Temporal Visualization with very
Large Data Sets
Flint Hills Ecoregion: Challenges
Land Management: Detailed Detection of Systems Condition and Change
Thank you and our research
sponsors!
www.k-state.edu/ecoforecast