The MEDIGRID Distributed
Framework for Multi-risk
Assessment of Natural Hazards
Isabella Bovolo1, Simon Abele1, James
Bathurst1 , David Caballero2, Marek
Ciglan3, Branislav Simo3
1University
of Newcastle upon Tyne, UK
2Tecnoma, Spain,
3Institute of Informatics, Slovakia
GOCE-CT-2003-004044
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Outline




7th – 9th March, 2007
Introduction
MEDIGRID System
Example application
Conclusion
European GeoInformatics Workshop
Edinburgh
Outline




7th – 9th March, 2007
Introduction
MEDIGRID System
Example application
Conclusion
European GeoInformatics Workshop
Edinburgh
Mediterranean Grid of Multi-risk
Data and Models
Project Aim
 Integrate several, independent, preexisting, natural hazard models into
one single system.
 Run any of the models in sequence
using data stored remotely
 multi-model, multi-hazard assessment
tool
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Challenges

Models
•
•
•
•


developed in isolation
different software architectures
different data needs
applicable to different spatial / temporal scales
Identification of common data themes and links
between models
How do models communicate? Common data
formats for data exchange between models
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Models
FMIS
FIRESTATION
Wind field /FMC / ROS
&
Fire Spead Engine
Hydrology,
Erosion,
Landslide
Model
SHETRAN
Flood Model
Forest Fire
Model
ALADIN,MM5,HSPF,NLC, DAVEF &
Meteorology, Hydrology and Hydraulics
SPREAD
PROMETHEUS
Vegetation
7th – 9th March,
2007
European GeoInformatics Workshop
Erosion
Model
Recovery Model Edinburgh
Models and Data Flows
SHETRAN
Hydrology,
Erosion,
Landslide
Model
Vegetation
Maps
Forest Fire
Model
7th – 9th March, 2007
Erosion
Maps
Vegetation
European GeoInformatics Workshop
Recovery Model Edinburgh
Test – sites



Western Spain
Southern France
Central Portugal
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Outline




7th – 9th March, 2007
Introduction
MEDIGRID System
Example application
Conclusion
European GeoInformatics Workshop
Edinburgh
MEDIGRID System

Provide modular decision support
framework for assessing multiple hazards
based on Grid-enabled applications and
distributed architecture.
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Globus Toolkit

Globus Toolkit environment installed on
all MEDIGRID process & data storage
nodes
• Common grid technology
• Secure infrastructure
• Allows data / computational resources to
be deployed via internet

Allow users to access the system
remotely through a web portal
(containing n*portlets)
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
MEDIGRID System
1. User Interface
6
2. System
Services
5
3
3. User Services
2
4
1
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
1. User Interface
GridSphere portlet container:
 Open-source portlet framework
 Offers common Grid portal features / functionality
 Provides shared access point to Grid services &
resources
Allows:
•
•
•
•
Single sign-on
Credential management
Job submission / monitoring
Resources management



Models
Model connectivities
Data
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
2. System Services
Issues
 Variations in model & data security /
access/use policies across partner sites
 Input data must be stored, maintained,
accessed, be searchable
• Inputs / Results should be a single data set

Current Grid Middleware have not
implemented multi-platform support for
WS-* and Grid standards; e.g. GridFTP,
GRAM – only work on Linux platform
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
2. System Services

Several customised components
developed:
- Data services
-
-
Data Transfer Service
Data Resource Management
- Job submission service
Implemented with Globus Java WSRF Core
• Industry standard
• Link between Web service and Grid
technologies
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Data services
architecture
User
Portal
Data transfer WS interface
Auth
&
Authz
Medigrid
Data
Transfer
7th – 9th March, 2007
GridFTP
WS interface
WS interface
Replica
Location
Service
Metadata
Catalog
European GeoInformatics Workshop
Edinburgh
Data Access policy
Data security

Grid Certificates – signed by Medigrid
Certificate Authority (CA)
• Hosts (i.e. computational/data nodes)
• Individual users

Access Control Lists (ACL) define what
resources hosts & users can access
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Data services
architecture
User
Portal
Data transfer WS interface
Auth
&
Authz
Medigrid
Data
Transfer
7th – 9th March, 2007
GridFTP
WS interface
WS interface
Replica
Location
Service
Metadata
Catalog
European GeoInformatics Workshop
Edinburgh
Data Management
Data Transfer Service (DTS)

Custom MEDIGRID Grid service
• Extends GridFTP


Provides secure data transfer mechanism,
applies data access / security policies
Data Transfer Operations
• Upload data to Medigrid sites
• Download data from Medigrid sites
• Third-party transfer between Medigrid sites
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Data Management
Data Transfer Service, cont…
 Integrated with:
• Replica Location Servce (RLS)
(Globus Toolkit)
• Metadata catalog Service (MCS)
(GriPhyN / NVO project component)
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Data services
architecture
User
Portal
Data transfer WS interface
Auth
&
Authz
Medigrid
Data
Transfer
7th – 9th March, 2007
GridFTP
WS interface
WS interface
Replica
Location
Service
Metadata
Catalog
European GeoInformatics Workshop
Edinburgh
Metadata Catalog
Service
MCS components:
 set of standard interfaces and APIs for
storing and accessing resource metadata
 query mechanisms for accessing the
database
 allows multiple metadata categories to be
associated with any logical file name
 domain-independent metadata attributes
 extensibility for user-defined attributes
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Data Services
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Virtual Filesystem
Browser
File / Directory
View
Copy
Move
Delete
Create
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Metadata attributes
portlet
View metadata attributes
Edit metadata attributes
Add metadata attributes
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Metadata search
portlet
metadata search based
on attributes values
● specify multiple query
conditions
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Job Submission Service



WSRF Web Service & client tools
Runs on Windows and Linux
Acts as an interface to a defined, fixed
executable;
• Executable is configurable via user supplied
parameters, however,
• does not allow specification of new .exe files



Monitors job state
Multiple jobs to computational services can
be submitted and managed.
Jobs are queued and executed one at a time
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
3. User Services

Computational nodes
• Host hazard assessment applications

Simple & complex
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Model adaptations

Modify code to be standalone
executable
• Remove graphical user interface
• command line + arguments


Convert to web-service to run remotely
over the internet
Develop pre- and post-processing tools
to translate data to and from a
common format (Grass ASCII)
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
3. User Services

Through portal (containing portlets),
users can;
• Run models and…





Specify model interconnectivities
Choose pre-defined configuration input files
Customise parameters
Select outputs required
Download / View results
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Site 1
Medigrid
Central
Data
Services
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
FFireRes
ffire_data_1
ffire_data_2
ffire_data_3
RLS
Site 2
MEDIGRID
System
Example
WorkFlow
ffire_data_4
ffire_data_5
ffire_data_6
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
Site 1
Medigrid
Central
Data
Services
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
FFireRes
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
RLS
User
ffire_data_4
ffire_data_5
ffire_data_6
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
Site 1
Medigrid
Central
Data
Services
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
FFireRes
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
RLS
User
ffire_data_4
ffire_data_5
ffire_data_6
DataRes1
file_1
file_2
file_3
Search
Metadata
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
Site 1
Medigrid
Central
Data
Services
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
FFireRes
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
RLS
User
ffire_data_4
ffire_data_5
ffire_data_6
DataRes1
file_1
file_2
file_3
User finds
LFNs of
required data
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
Site 1
Medigrid
Central
Data
Services
Submit Job,
specify LFNs
of data sets
(e.g.
ffire_data_1)
ffire_data_1
ffire_data_2
ffire_data_3
RLS
FFireRes
Site 2
User
ffire_data_4
ffire_data_5
ffire_data_6
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
Site 1
Medigrid
Central
Data
Services
Locate
ffire_data_1
FFireRes
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
RLS
User
ffire_data_4
ffire_data_5
ffire_data_6
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
Site 1
Medigrid
Central
Data
Services
ffire_data_1:
Site 1
FFireRes
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
RLS
User
ffire_data_4
ffire_data_5
ffire_data_6
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
Site 1
Medigrid
Central
Data
Services
Perform
authz
FFireRes
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
RLS
User
ffire_data_4
ffire_data_5
ffire_data_6
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
Site 1
Medigrid
Central
Data
Services
Download data
ffire_data_1
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
RLS
FFireRes
User
ffire_data_4
ffire_data_5
ffire_data_6
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
Site 1
Medigrid
Central
Data
Services
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
FFireRes
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
RLS
User
ffire_data_4
ffire_data_5
ffire_data_6
ffire_data_1
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
Site 1
Medigrid
Central
Data
Services
Register
data file
replica
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
RLS
FFireRes
User
ffire_data_4
ffire_data_5
ffire_data_6
ffire_data_1
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
DataRes1
Run
Job
file_1
file_2
file_3
FFireRes
Site 1
Medigrid
Central
Data
Services
file_1
file_2
file_3
FFireRes
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
RLS
DataRes1
User
ffire_data_4
ffire_data_5
ffire_data_6
ffire_data_1
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
DataRes1
Job
Finished
file_1
file_2
file_3
FFireRes
Site 1
Medigrid
Central
Data
Services
file_1
file_2
file_3
FFireRes
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
RLS
DataRes1
User
ffire_data_4
ffire_data_5
ffire_data_6
ffire_data_1
job_output_1
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
DataRes1
file_1
file_2
file_3
FFireRes
Site 1
Medigrid
Central
Data
Services
Job
Finished
Register new
data file
(job_output_1)
RLS
file_1
file_2
file_3
FFireRes
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
DataRes1
User
ffire_data_4
ffire_data_5
ffire_data_6
ffire_data_1
job_output_1
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
Site 1
Medigrid
Central
Data
Services
ffire_data_1
ffire_data_2
ffire_data_3
RLS
Job
Finished
FFireRes
Site 2
User
ffire_data_4
ffire_data_5
ffire_data_6
ffire_data_1
job_output_1
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
Site 1
Medigrid
Central
Data
Services
DataRes1
DataRes1
file_1
file_2
file_3
file_1
file_2
file_3
FFireRes
FFireRes
Site 2
ffire_data_1
ffire_data_2
ffire_data_3
Job
Finished
RLS
User
ffire_data_4
ffire_data_5
ffire_data_6
ffire_data_1
job_output_1
DataRes1
file_1
file_2
file_3
LSlidesRes
Site 3
Metadata
Catalog
7th – 9th March, 2007
European GeoInformatics Workshop
lslidesdata_1
lslidesdata_2
lslidesdata_3
Edinburgh
Visualisation

Users can:
•
•
•
•
download files: text / HDF
1D – graphs
2D – maps, (GRASS GIS -> png, jpg)
3D – pictures / virtual env.
(Blender/VRML)
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Outline




7th – 9th March, 2007
Introduction
MEDIGRID System
Example application
Conclusion
European GeoInformatics Workshop
Edinburgh
Case Study Area





Extremadura
Semi-arid, Med climate
<600 mm year irregular
rainfall
-ve hydrological balance
in area (aquifers not
guaranteed to totally
recharge)
Depends on large nearby
rivers
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Valencia de Alcántara
(02/08/2003)
The fire:
 Started by lightning in Portugal
 Crossed Spanish border the
03/08/03
 Burned > 13,000 ha in one week
 Mostly pine forest, livestock and
agricuture
 Effect on vegetation and
catchment hydrology
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Fire area
Spain - Valencia de Alcántara
2
%
2
%
02/08/2003
DEM20m
2
%
*
#
13,692 ha
burned
2
%
*
#
Height (m)
120 - 280
2
%
2
%
280 - 380
2
%
30 km
7th – 9th March, 2007
380 - 480
2
%
480 - 580
580 - 950
Fire area
European GeoInformatics
Workshop
Edinburgh
Fire Spread Engine


Semi-empirical model
Estimates surface fire propagation as
function of time
• discrete process of ignitions across a regular
grid using cellular-automata algorithms

Driven by surface fuels only
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Fire Simulations
• Simulations using
different ignition points
• 20 simulation hours
(14s realtime)
• Time interval 60 minutes
• 3% dead fuel moisture
level
• 60 % live fuel moisture
elevel
• 340° wind vector azimuth
• 22 km / hr windspeed
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
SHETRAN model
Physically based, spatially distributed, catchment hydrology and
sediment transport modelling system capable of simulating:
Evapotranspiration
Canopy
interception
Root zone
3D
Variably
saturated
subsurface
flow model
7th – 9th March, 2007
SHETRAN V4
Landslides,
erosion &
sediment
Overland & transport
Snowmelt
channel


flow

Stream – aquifer interactions
European GeoInformatics Workshop
land phase of the
hydrological cycle
soil erosion and
sediment yield
arising from
raindrop impact
and overland flow
and shallow
landsliding
models catchment
up to 500 km2
using grids of
100 m to 2000 m
Edinburgh
SHETRAN simulations
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Results
DEM20m
*
#
Height (m)
120 - 280
2
%
280 - 380
Maps of burned areas
 Hydrology
 Sediment Discharge
 Erosion maps
 Landslide maps
Aids:
 Pre- and Post-fire
management

7th – 9th March, 2007
European GeoInformatics Workshop
2
%
380 - 480
480 - 580
580 - 950
Fire area
2
%
Edinburgh
Outline




7th – 9th March, 2007
Introduction
MEDIGRID System
Example application
Conclusion
European GeoInformatics Workshop
Edinburgh
Conclusion


Prototype MEDIGRID system testbed
provides a decision support
framework for fire and other natural
hazards impact assessment
Early real world example of large
scale Env. Grid Computing system
• Provides concepts, advice, philosophy
on challenges to others
• Suitable for use by other scientific
communities
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Conclusion


MEDIGRID system will serve as
platform for better collaboration
between researchers / data /
hardware / software
Improve decision making process
and outcomes of environmental
impact assessment
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh
Further Information

s.j.abele@ncl.ac.uk

http://ups.savba.sk/medigrid/

http://www.eu-medigrid.org/

Computers & Geosciences
7th – 9th March, 2007
European GeoInformatics Workshop
Edinburgh