Enabling Grids for E-sciencE
FloodGrid application
Ladislav Hluchy, Viet D. Tran
Institute of Informatics, SAS
Slovakia
www.eu-egee.org
INFSO-RI-508833
History of Flood application
Enabling Grids for E-sciencE
•
Flood application is continually developed in
– ANFAS: datA fusioN for Flood Analysis and decision Support,(2000-03) IST1999-11676
 Data fusion, hydraulic modeling
 Cluster computing
 Remote processing
– CrossGrid: Development of Grid Environment for Interactive Applications (200205) IST-2001-32243




More models (meteorology, hydrology)
Grid computing
Metadata catalog
Portal
– EGEE: Enabling Grids for E-sciencE (2004-2006) INFSO-RI-508833
 Porting to gLite
 Working in Earth Science Research Virtual Organization (ESR VO)
– K-Wf Grid: Knowledge-based Workflow System for Grid Applications (20042006) FP6-511385
 Porting to web services
 Building knowledge system for flood application
•
Collaboration with Slovak Hydro-meteorological Institute (SHMI) and
Slovak Water Research Institute (WRI)
INFSO-RI-508833
2
Flood forecasting problem
Enabling Grids for E-sciencE
Data sources
Meteorological simulation
Hydrological simulation
Hydraulic simulation
Portal
INFSO-RI-508833
3
Flood forecasting problem
Enabling Grids for E-sciencE
• Many types of simulations
– Meteorological models: MM5, ALADIN
– Hydrological models: HSPF, NLC, HEC, MIKE-1D, …
– Hydraulic models: FESWMS, DaveF, MIKE-2D, TELEMAC, …
• Many types of data
– Real-time meteorological and hydrological data from hydrometeorological stations
– Boundary conditions from ALADIN-LACE (Local Area Central
European) model from Prague
– Digital elevation models measured using LIDAR (Light Detection and
Ranging)
– River profiles and networks
– Input and output data of different simulations
• Many types of users
–
–
–
–
–
Experts in meteorology, hydrology and hydraulics
River authorities, water management authorities
Hydropower stations, farms
Governments
Public
INFSO-RI-508833
4
Flood forecasting and Grid
Enabling Grids for E-sciencE
• For critical situations
– Needs to run many scenarios at the same time (worst-case
scenarios for risk analysis)
– Needs to have results as accurate as possible (higher
resolutions for simulations -> higher computational power)
– Needs to have results as soon as possible (every minute is
important)
Needs grid computational power
• For collaborations:
– Different user groups: meteorology, hydrology, hydraulics, river
authorities, crisis team, …
– Different countries: for international rivers like Danube
– Sharing resources: data, computational powers, expertise, …
 Grid is the technology for collaborations
INFSO-RI-508833
5
Simplified scenario
Enabling Grids for E-sciencE
Portal
User
Workflow
management
Workflow
Storage
element
Meteorology
simulation
Hydrology
simulation
Meteorology
visualization
Hydraulics
simulation
Hydrology
visualization
Hydraulics
visualization
OPERATIONS IN GRID
INFSO-RI-508833
6
Full scenario
Enabling Grids for E-sciencE
INFSO-RI-508833
7
FloodGrid portal
Enabling Grids for E-sciencE
• Portal is accessible from anywhere:
– Needs only network connection and web browsers
– Users can use portal also from mobile devices (very important
for field work)
• Portal is the “meeting point” for users:
– Users need to collaborate on flood forecasting
• Portal hides complexity of Grid computing:
– The main users of the portal may be experts in meteorology,
hydrology, … but not in Grid computing
• Portal increases the security
– Some data and information from flood forecasting are sensitive
– Portal has additional security level
– Users cannot run other code than the portal allows
INFSO-RI-508833
8
FloodGrid Portal
Enabling Grids for E-sciencE
Portal
Workflow
management
Data
management
Visualization
Collaboration
Models
ALADIN
HSPF
Data
DaveF
Computational
resources
GRID
INFSO-RI-508833
9
Workflow management
Enabling Grids for E-sciencE
• Manages and executes jobs with data dependences
• Cooperates with gLite resource broker to find suitable
computing element for running simulation
• Monitors status of jobs
• Abilities to use predefined workflow templates,
spawning running workflow, modifying parameters of
jobs
INFSO-RI-508833
10
Workflow management module
Enabling Grids for E-sciencE
Portal
Workflow selection
portlet
Workflow definition
portlet
Workflow manager
Workflow execution
service
Workflow template
database
Job execution adaptors
CrossGrid
JSS
LCG
gLite
Hydrology
Hydraulic
Web service
EGEE testbed
Meteorology
INFSO-RI-508833
Visualization
11
Workflow management
Enabling Grids for E-sciencE
INFSO-RI-508833
12
Data management
Enabling Grids for E-sciencE
• Many kinds of data in FloodGrid
–
–
–
–
Meteorological, hydrological, hydraulic
Generated by simulations or obtained from sensors
Permanent or periodically updated
Publicly available or with restricted access
• Using metadata catalog for describing data
• Data are stored in storage elements and are accessed
via Grid protocols
• Operation: query, adding, modification, deleting
INFSO-RI-508833
13
Data management module
Enabling Grids for E-sciencE
Portal
Metadata query
Metadata service
Metadata
catalog
Data
identifier
(GUID, LFN)
Grid
testbed
Data access
Replica Location
Service
GridFTP
Replica 1
INFSO-RI-508833
Replica 2
Replica 3
Replica 4
14
Visualization
Enabling Grids for E-sciencE
• Multiple visualization modes according to models and
visualization tools
–
–
–
–
Texts
Pictures
Animations
Virtual reality
INFSO-RI-508833
15
Visualization
Enabling Grids for E-sciencE
INFSO-RI-508833
16
Visualization in Virtual Reality
Enabling Grids for E-sciencE
• In cooperation with GUP, Joh. Kepler University, Linz
INFSO-RI-508833
17
Collaboration
Enabling Grids for E-sciencE
• Different users groups (experts in meteorology,
hydrology, hydraulics, crisis team, river authorities)
need to collaborate on flood application
• Portal provides different means of communication
among users: chats, mailing lists, discussion groups,
file sharing
• Collaboration via other shared tools of portal
INFSO-RI-508833
18
Collaboration
Enabling Grids for E-sciencE
INFSO-RI-508833
19
Security
Enabling Grids for E-sciencE
• Two level of securities: Portal level and Grid level
• Portal level:
– Secure HTTP access
– Authentication by user names/passwords
– Access controls for every tools/resources at portal (individually or by
groups)
– Only for accessing tools/resources located on portal. For accessing
physical data, models or computing resource, Grid certificates are
needed
• Grid level:
– Authentication by Grid certificates
– Using Grid communication protocols
INFSO-RI-508833
20
Security
Enabling Grids for E-sciencE
PORTAL ACCESS LEVEL
Portal
Workflow
management
Data
management
Visualization
Collaboration
GRID ACCESS LEVEL
Models
ALADIN
HSPF
Data
DaveF
Computational
resources
GRID
INFSO-RI-508833
21
Conclusion
Enabling Grids for E-sciencE
• Flood forecasting is a complex application with many
models, data types and user groups
• FloodGrid portal has been developed with:
–
–
–
–
Workflow management module
Data management module
Visualization
Collaboration tools
• Current and future works
– New data management with improved data access controls
(using ACL support in gLite)
INFSO-RI-508833
22