The Developer Perspective
GEMEDA
Grid-Enabled Microeconometric Data Analysis
Design
•
initial plan was to extend an existing
demonstrator that had grid-enabled a related
problem
•
start point: review of current technologies
•
ad-hoc requirements capture: dialogue
between developers and users
– regular meeting of the full project team
– one-to-one meetings/communication for specific
issues : data, modelling, presentation
Technical Strategy
• Hitch onto established trends & ongoing
investments:
Web service technologies (portals avoid local
deployment of complex middleware stacks)
OGSA-DAI
NGS Oracle for databases, MPI for parallelism
The establishment of a critical mass of
compatible Grid-enabled tools is a
necessary condition for success of
e-Social Science in the UK
Development Issues
• small scale project with funding, time and
scheduling constraints
• e-Science research staff recruitment
• reliability, immaturity of documentation, and error
reporting for middleware services (esp. NGS)
• software re-use (human capital)
– Athens authentication code from ConvertGrid
– GeoTools (java mapping library) from Hydra I Grid
• dialogue between developer and user (e.g.
what is a variable?)
Lessons Learnt
• development is hostage to staffing issues
• what the user(s) want isn't necessarily what
they need
• specialist elements of the service are often the
easiest (in a relative sense) to develop
– parallel code
– visualization displays.
• linking everything together is the difficult bit
Software Used:
Linux Mandkrake 10.1
Java 1.4.2_06
Tomcat 5.0.28
Globus Toolkit 3.2
OGSA-DAI 4
Java Cog Kit 4
Spring J2EE application framework 1.2.3
DWR Ajax library
ProFTP
Oracle 9i RAC edition
Python 2.3 (dataset creation &
wrapper to second Java
instance & Cog Kit code)
Eclipe IDE
Fortran95+MPI
C (utiliies for converting output for use with the map)
GeoTools (Java library for mapping)
Future Plans
• The project has now concluded
– Ensuring long term sustainability is a problem
– The plan is for the service/portal to become part of
the NCeSS demonstrator suite.
– Re-engineering may be required.
(e.g. for Globus WSRF compliance)
NGS Oracle hosted datasets
NGS HPC computing nodes
Manchester, Oxford, Leeds, CCLRC RAL
BHPS
GEMEDA parallelized
SARs
SARs
Household
Individual
analysis code grid service
NGS MyProxy server
OGSA-DAI
Grid Security Infrastructure
DAISGR
Globus Toolkit Core
(DAI Service Group Registry)
GEMEDA Architecture Diagram
Axis
Athens Server
Apache Tomcat
GEMEDA
FTP server
Apache Tomcat
Spring
Framework
JSP Web interface +
Results visualization applet