Enabling Grids for E-sciencE
What is Grid Computing?
Richard Hopkins
Training Outreach and Education
National e-Science Centre
rph@nesc.ac.uk
www.eu-egee.org
INFSO-RI-508833
Contents
Enabling Grids for E-sciencE
• Introduction to
– e-Research and e-Science
– Grids
– e-Infrastructure
• Grid concepts
• Grids - Where are we now?
INFSO-RI-508833
3
Enabling Grids for E-sciencE
‘e-Science is about global collaboration
in key areas of science, and the next
generation of infrastructure that will
enable it.’
John Taylor
Director General of Research Councils
Office of Science and Technology
INFSO-RI-508833
4
Enabling Grids for E-sciencE
‘e-Science is about global collaboration
in key areas of science, and the next
generation of infrastructure that will
enable it.’
Networks + Grids
– Networks connect resources
– Grids enable “virtual computing”
INFSO-RI-508833
5
Virtual Observatories
Observations made across entire electromagnetic spectrum
ROSAT ~keV DSS Optical 2MASS 2m IRAS 25m
IRAS 100m
GB 6cm
NVSS 20cm WENSS 92cm
e.g. different views of a local galaxy
Need all of them to understand physics fully
Databases are located throughout the world
Peter Clarke
6
Biomedical Research Informatics
Delivered by Grid Enabled Services
CFG Virtual
Publically Curated Data
Ensembl
Organisation
OMIM
Glasgow
SWISS-PROT
Private
Edinburgh
MGI
VO Authorisation
Private
data
Oxford
data
Information
Integrator
HUGO
…
RGD
Leicester
DATA
HUB
Private
data
Netherlands
Synteny
Grid
Service
Private
data
Private
data
London
Private
data
+
http://www.brc.dcs.gla.ac.uk/projects/bridges/
Enabling Grids for E-sciencE
DAME: Grid based tools and Inferstructure for Aero-Engine Diagnosis
and Prognosis
Engine flight data
London Airport
Airline
office
New York Airport
•“A Significant factor in the success of the Rolls-Royce
campaign to power the Boeing 7E7 with the Trent 1000
was the emphasis on the new aftermarket support service
for the engines provided via DS&S. Boeing personnel
were shown DAME as an example of the new ways of
gathering and processing the large amounts of data that
could be retrieved from an advanced aircraft such as the
7E7, and they were very impressed”, DS&S 2004
Grid
Diagnostics Centre
Maintenance Centre
American data center
European data center
XTO
Companies:
Rolls-Royce
DS&S
Cybula
Universities:
York,
Leeds,
Sheffield, Oxford
Engine Model
Case Based Reasoning
Follow-on project: BROADEN
INFSO-RI-508833
Signal Data Explorer
8
climateprediction.net and GENIE
• Largest climate model
ensemble
• >45,000 users, >1,000,000
model years
Response of Atlantic
circulation to freshwater
forcing
2K
10K
UK Grid for
Particle Physics
GridPP www.gridpp.ac.uk
ATLAS detectors, 2/3/06
Connecting people: Access Grid
Enabling Grids for E-sciencE
http://www.accessgrid.org/
Cameras
Microphones
INFSO-RI-508833
11
What is e-Research?
Enabling Grids for E-sciencE
• Collaborative research that is made possible by the
sharing across the Internet of resources (data,
instruments, computation, people’s expertise...)
–
–
–
–
Crosses organisational boundaries
Often very compute intensive
Often very data intensive
Sometimes large-scale collaboration
• Began with focus in the “big sciences” hence
initiatives are often badged as “e-science”
• Relevance of “e-science technologies” to new user
communities (social science, arts, humanities…) led to
the term “e-research”
INFSO-RI-508833
12
e-Research and Grids
Enabling Grids for E-sciencE
Collaborative
“virtual computing”
Improvised cooperation
People with shared goals
INFSO-RI-508833
Sharing data, computers, software
Enabled by Grids:
National, regional
International: EGEE grid
Email
File exchange
ssh access to run programs
Enabled by networks:
national, regional and
International: GEANT
13
Grids: a foundation for e-Research
enabling a whole-system approach
computers
software
Grid
sensor nets
instruments
Diagram derived from
Ian Foster’s slide
colleagues
Shared data
archives
What is Grid Computing?
Enabling Grids for E-sciencE
• The grid vision is of “Virtual
computing” (+ information
services to locate computation,
storage resources)
– Compare: The web: “virtual
documents” (+ search engine
to locate them)
• MOTIVATION: collaboration
through sharing resources
(and expertise) to expand
horizons of
– Research
– Commerce – engineering, …
– Public service – health,
environment,…
INFSO-RI-508833
15
The Grid Metaphor
Enabling Grids for E-sciencE
Mobile Access
G
R
I
D
Workstation
M
I
D
D
L
E
W
A
R
E
Supercomputer, PC-Cluster
Data-storage, Sensors, Experiments
Visualising
Internet, networks
INFSO-RI-508833
16
What is e-Infrastructure? – Political
view
Enabling Grids for E-sciencE
• A shared resource
– That enables science,
research, engineering,
medicine, industry, …
– It will improve UK / European /
… productivity
 Lisbon Accord 2000
 E-Science Vision SR2000 –
John Taylor
– Commitment by UK
government
 Sections 2.23-2.25
– Always there
 c.f. telephones,
transport, power, internet
INFSO-RI-508833
17
What is e-Infrastructure?
Enabling Grids for E-sciencE
Grids: permit resource sharing
across administrative domains
•
Networks: permit
communication across
geographical distance
•
Supporting organisations
Collaboration
– Operations for grids, networks
•
Resources
–
–
–
–
•
Computers
Digital libraries
Research data
Instruments
Middleware
– Authentication, Authorisation
– Registries, search engines
– Toolkits, environments
 E.g. for collaboration
INFSO-RI-508833
Grid
Operations, Support and
training
•
Network
infrastructure
& Resources
18
Enabling Grids for E-sciencE
Grid concepts
INFSO-RI-508833
19
Virtual organisations and grids
Enabling Grids for E-sciencE
• What’s a Virtual Organisation?
– People in different organisations seeking to cooperate and share
resources across their organisational boundaries E.g. A research
collaboration
• Each grid is an infrastructure enabling one or more “virtual
organisations” to share and access resources
• Key concept: The ability to negotiate resource-sharing
arrangements among a set of participating parties (providers
and consumers) and then to use the resulting resource pool
for some purpose. (Ian Foster)
INFSO-RI-508833
20
Typical current grid
Enabling Grids for E-sciencE
• Virtual organisations
negotiate with sites to
agree access to
resources
• Grid middleware runs
on each shared
resource to provide
INTERNET
– Data services
– Computation
services
– Single sign-on
• Distributed services
(both people and
middleware) enable
the grid
INFSO-RI-508833
21
Empowering VO’s
Enabling Grids for E-sciencE
Application
Application
toolkits, …
Higher-level grid
services (brokering,…)
Basic Grid services:
AA, job submission, info, …
Where computer science meets
the application communities!
VO-specific developments:
– Portals
– Virtual Research
Environments
– Semantics, ontologies
– Workflow
– Registries of VO services
Production grids provide these
services.
INFSO-RI-508833
22
Example – Biomedical applications
Enabling Grids for E-sciencE
Biomedical community and the Grid, EGEE User Forum, March 1st 2006, I. Magnin
INFSO-RI-508833
23
Workflow example
Enabling Grids for E-sciencE
• Taverna in MyGrid http://www.mygrid.org.uk/
• “allows the e-Scientist to describe and enact their experimental
processes in a structured, repeatable and verifiable way”
• GUI
• Workflow
language
• enactment engine
INFSO-RI-508833
24
The many scales of grids
Enabling Grids for E-sciencE
International instruments,..
National datacentres,
HPC, instruments
Institutes’ data;
Wider collaboration
greater resources
International grid (EGEE)
National grids (e.g.
National Grid Service)
Regional grids (e.g.
White Rose Grid)
Campus grids
Condor pools,
clusters
Desktop
INFSO-RI-508833
25
Grid security and trust -1
Enabling Grids for E-sciencE
• Providers of resources (computers, databases,..) need risks to
be controlled: they are asked to trust users they do not know
– They trust a VO
– The VO trusts its members
• User’s need
– single sign-on: to be able to logon to a machine that can pass the
user’s identity to other resources
– To trust owners of the resources they are using
• Build middleware on layer providing:
– Authentication: know who wants to use resource
– Authorisation: know what the user is allowed to do
– Security: reduce vulnerability, e.g. from outside the firewall
– Non-repudiation: knowing who did what
• The “Grid Security Infrastructure” middleware is the basis of
(most) production grids
INFSO-RI-508833
26
Grid security and trust -2
Enabling Grids for E-sciencE
• Achieved by Certification:
– User’s identity has to be certified by one of the national
Certification Authorities (CAs)
 mutually recognized http://www.gridpma.org/
– In UK go to http://www.grid-support.ac.uk/ca/ralist.htm to
find CA’s local “Registration Authorities”
– Resources are also certified by CAs
• User
– User joins a VO
– Digital certificate is basis of AA
– Identity passed to resources you use, where it is mapped to a
local account
• Policies express the rights for a Virtual Organization
to use resources
INFSO-RI-508833
27
Enabling Grids for E-sciencE
Grids – Where are we Now?
INFSO-RI-508833
28
Enabling Grids for E-sciencE
If “The Grid”
vision leads us
here…
… then where are
we now?
INFSO-RI-508833
29
Grids: where are we now?
Enabling Grids for E-sciencE
• Many key concepts identified and known
• Many grid projects have tested, and benefit from, these
• Major efforts now on establishing:
– Production Grids for multiple VO’s
 “Production” = Reliable, sustainable, with commitments to quality of
service
• In Europe, EGEE
• In UK, National Grid Service
• In US, Teragrid and OSG
 One stack of middleware that serves many research communities
 Establishing operational procedures and organisation
– Standards (a slow process)
(e.g. Open Grid Forum, http://www.gridforum.org/ )
• Service orientation - “the way to build grids”
INFSO-RI-508833
30
Where are we now? –user’s view
Enabling Grids for E-sciencE
Research
Pilot
projects
Early
adopters
Routine
production
Unimagined
possibilities
Networks
Grids
Web
Arts
Sciences,
Humanities
engineering
e-Soc-Sci
Early production grids:
UK – National Grid
Service
International - EGEE
INFSO-RI-508833
31
National grid initiatives now include…
Enabling Grids for E-sciencE
CroGrid
INFSO-RI-508833
33
Summary: what is grid computing?
Enabling Grids for E-sciencE
• Grids: virtual computing
across administrative
domains
– Data
– Computation
– Collaboration
• Orchestration of
services in support of
– Research, diagnostics,
engineering, public
service,..
– Resource utilisation and
sharing
INFSO-RI-508833
Grid
Operations, Support and
training
Collaboration
Network
infrastructure
& Resource
centres
34
Further reading
Enabling Grids for E-sciencE
•
•
•
•
•
•
Open Grid Forum http://www.ogf.org/
National e-Science Centre http://www.nesc.ac.uk
UK All Hands Meeting http://www.allhands.org.uk/
National Grid Service http://www.ngs.ac.uk
EGEE www.eu-egee.org
The Grid Cafe www.gridcafe.or
• The Grid Core Technologies, Maozhen Li and Mark
Baker, Wiley, 2005
INFSO-RI-508833
35