Current Grids
Dave Berry
Research Manager, NeSC
EGEE is funded by the European Union under contract IST-2003-508833
BNCOD: Current Grids – July 6th, 2004 - 1
Acknowledgements
• This talk includes slides from previous tutorials and talks
delivered by:
•
•
•
•
the National e-Science Centre
the Condor team
the Globus Alliance
Andrew Grimshaw (U. of Virginia)
• Prepared by Dave Berry, NeSC
BNCOD: Current Grids – July 6th, 2004 - 2
1
Overview
•
•
•
•
•
•
•
•
The Grid metaphor
Cycle stealing Grids
Cluster management
Data Grids
Metacomputing Grids
Convergence and co-operation
Portals
A sample infrastructure
BNCOD: Current Grids – July 6th, 2004 - 3
The Grid Metaphor
Mobile Access
G
R
I
D
Workstation
M
I
D
D
L
E
W
A
R
E
Supercomputer, PC-Cluster
Data-storage, Sensors, Experiments
Visualising
Internet, networks
BNCOD: Current Grids – July 6th, 2004 - 4
2
Cycle stealing Grids
• Use idle CPU cycles for productive work
• This slide shows a Condor system
BNCOD: Current Grids – July 6th, 2004 - 5
A popular example: SETI@Home
Collect data
Find candidate
signals
Check data
integrity
Remove Radio
Interference
1997: Entropia
1999: United Devices
Identify Final
Candidates
BNCOD: Current Grids – July 6th, 2004 - 6
3
Cluster management
• Cluster: off-the-shelf processors linked to provide a
high-capacity computing resource
• Cluster management: scheduling jobs onto free processors
•
•
Some similarities to cycle stealing
Some solutions based on Condor
• Example systems
•
•
•
•
•
Platform LSF
NASA/Veridian PBS
Sun Grid Engine
IBM LoadLeveller
Nimrod
BNCOD: Current Grids – July 6th, 2004 - 7
Data Grids
Data Grid Capabilities
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
Federates multiple data
sources
Provides global naming
Works with local and
virtual file systems –
NFS, XFS, CIFS
Accesses data in DAS,
NAS, SAN
Uses standard interfaces
Caches data locally
Server
Data
Partner
Application
Users
Applications
Legion G R I D
Wide-area access to data at its
source location based on business
policies, eliminating manual copying
and errors caused by accessing
out-of-date copies
Server
Data
Department A
Desktop Server
Data
Department B
Cluster
Application
Vendor
BNCOD: Current Grids – July 6th, 2004 - 8
4
Metacomputing Grids
Site Resources
26
HPSS
4
Site Resources
HPSS
24
External
Networks
8
Caltech
SDSC
4.1 TF
225 TB
HPSS
5
Argonne
External
Networks
External
Networks
Site Resources
External
Networks
NCSA/PACI
8 TF
240 TB
Site Resources
UniTree
BNCOD: Current Grids – July 6th, 2004 - 9
1998: “The Grid”
• Various Toolkits
•
•
•
Distribution
Various Protocols
FTP
• Security
•
Single Sign on
• Resource Sharing
•
•
•
Discovery
Process Creation
Scheduling
• Portability
•
APIs
• Government Agency Buy in
BNCOD: Current Grids – July 6th, 2004 - 10
5
The Globus Toolkit (v2)
• Grid Security Infrastructure (GSI)
•
X.509 authentication with delegates and single sign-on
• Grid Resource Allocation Mgmt (GRAM)
•
Remote allocation, reservation, monitoring, control of compute
resources
• GridFTP protocol (FTP extensions)
•
High-performance data access & transport
• Grid Resource Information Service (GRIS) +
Monitoring and Discovery Service (MDS)
•
Access to structure & state information
• XIO
•
TCP, UDP, IP multicast, and file I/O
BNCOD: Current Grids – July 6th, 2004 - 11
Convergence and co-operation
600 Condor
jobs
personal
yourPool
Condor
workstation
Condor
Friendly Condor Pool
PBS
LSF
Condor
Globus Grid
BNCOD: Current Grids – July 6th, 2004 - 12
6
Portals: Browser interfaces to Grid systems
https+java/xml+rfb
WEB Browser
GENIUS
Local
WS
EnginFrame
Apache
EDG
UI
EDG+GSI
the Grid
Roberto Barbera
BNCOD: Current Grids – July 6th, 2004 - 13
UK e-Science Grid
Globus Alliance
e-Science
Institute
Guaranteed
resources
HPC(x)
Digital
Curation
Centre
Grid
Operations
& Support
Centre
CeSC (Cambridge)
Open
Middleware
Infrastructure
Institute
www.nesc.ac.uk
BNCOD: Current Grids – July 6th, 2004 - 14
7
Unfinished business
•
•
•
•
•
•
•
•
•
Provisioning
Deployment, configuration and update
Resource usage and billing
Workflow description and enactment
Resource description and discovery
Workflow, reservation and advanced scheduling
Quality of Service specification and assurance
Security, trust and provenance
…
BNCOD: Current Grids – July 6th, 2004 - 15
Questions?
BNCOD: Current Grids – July 6th, 2004 - 16
8