Grid Computing (3)
(Special Topics in Computer Engineering)
Veera Muangsin
13 February 2004
1
Outline
•
•
•
•
High-Performance Computing
Grid Computing
Grid Applications
Grid Architecture
• Grid Middleware
• Grid Services
2
Before the Grid
User
Application
The User is responsible for
resolving the complexities of
the environment
Network
Site A
• independent sites
• independent
hardware and
software
• independent user
ids
• security policy
requiring local
connection to the
machine.
Site B
3
First Step to the Grid
Metacenter
User
Application
A layer of abstraction is added that hides some of
the complexities associated with running jobs in a
distributed computing environment, however,
limitations exist
Network
Centralized Scheduler and file staging
Site A
Site B
• Two or more
resources
connected in a
controlled user
environment
Constraints
• common
architecture
• single name
space
• common
scheduler
4
The Grid Today
1 Request info from
the grid
2 Get response
3 Make selection and
submit job
User
Application
1
2
3
The underlying infrastructure is abstracted into
Middleware
defined APIsGrid
thereby
simplifying developer and the
user access to resources,
however, this layer is not
Infrastructure
intelligent
Network
Site A
Common Middleware
- abstracts
independent,
hardware, software,
user ids, into a
service layer with
defined APIs
- comprehensive
security,
- allows for site
autonomy
- provides a common
infrastructure based
on middleware
Site B
5
The Near Future Grid
User
Application
Resources are accessed via various
intelligent services that access
Intelligent,
Customized Middleware
infrastructure APIs
Grid Middleware - Infrastructure APIs
The result: The (service
Scientist
oriented) and Application
Developer can Infrastructure
focus on science and not
on systems
management
Network
Site A
Customizable Grid
Services built on
defined Infrastructure
APIs
• automatic selection
of resources
• information products
tailored to users
• accountless
processing
• flexible interface:
web based, command
line, APIs
Site B
6
Layered Grid Architecture
(By Analogy to Internet Architecture)
“Coordinating multiple resources”:
ubiquitous infrastructure services,
app-specific distributed services
Collective
Application
“Sharing single resources”:
negotiating access, controlling use
Resource
“Talking to things”: communication
(Internet protocols) & security
Connectivity
Transport
Internet
“Controlling things locally”: Access
to, & control of, resources
Fabric
Link
7
Internet Protocol Architecture
Application
Grid Components
Applications and Portals
Scientific
Engineering
Collaboration
…
Prob. Solving Env.
Development Environments and Tools
Languages
Libraries
Debuggers
Monitoring
Resource Brokers
Web enabled Apps
…
Distributed Resources Coupling Services
Comm.
Sign on & Security
Information
Process
Data Access
Web tools
…
QoS
Grid
Apps.
Grid
Tools
Grid
Middleware
Local Resource Managers
Operating Systems
Computers
Queuing Systems
Clusters
Libraries & App Kernels
Networked Resources across
Organisations
Storage Systems
Data Sources
…
…
TCP/IP & UDP
Scientific Instruments
Grid
Fabric
Example:
High-Throughput Computing System
API
SDK
App
High Throughput Computing System
Collective Dynamic checkpoint,
(App)
failover, staging
job management,
C-point
Protocol
Checkpoint
Repository
Collective
Brokering, certificate authorities
(Generic)
API
Resource Access to data, access to computers,
access to network performance data
Connect Communication, service discovery (DNS),
authentication, authorization, delegation
Fabric Storage systems, schedulers
SDK
Access
Protocol
Compute
Resource
9
Example:
Data Grid Architecture
App
Discipline-Specific Data Grid Application
Collective Coherency control, replica selection, task management,
virtual data catalog, virtual data code catalog, …
(App)
Collective Replica catalog, replica management, co-allocation,
(Generic) certificate authorities, metadata catalogs,
Resource
Access to data, access to computers, access to network
performance data, …
Communication, service discovery (DNS),
Connect authentication, authorization, delegation
Fabric Storage systems, clusters, networks, network caches, …
10
Globus Toolkit
• Grid computing middleware
– Software between the hardware and high-level
services
– Basic libraries, services, command-line programs
• Most common middleware used in grids
• Integrated with Web Service
11
Globus Software Architecture
•login
•execute commands
•copy files
information about
resources and services
Monitoring and
Discovery Service
(MDS)
LDAP
distributed
directory service
•get and put files
•3rd party copy
•interactive file
management
•parallel transfers
Grid
Grid FTP
SSH
Grid Security Infrastructure
(GSI)
X.509 Certificates
SSL/TLS
credentials for
users, services,
hosts
•execute remote
applications
•stage executable, stdin,
stdout, stderr
Globus Resource Allocation
Manager (GRAM)
PBS
LSF
fork/exe
c
job management
systems
•authentication
•secure communication •single sign on
•delegation of
credentials
•authorization
12
Globus Deployment Architecture
User
Globus
client
system
Grid FTP
Client
MDS
server
system
Grid FTP
Server
Globus
server
system
User
Web portal
application/tool
GRAM
Grid SSH
MDS
Client
Client
Client
Clients are
programs and
libraries
MDS GIIS
GRAM
Server
Grid SSH
Server
Grid SSH
Server
GRAM
Server
PBS
MDS
GRIS
MDS
GRIS
LSF
Grid FTP
Server
Globus
server
system
13
Globus Toolkit™
• A software toolkit addressing key technical
problems in the development of Grid enabled
tools, services, and applications
– Offer a modular “bag of technologies”
– Enable incremental development of grid-enabled
tools and applications
– Implement standard Grid protocols and APIs
– Make available under liberal open source license
14
General Approach
• Define Grid protocols & APIs
– Protocol-mediated access to remote resources
– Integrate and extend existing standards
– “On the Grid” = speak “Intergrid” protocols
• Develop a reference implementation
– Open source Globus Toolkit
– Client and server SDKs, services, tools, etc.
• Grid-enable wide variety of tools
– Globus Toolkit, FTP, SSH, Condor, SRB, MPI, …
15
Four Key Protocols
• The Globus Toolkit™ centers around four
key protocols
– Connectivity layer:
• Security: Grid Security Infrastructure (GSI)
– Resource layer:
• Resource Management: Grid Resource Allocation
Management (GRAM)
• Information Services: Grid Resource Information
Protocol (GRIP)
• Data Transfer: Grid File Transfer Protocol
(GridFTP)
16
The Globus Toolkit™:
Security Services
The Globus Project™
Argonne National Laboratory
USC Information Sciences Institute
http://www.globus.org
17
Why Grid Security is Hard
• Resources are often located in distinct administrative
domains
– Each resource has own policies & procedures
• Set of resources used by a single computation may be
large, dynamic, and unpredictable
– Not just client/server, requires delegation
• It must be broadly available & applicable
– Standard, well-tested, well-understood protocols; integrated
with wide variety of tools
18
Grid Security Infrastructure (GSI)
• Extensions to standard protocols & APIs
– Standards: SSL/TLS, X.509 & CA, GSS-API
– Extensions for single sign-on and delegation
• Globus Toolkit reference implementation of GSI
– SSLeay/OpenSSL + GSS-API + SSO/delegation
– Tools and services to interface to local security
• Simple ACLs; SSLK5/PKINIT for access to K5, AFS; …
– Tools for credential management
•
•
•
•
Login, logout, etc.
Smartcards
MyProxy: Web portal login and delegation
K5cert: Automatic X.509 certificate creation
19
GSI in Action
“Create Processes at A and B that Communicate & Access Files at C”
User
Single sign-on via “grid-id”
& generation of proxy cred.
User Proxy
Proxy
credential
Or: retrieval of proxy cred.
from online repository
Remote process
creation requests*
GSI-enabled Authorize
Site A
GRAM server Map to local id
(Kerberos)
Create process
Generate credentials
Computer
Process
Kerberos
ticket
Communication*
Local id
Restricted
proxy
Ditto
Remote file
access request*
* With mutual authentication
Site C
(Kerberos)
Storage
system
GSI-enabled
GRAM server Site B
(Unix)
Computer
Process
Local id
Restricted
proxy
GSI-enabled
FTP server
Authorize
Map to local id
Access file
20
Review of
Public Key Cryptography
• Asymmetric keys
– A private key is used to encrypt data.
– A public key can decrypt data encrypted with the
private key.
• An X.509 certificate includes…
– Someone’s subject name (user ID)
– Their public key
– A “signature” from a Certificate Authority (CA) that:
• Proves that the certificate came from the CA.
• Vouches for the subject name
• Vouches for the binding of the public key to the subject
21
Public Key Based Authentication
• User sends certificate over the wire.
• Other end sends user a challenge string.
• User encodes the challenge string with private key
– Possession of private key means you can authenticate as
subject in certificate
• Public key is used to decode the challenge.
– If you can decode it, you know the subject
• Treat your private key carefully!!
– Private key is stored only in well-guarded places, and only
in encrypted form
22
User Proxies
• Minimize exposure of user’s private key
• A temporary, X.509 proxy credential for use
by our computations
–
–
–
–
We call this a user proxy certificate
Allows process to act on behalf of user
User-signed user proxy cert stored in local file
Created via “grid-proxy-init” command
• Proxy’s private key is not encrypted
– Rely on file system security, proxy certificate file
must be readable only by the owner
23
Delegation
• Remote creation of a user proxy
• Results in a new private key and X.509
proxy certificate, signed by the original key
• Allows remote process to act on behalf of
the user
• Avoids sending passwords or private keys
across the network
24
GSI Applications
• Globus Toolkit™ uses GSI for authentication
• Many Grid tools, directly or indirectly, e.g.
– Condor-G, SRB, MPICH-G2, Cactus, GDMP, …
• Commercial and open source tools, e.g.
– ssh, ftp, cvs, OpenLDAP, OpenAFS
– SecureCRT (Win32 ssh client)
• And since we use standard X.509 certificates,
they can also be used for
– Web access, LDAP server access, etc.
25
The Globus Toolkit™:
Resource Management Services
The Globus Project™
Argonne National Laboratory
USC Information Sciences Institute
http://www.globus.org
26
The Challenge
• Enabling secure, controlled remote access to
heterogeneous computational resources and
management of remote computation
–
–
–
–
Authentication and authorization
Resource discovery & characterization
Reservation and allocation
Computation monitoring and control
• Addressed by new protocols & services
– GRAM protocol as a basic building block
– Resource brokering & co-allocation services
– GSI for security, MDS for discovery
27
Resource Management
• The Grid Resource Allocation Management
(GRAM) protocol and client API allows
programs to be started on remote resources,
despite local heterogeneity
• Resource Specification Language (RSL) is
used to communicate requirements
• A layered architecture allows applicationspecific resource brokers and co-allocators to be
defined in terms of GRAM services
– Integrated with Condor, PBS, MPICH-G2, …
28
Resource Management Architecture
RSL
specialization
Broker
RSL
Queries
& Info
Application
Information
Service
Ground RSL
Co-allocator
Local
resource
managers
GRAM
LSF
Simple ground RSL
GRAM
Condor
GRAM
NQE
29
Globus Toolkit Implementation
• Gatekeeper
– Single point of entry
– Authenticates user, maps to local security
environment, runs service
– In essence, a “secure inetd”
• Job manager
– A gatekeeper service
– Layers on top of local resource management
system (e.g., PBS, LSF, etc.)
– Handles remote interaction with the job
30
GRAM Components
MDS client API calls
to locate resources
Client
MDS: Grid Index Info Server
Site boundary
MDS client API calls
to get resource info
GRAM client API calls to
MDS:
request resource allocation
and process creation.
GRAM client API state
change callbacks
Grid Security
Grid Resource Info Server
Query current status
of resource
Local Resource Manager
Infrastructure
Request
Create
Gatekeeper
Job Manager
Parse
RSL Library
Monitor &
control
Allocate &
create processes
Process
Process
Process
31
Job Submission Interfaces
• Globus Toolkit includes several command
line programs for job submission
– globus-job-run: Interactive jobs
– globus-job-submit: Batch/offline jobs
– globusrun: Flexible scripting infrastructure
• Others are building better interfaces
– General purpose
• Condor-G, PBS, GRD, Hotpage, etc
– Application specific
• ECCE’, Cactus, Web portals
32
The Globus Toolkit™:
Information Services
The Globus Project™
Argonne National Laboratory
USC Information Sciences Institute
http://www.globus.org
33
Grid Information Services
• System information is critical to operation
of the grid and construction of applications
– What resources are available?
• Resource discovery
– What is the “state” of the grid?
• Resource selection
– How to optimize resource use
• Application configuration and adaptation?
• We need a general information
infrastructure to answer these questions
34
Examples of Useful Information
• Characteristics of a compute resource
– IP address, software available, system
administrator, networks connected to, OS
version, load
• Characteristics of a network
– Bandwidth and latency, protocols, logical
topology
• Characteristics of the Globus infrastructure
– Hosts, resource managers
35
Grid Information: Facts of Life
• Information is always old
– Time of flight, changing system state
– Need to provide quality metrics
• Distributed state hard to obtain
– Complexity of global snapshot
• Component will fail
• Scalability and overhead
• Many different usage scenarios
– Heterogeneous policy, different information
organizations, etc.
36
Grid Information Service
• Provide access to static and dynamic
information regarding system components
• A basis for configuration and adaptation in
heterogeneous, dynamic environments
• Requirements and characteristics
–
–
–
–
Uniform, flexible access to information
Scalable, efficient access to dynamic data
Access to multiple information sources
Decentralized maintenance
37
The GIS Problem: Many
Information Sources, Many Views
R
R
?
R
VO C
R
R
R
R
?
R
VO A
R
R
?
R
R
R
R
?
R
R
R
VO B
38
Information Protocols
• Grid Resource Registration Protocol
– Support information/resource discovery
– Designed to support machine/network failure
• Grid Resource Inquiry Protocol
– Query resource description server for
information
– Query aggregate server for information
– LDAP V3.0 in Globus 1.1.3
39
GIS Architecture
Customized Aggregate Directories
Users
Enquiry
A
A
Protocol
Registration
Protocol
R
R
R
R
Standard Resource Description Services
40
Metacomputing Directory Service
• Use LDAP as Inquiry
• Access information in a distributed directory
– Directory represented by collection of LDAP servers
– Each server optimized for particular function
• Directory can be updated by:
– Information providers and tools
– Applications (i.e., users)
– Backend tools which generate info on demand
• Information dynamically available to tools and
applications
41
Two Classes Of MDS Servers
• Grid Resource Information Service (GRIS)
– Supplies information about a specific resource
– Configurable to support multiple information providers
– LDAP as inquiry protocol
• Grid Index Information Service (GIIS)
– Supplies collection of information which was gathered from
multiple GRIS servers
– Supports efficient queries against information which is
spread across multiple GRIS server
– LDAP as inquiry protocol
42
Grid Resource Information Service
• Server which runs on each resource
– Given the resource DNS name, you can find the GRIS server
(well known port = 2135)
• Provides resource specific information
– Much of this information may be dynamic
• Load, process information, storage information, etc.
• GRIS gathers this information on demand
• “White pages” lookup of resource information
– Ex: How much memory does machine have?
• “Yellow pages” lookup of resource options
– Ex: Which queues on machine allows large jobs?
43
Grid Index Information Service
• GIIS describes a class of servers
– Gathers information from multiple GRIS servers
– Each GIIS is optimized for particular queries
• Ex1: Which Alliance machines are >16 process SGIs?
• Ex2: Which Alliance storage servers have >100Mbps bandwidth to
host X?
– Akin to web search engines
• Organization GIIS
– The Globus Toolkit ships with one GIIS
– Caches GRIS info with long update frequency
• Useful for queries across an organization that rely on relatively
static information (Ex1 above)
• Can be merged into GRIS
44
Logical MDS Deployment
Grads
Gusto
GIIS
ISI
GRISes
45
Example: Discovering CPU Load
• Retrieve CPU load fields of compute resources
% grid-info-search -L “(objectclass=GlobusComputeResource)” \
dn cpuload1 cpuload5 cpuload15
dn: hn=lemon.mcs.anl.gov, ou=MCS, o=Argonne National Laboratory,
o=Globus, c=US
cpuload1: 0.48
cpuload5: 0.20
cpuload15: 0.03
dn: hn=tuva.mcs.anl.gov, ou=MCS, o=Argonne National Laboratory,
o=Globus, c=US
cpuload1: 3.11
cpuload5: 2.64
cpuload15: 2.57
46
The Globus Toolkit™:
Data Management Services
The Globus Project™
Argonne National Laboratory
USC Information Sciences Institute
http://www.globus.org
47
Data Intensive Issues Include …
• Harness [potentially large numbers of] data,
storage, network resources located in
distinct administrative domains
• Respect local and global policies governing
what can be used for what
• Schedule resources efficiently, again subject
to local and global constraints
• Achieve high performance, with respect to
both speed and reliability
• Catalog software and virtual data
48
Desired Data Grid Functionality
•
•
•
•
•
•
•
High-speed, reliable access to remote data
Automated discovery of “best” copy of data
Manage replication to improve performance
Co-schedule compute, storage, network
“Transparency” wrt delivered performance
Enforce access control on data
Allow representation of “global” resource
allocation policies
49
A Model Architecture for Data Grids
Metadata
Catalog
Attribute
Specification
Application
Logical Collection and
Logical File Name
Selected
Replica
Replica
Selection
MDS
NWS
Disk Cache
Tape Library
Disk Array
Replica Location 1
Multiple Locations
Performance
Information &
Predictions
GridFTP Control Channel
GridFTP
Data
Channel
Replica
Catalog
Disk Cache
Replica Location 2
Replica Location 3
50
Globus Toolkit Components
Two major Data Grid components:
1. Data Transport and Access
 Common protocol
 Secure, efficient, flexible, extensible data movement
 Family of tools supporting this protocol
2. Replica Management Architecture
 Simple scheme for managing:
 multiple copies of files
 collections of files
51
Access/Transport Protocol Requirements
• Suite of communication libraries and related tools
that support
–
–
–
–
–
–
–
GSI, Kerberos security
Third-party transfers
Parameter set/negotiate
Partial file access
Reliability/restart
Large file support
Data channel reuse
– Integrated instrumentation
– Loggin/audit trail
– Parallel transfers
– Striping (cf DPSS)
– Policy-based access control
– Server-side computation
– Proxies (firewall, load bal)
• All based on a standard, widely deployed protocol
52
And The Protocol Is … GridFTP
• Why FTP?
– Ubiquity enables interoperation with many commodity
tools
– Already supports many desired features, easily extended
to support others
– Well understood and supported
• We use the term GridFTP to refer to
– Transfer protocol which meets requirements
– Family of tools which implement the protocol
• Note GridFTP > FTP
• Note that despite name, GridFTP is not restricted to
file transfer!
53
GridFTP: Basic Approach
• FTP protocol is defined by several IETF RFCs
• Start with most commonly used subset
– Standard FTP: get/put etc., 3rd-party transfer
• Implement standard but often unused features
– GSS binding, extended directory listing, simple
restart
• Extend in various ways, while preserving
interoperability with existing servers
– Striped/parallel data channels, partial file, automatic
& manual TCP buffer setting, progress monitoring,
54
extended restart
Replica Management
• Maintain a mapping between logical names
for files and collections and one or more
physical locations
• Important for many applications
– Example: CERN HLT data
•
•
•
•
•
Multiple petabytes of data per year
Copy of everything at CERN (Tier 0)
Subsets at national centers (Tier 1)
Smaller regional centers (Tier 2)
Individual researchers will have copies
55
Replica Catalog Structure:
A Climate Modeling Example
Replica Catalog
Logical Collection
Logical Collection
C02 measurements 1998
C02 measurements 1999
Filename: Jan 1998
Filename: Feb 1998
…
Location
Location
jupiter.isi.edu
sprite.llnl.gov
Filename: Mar 1998
Filename: Jun 1998
Filename: Oct 1998
Protocol: gsiftp
UrlConstructor:
gsiftp://jupiter.isi.edu/
nfs/v6/climate
Filename: Jan 1998
…
Filename: Dec 1998
Protocol: ftp
UrlConstructor:
ftp://sprite.llnl.gov/
pub/pcmdi
Logical
File Parent
Logical File
Logical File
Jan 1998
Feb 1998
Size: 1468762
56
Replica Catalog Services
as Building Blocks: Examples
• Combine with information service to build
replica selection services
– E.g. “find best replica” using performance info
from NWS and MDS
– Use of LDAP as common protocol for info and
replica services makes this easier
• Combine with application managers to build
data distribution services
– E.g., build new replicas in response to frequent
accesses
57
Replica Catalog Directions
• Many data grid applications do not require
tight consistency semantics
– At any given time, you may not be able to
discover all copies
– When a new copy is made, it may not be
immediately recognized as available
• Allows for much more scalable design
– Distributed catalogs: local catalogs which
maintain their own LFN -> PFN mapping
– Soft-state updates as basis for building various
58
configurations of global catalogs
Virtual Data in Action
• Data request may




Major Archive
Facilities
Access local data
Compute locally
Compute remotely
Access remote data
Network caches &
regional centers
• Scheduling subject to
local & global policies
• Local autonomy
Local
sites
?
59
Evolution of Grid Technologies
• Initial exploration (1996-1999; Globus 1.0)
– Extensive appln experiments; core protocols
• Data Grids (1999-??; Globus 2.0+)
– Large-scale data management and analysis
• Open Grid Services Architecture (2001-??, Globus 3.0)
– Integration w/ Web services, hosting environments, resource
virtualization
– Databases, higher-level services
• Radically scalable systems (2003-??)
– Sensors, wireless, ubiquitous computing
60
Increased functionality,
standardization
Grids and Open Standards
App-specific
Services
Web services
X.509,
LDAP,
FTP, …
Custom
solutions
Open Grid
Services Arch
GGF: OGSI, …
(+ OASIS, W3C)
Globus Toolkit Multiple implementations,
including Globus Toolkit
Defacto standards
GGF: GridFTP, GSI
Time
61
“Web Services”
• Increasingly popular standards-based framework for
accessing network applications
– W3C standardization; Microsoft, IBM, Sun, others
• WSDL: Web Services Description Language
– Interface Definition Language for Web services
• SOAP: Simple Object Access Protocol
– XML-based RPC protocol; common WSDL target
• WS-Inspection
– Conventions for locating service descriptions
• UDDI: Universal Desc., Discovery, & Integration
– Directory for Web services
62
The Need to Support
Transient Service Instances
• “Web services” address discovery & invocation of
persistent services
– Interface to persistent state of entire enterprise
• In Grids, must also support transient service instances,
created/destroyed dynamically
– Interfaces to the states of distributed activities
– E.g. workflow, video conf., dist. data analysis
• Significant implications for how services are managed,
named, discovered, and used
– In fact, much of our work is concerned with the management
of service instances
63
Open Grid Services Architecture
• Service orientation to virtualize resources
• From Web services:
– Standard interface definition mechanisms: multiple
protocol bindings, multiple implementations, local/remote
transparency
• Building on Globus Toolkit:
–
–
–
–
Grid service: semantics for service interactions
Management of transient instances (& state)
Factory, Registry, Discovery, other services
Reliable and secure transport
• Multiple hosting targets: J2EE, .NET, “C”, …
64
Open Grid Services Architecture
More specialized &
domain-specific
services
Other
schemas
OGSA services: registry,
authorization, monitoring, data
access, management, etc., etc.
OGSA schemas
Open Grid Services Infrastructure
Web Services
Host. Env.
& Protocol Bindings
HostingEnvironment
Environment
Hosting
Transport
Protocol
Priorities:

Data access and
integration

Security

SLA negotiation

Manageability

Monitoring

…
65
OGSA Service Model
• System comprises (a typically few) persistent
services & (potentially many) transient services
• All services adhere to specified Grid service
interfaces and behaviors
– Reliable invocation, lifetime management,
discovery, authorization, notification, upgradeability,
concurrency, manageability
• Interfaces for managing Grid service instances
– Factory, registry, discovery, lifetime, etc.
=> Reliable, secure mgmt of distributed state
66
The Grid Service
• A (potentially transient) Web service with
specified interfaces & behaviors, including
–
–
–
–
–
–
–
–
Creation (Factory)
Global naming (GSH) & references (GSR)
Lifetime management
Registration & Discovery
Authorization
Notification
Concurrency
Manageability
67
Use of Web Services (1)
• A Grid service interface is a WSDL portType
• A Grid service definition is a WSDL extension
(serviceType) containing:
– A set of one or more portTypes supported by the
service
– portType & serviceType compatibility statements,
to support upgradability
• For discovery of compatible services when interfaces
are upgraded
– Implementation version information
68
Use of Web Services (2)
• A GSR is a WSDL document with extensions:
– Extension to service element to reference serviceType
– Service element extensions to carry the GSH, and the
expiration time of the GSR
• A GSH is an URL, with the following properties:
– Globally unique for all time
– http get on GSH + “.wsdl” returns GSR
– Can derive GSH to Mapper from it
• Registry returns WS-Inspection documents
69
Services
Portals
Grids: An Emerging, Common Computing and Data Infrastructure
for Science and Engineering
Web Portal Access to Application and
Grid Services
Resource
Brokering
Data Management:
replication and
metadata
Resource
Discovery
Fault
Management
Workflow
Management
Encapsulation for
Script Based Services
Scheduling and Access
to Computing
...
Applications
Accounting
Encapsulation as
Java Based Services
Uniform Data
Access
Monitoring
and Events
Grid Communication Functions
Operational Support
Basic Grid
Functions
Encapsulation as
Web Services
Specialized Portal Access (high
performance displays, PDAs, etc.)
transport services
security services
Communications
optical networks
Internet
national supercomputer
facilities
space-based networks
Distributed Resources
...
scientific instruments
tertiary storage
clusters
Condor pools
of workstations
70
Grids: A Common Computing and Data Infrastructure for
Science and Engineering
Portals: Services Presented to the Users to Accomplish Tasks
User
Environment
Portals
STS/SLI
Mission
Analysis
Collaboration
Portals
ISS
Training
ES
Modeling
Aviation
Capacity
MER/CIP
Application Domain Specific
Portals
Application Domain
Independent Portals
Domain Independent
Grid Web Services
Coupling
Zooming
Archive Gateways
System Models
Instrument &
Sensor Gateways
Flight Simulation
Computational
Simulation
Data Processing &
Analysis
Visualization
Collaboration
Services
Experiment
Management
Programming
Services
Events
Monitoring
Data Management
Workflow
Management
Grid Web Services: Grid Functions and Application Functions Packaged for Building Portals
Domain Specific Web Services –
Encapsulated Applications
Grid Common Services: Uniform Access, Security, and Management of Compute, Data, and Instrument Resources
Multi-Site Compute, Data, and Instrument Resources
71
Combining Grid and Web Services
composition
frameworks
(e.g. XCAT)
Job Submission /
Control
Grid ssh
File Transfer
CORBA
GRAM
Data Management
Monitoring
Events
……
Credential
Management
Workflow
Management
other services:
•visualization
•interface builders
•collaboration tools
•numerical grid
generators
•etc.
Python, Java, etc.,
JSPs
CoG Kits implementing
Web Services in
servelets, servers, etc.
Apache SOAP,
.NET, etc.
Apache Tomcat&WebSphere
&Cold Fusion=JVM + servlet
instantiation + routing
Resources
Condor-G
SRB/
Metadata
Catalogue
Data Replica and
Metadata Catalog
GridFTP
Grid
Monitoring
Architecture
Grid X.509
Certification
Authority
Grid
Information
Service
Grid Web Service
Description (WSDL)
& Discovery (UDDI)
MPI
Secure,
Reliable
Group Comm.
Grid Protocols and Grid Security Infrastructure
Environment
Management
(LaunchPad,
HotPage)
Grid Services:
Collective and Resource Access
Grid Protocols and Grid Security Infrastructure
http, https. etc.
Problem
Solving
Environments
(AVS, SciRun,
Cactus)
PDA
Web Browser
X Windows
Discipline /
Application
Specific
Portals
(e.g. SDSC
TeleScience)
Web
Services
XML / SOAP over Grid Security Infrastructure
Clients
Application
Portals
Compute
(many)
Storage
(many)
Communication
Instruments
(various)
72
For More Information
• Globus Project™
– www.globus.org
• Grid Forum
– www.gridforum.org
• Book (Morgan Kaufman)
– www.mkp.com/grids
73