Grids: Concepts,
Technologies and
Applications
Geoffrey Fox
Computer Science, Informatics, Physics
Pervasive Technology Laboratories
Indiana University Bloomington IN 47401
April 25 2005
gcf@indiana.edu
http://www.infomall.org
1
So what is a Grid?




Supporting human decision making with a network of at least
four large computers, perhaps six or eight small computers,
and a great assortment of disc files and magnetic tape units not to mention remote consoles and teletype stations - all
churning away. (Licklider 1960)
Coordinated resource sharing and problem solving in
dynamic multi-institutional virtual organizations
Infrastructure that will provide us with the ability to
dynamically link together resources as an ensemble to support
the execution of large-scale, resource-intensive, and
distributed applications.
Realizing thirty year dream of science fiction writers that
have spun yarns featuring worldwide networks of
interconnected computers that behave as a single entity.
2
Internet Scale Distributed Services



Grids use Internet technology and are distinguished by
managing or organizing sets of network connected resources
• Classic Web allows independent one-to-one access to
individual resources
• Grids integrate together and manage multiple Internetconnected resources: People, Sensors, computers, data
systems
Organization can be explicit as in
• TeraGrid which federates many supercomputers;
• Deep Web Technologies IR Grid which federates multiple
data resources;
• CrisisGrid which federates first responders, commanders,
sensors, GIS, (Tsunami) simulations, science/public data
Organization can be implicit as in Internet resources such as
curated databases and simulation resources that “harmonize a
community”
3
Different Visions of the Grid






Grid just refers to the technologies
• Or Grids represent the full system/Applications
DoD’s vision of Network Centric Computing is just a Grid
(linking sensors, warfighters, commanders, backend resources)
and they are building the GIG (Global Information Grid)
Utility Computing or X-on-demand (X=data, computer ..) is
major computer Industry interest in Grids
e-Science or Cyberinfrastructure are virtual organization Grids
supporting global distributed science (note sensors, instruments
are people are all distributed
Skype (Kazaa) VOIP system is a Peer-to-peer Grid (and
VRVS/GlobalMMCS like Internet A/V conferencing are
Collaboration Grids)
Commercial 3G Cell-phones and DoD ad-hoc network initiative
are forming mobile Grids
4
e-moreorlessanything and the Grid






e-Business captures an emerging view of corporations as
dynamic virtual organizations linking employees, customers
and stakeholders across the world.
• The growing use of outsourcing is one example
e-Science is the similar vision for scientific research with
international participation in large accelerators, satellites or
distributed gene analyses.
The Grid integrates the best of the Web, traditional
enterprise software, high performance computing and Peerto-peer systems to provide the information technology einfrastructure for e-moreorlessanything.
A deluge of data of unprecedented and inevitable size must
be managed and understood.
People, computers, data and instruments must be linked.
On demand assignment of experts, computers, networks and
storage resources must be supported
5
More Broad Classes of Grid Applications





Enterprise Grid supports information system for an
organization; includes “university computer center”, “(digital)
library”, sales, marketing, manufacturing …
Outsourcing Grid links different parts of an enterprise together
(Gridsourcing)
• Manufacturing plants with designers
• Animators with electronic game or film designers and
producers
• Coaches with aspiring players (e-NCAA or e-NFL etc.)
Customer Grid links businesses and their customers as in many
web sites such as amazon.com
e-Multimedia can use secure peer-to-peer Grids to link creators,
distributors and consumers of digital music, games and films
respecting rights
Distance education Grid links teacher at one place, students all
over the place, mentors and graders; shared curriculum,
6
homework, live classes …
e-Defense and e-Crisis

Grids support Command and Control and provide
Global Situational Awareness
• Link commanders and frontline troops to themselves and to
archival and real-time data; link to what-if simulations
• Dynamic heterogeneous wired and wireless networks
• Security and fault tolerance essential

System of Systems; Grid of Grids
• The command and information infrastructure of each ship is
a Grid; each fleet is linked together by a Grid; the President
is informed by and informs the national defense Grid
• Grids must be heterogeneous and federated

Crisis Management and Response enabled by a Grid
linking sensors, disaster managers, and first responders
with decision support
7
Types of Computing Grids




Running “Pleasing Parallel Jobs” as in United Devices,
Entropia (Desktop Grid) “cycle stealing systems”
Can be managed (“inside” the enterprise as in Condor)
or more informal (as in SETI@Home)
Computing-on-demand in Industry where jobs spawned
are perhaps very large (SAP, Oracle …)
Support distributed file systems as in Legion (Avaki),
Globus with (web-enhanced) UNIX programming
paradigm
• Particle Physics will run some 30,000 simultaneous jobs



Linking Supercomputers as in TeraGrid
Pipelined applications linking data/instruments,
compute, visualization
Seamless Access where Grid portals allow one to choose
one of multiple resources with a common interfaces
8
Utility and Service Computing




An important business application of Grids is believed to be
utility computing
Namely support a pool of computers to be assigned as needed to
take-up extra demand
• Pool shared between multiple applications
Natural architecture is not a cluster of computers connected to
each other but rather a “Farm of Grid Services” connected to
Internet and supporting services such as
• Web Servers
• Financial Modeling
• Run SAP
• Data-mining
• Simulation response to crisis like forest fire or earthquake
• Media Servers for Video-over-IP
Note classic Supercomputer use is to allow full access to do
“anything” via ssh etc.
• In service model, one pre-configures services for all programs
and you access portal to run job with less security issues
9
Some Important Styles of Grids






Computational Grids were origin of concepts and link computers
across the globe – high latency stops this from being used as
parallel machine
Knowledge and Information Grids link sensors and information
repositories as in Virtual Observatories or BioInformatics
• More detail on next slide
Education Grids link teachers, learners, parents as a VO with
learning tools, distant lectures etc.
e-Science Grids link multidisciplinary researchers across
laboratories and universities
Community Grids focus on Grids involving large numbers of
peers rather than focusing on linking major resources – links
Grid and Peer-to-peer network concepts
Semantic Grid links Grid, and AI community with Semantic web
(ontology/meta-data enriched resources) and Agent concepts
10
Information/Knowledge Grids


Distributed (10’s to 1000’s) of data sources (instruments,
file systems, curated databases …)
Data Deluge: 1 (now) to 100’s petabytes/year (2012)
• Moore’s law for Sensors




Possible filters assigned dynamically (on-demand)
• Run image processing algorithm on telescope image
• Run Gene sequencing algorithm on compiled data
Needs decision support front end with “what-if”
simulations
Metadata (provenance)
critical to annotate data
Integrate across experiments
as in multi-wavelength
astronomy
Data Deluge comes from pixels/year available
11
Repositories
Federated Databases
Database
Sensors
Streaming
Data
Field Trip Data
Database
Sensor Grid
Database Grid
Research
SERVOGrid
Education
Compute Grid
Data
Filter
Services Research
Simulations
?
GIS
Discovery Grid
Services
Customization
Services
From
Research
to Education
Analysis and
Visualization
Portal
Grid of Grids: Research Grid and Education Grid
Education
Grid
Computer
12
Farm
iSERVO in a nutshell

Designed to link data-sets (repositories and real time),
computations and earthquake scientists in ACES (Asia
Pacific) Cooperation
• Australia China Japan USA




Exemplified by SERVOGrid in USA led by JPL
Supports simulation and datamining as services
Adopts conservative WS-I+ Web Service
Interoperability standards
Builds full “Grid” in a library fashion as a Grid of Grids
• GIS (Geographic Information System) Grid built as a set of OGC
compatible Web Services “talking” GML
• iSERVO federates separate Grids in each
country/organization/function
• A Grid is “just” a collection of Services aka distributed programs


Multi-scale simulations supported by Grid workflow
13
Portals based on NSF Middleware Initiative NMI Open
Grid Computing Environment OGCE
DAME
In flight data
~5000 engines
~ Gigabyte per aircraft per
Engine per transatlantic flight
Airline
Global Network
Such as SITA
Ground
Station
Engine Health (Data) Center
Maintenance Centre
Internet, e-mail, pager
Rolls Royce and UK e-Science Program
Distributed Aircraft Maintenance Environment
14
NASA Aerospace Engineering Grid
Wing Models
•Lift Capabilities
•Drag Capabilities
•Responsiveness
Airframe Models
Stabilizer Models
•Deflection capabilities
•Responsiveness
Crew
Capabilities
- accuracy
- perception
- stamina
- re-action
times
- SOP’s
Human Models
Engine Models
•Braking performance
•Steering capabilities
•Traction
•Dampening capabilities
Landing Gear Models
•Thrust performance
•Reverse Thrust performance
•Responsiveness
•Fuel Consumption
simulations
are produced
by coupling
ItWhole
takes asystem
distributed
virtual organization
to design,
simulate
andall
build
a complex
system simulations
like an aircraft
of the
sub-system
15
Virtual Observatory Astronomy Grid
Integrate Experiments
Radio
Far-Infrared
Visible
Dust Map
Visible + X-ray
16
Galaxy Density Map
e-Chemistry Laboratory
Experiments-on-demand
Grid-enabled Output Streams
Simulation
Video
Diffractometer
Properties
Analysis
Structures
Database
GridGlobus
Resources
X-Ray
e-Lab
Properties
e-Lab
Fig. 23: A Combinatorial Chemistry Grid (Chapter 42)
17
CERN LHC Data Analysis Grid
18
OGSA-DAI
Grid Services
Grid
Grid Data
Assimilation
HPC
Simulation
Analysis
Control
Visualize
This Type of Grid
integrates with
Parallel computing
Multiple HPC
facilities but only
use one at a time
Many simultaneous
data sources and
sinks
Distributed Filters
massage data
For simulation
SERVOGrid (Complexity) Computing Model
19
Sources of Grid Technology







Grids support distributed collaboratories or virtual
organizations integrating concepts from
The Web
Agents
Distributed Objects (CORBA Java/Jini COM)
Globus, Legion, Condor, NetSolve, Ninf and other High
Performance Computing activities
Peer-to-peer Networks
With perhaps the Web and P2P networks being the most
important for “Information Grids” and Globus for
“Compute Grids”
20
The Essence of Grid Technology?



We will start from the Web view and assert that basic
paradigm is
Meta-data rich Web Services communicating via
messages
These have some basic support from some runtime
such as .NET, Jini (pure Java), Apache Tomcat+Axis
(Web Service toolkit), Enterprise JavaBeans,
WebSphere (IBM) or GT3/4 (Globus Toolkit 3/4)
• These are the distributed equivalent of operating system
functions as in UNIX Shell
• Called Hosting Environment or platform

W3C standard WSDL defines IDL (Interface
standard) for Web Services
21
Meta-data




Meta-data is usually thought of as “data about data”
The Semantic Web is at its simplest considered as
adding meta-data to web pages
For example, the hospital web-page has meta-data
telling you its location, phone-number, specialties which
can be used to automate Google-style searches to allow
planning of disease/accident treatment from web
Modern trend (Semantic Grid) is meta-data about webservices e.g. specify details of interface and useage
• Such as that a bioinformatics service is free or bandwidth
input is of limited amount

Provenance – history and ownership – of data very
important
22
A typical Web Service


In principle, services can be in any language (Fortran .. Java ..
Perl .. Python) and the interfaces can be method calls, Java RMI
Messages, CGI Web invocations, totally compiled away (inlining)
The simplest implementations involve XML messages (SOAP) and
programs written in net friendly languages like Java and Python
Web Services
WSDL interfaces
Portal
Service
Security
WSDL interfaces
Web Services
Payment
Credit Card
Catalog
Warehouse
Shipping
control
23
Typical Grid Architecture
Each Blob is a
Computer
Program!
System
Services
Portal
Services
User
Services
System
Services
Application
Service
Middleware
System
Services
System
Services
System
Services
“Core”
Grid
Raw (HPC)
Resources
Database
24
Classic Grid Architecture
Resources
Database
Database
Composition
Content Access
Netsolve
Security
Collaboration
Middle Tier
Brokers
Service Providers
Computing
Middle Tier becomes Web Services
Clients
Users and Devices
25
Peers
Database
Database
Service Facing
Web Service Interfaces
Event/
Message
Brokers
Event/
Message
Brokers
Event/
Message
Brokers
Peer to Peer Grid
Peers
User Facing
Web Service Interfaces
A democratic organization
26
Peer to Peer Grid
What is Happening?







Grid ideas are being developed in (at least) four communities
• Web Service – W3C, OASIS, (DMTF)
• Grid Forum (High Performance Computing, e-Science)
• Enterprise Grid Alliance (Commercial “Grid Forum” with a
near term focus)
Service Standards are being debated
Grid Operational Infrastructure is being deployed
Grid Architecture and core software being developed
• Apache has several important projects as do academia; large
and small companies
Particular System Services are being developed “centrally” –
OGSA framework for this in GGF; WS-* for
OASIS/W3C/Microsoft-IBM
Lots of fields are setting domain specific standards and building
domain specific services
USA started but now Europe is probably in the lead and Asia
will soon catch USA if momentum (roughly zero for USA)
continues
27
Technical Activities of Note





Look at different styles of Grids such as Autonomic (Robust
Reliable Resilient)
New Grid architectures hard due to investment required
Program the Grid – Workflow
Access the Grid – Portals, Grid Computing Environments
Low
Critical Services Such as
Level
WS-*
• Security – build message based not connection based
• Notification – event services
• Metadata – Use Semantic Web, provenance
• Fabric and Service Management
• Databases and repositories – instruments, sensors
• Computing – Submit job, scheduling, distributed file systems
• Visualization, Computational Steering
High Level
• Network performance
e.g. OGSA
28
Web services
Humans
Programs
Computational resources
message processing
service logic
BPEL, Java, .NET
Databases
SOAP and WSDL
• Web Services build
loosely-coupled,
distributed applications,
(wrapping existing codes
and databases) based on
the SOA (service oriented
architecture) principles.
• Web Services interact by
exchanging messages in
SOAP format
• The contracts for the
message exchanges that
implement those
interactions are described
via WSDL interfaces.
resources
Devices
<env:Envelope>
<env:Header>
...
</env:header>
<env:Body>
...
</env:Body>
</env:Envelope>
SOAP messages
29
Philosophy of Web Service Grids
• Much of Distributed Computing was built by natural
extensions of computing models developed for
sequential machines
• This leads to the distributed object (DO) model
represented by Java and CORBA
– RPC (Remote Procedure Call) or RMI (Remote Method
Invocation) for Java
• Key people think this is not a good idea as it scales
badly and ties distributed entities together too tightly
– Distributed Objects Replaced by Services
• Note CORBA was considered too complicated in
both organization and proposed infrastructure
– and Java was considered as “tightly coupled to Sun”
30
– So there were other reasons to discard
Plethora of Standards
• Java is very powerful partly due to its many “frameworks” that generalize libraries
e.g.
– Java Media Framework
– Java Database Connectivity JDBC
• Web Services have a correspondingly collections of specifications that represent
critical features of the distributed operating systems for “Grids of Simple Services”
– About 60 WS-* specifications introduced in last 2-3
years
– These are low level with higher level standards such as
access database (OGSA-DAI) or “Submit a job” built on
top of these
• Many battles both between standard bodies and between companies as each tries to
set standards they consider best; thus there are multiple standards for many of key
Web Service functionalities
• Microsoft a key player and stands to benefit as Web Services open up enterprise
software space to all participants
31
– e.g. MQSeries (IBM) and Tibco have to change their
WS-I Interoperability
• Critical underpinning of Grids and Web Services is
the gradually growing set of specifications in the
Web Service Interoperability Profiles
• Web Services Interoperability (WS-I)
Interoperability Profile 1.0a." http://www.ws-i.org.
gives us XSD, WSDL1.1, SOAP1.1, UDDI in basic
profile and parts of WS-Security in their first
security profile.
• We imagine the “60 Specifications” being checked
out and evolved in the cauldron of the real world
and occasionally best practice identifies a new 32
specification to be added to WS-I which gradually
Application Specific Grids
Generally Useful Services and Grids
Workflow WSFL/BPEL
Service Management (“Context etc.”)
Service Discovery (UDDI) / Information
Service Internet Transport  Protocol
Service Interfaces WSDL
Base Hosting Environment
Protocol HTTP FTP DNS …
Presentation XDR …
Session SSH …
Transport TCP UDP …
Network IP …
Data Link / Physical
Higher
Level
Services
Service
Context
Service
Internet
Bit level
Internet
(OSI
Stack)
Layered Architecture for Web Services and Grids
33
WS-* implies the The Service Internet





We have the classic (CISCO, Juniper ….) Internet routing the
flood of ordinary packets in OSI stack architecture
Web Services build the “Service Internet” or IOI (Internet on
Internet) with
• Routing via WS-Addressing not IP header
• Fault Tolerance (WS-RM not TCP)
• Security (WS-Security/SecureConversation not IPSec/SSL)
• Data Transmission by WS-Transfer not HTTP
• Information Services (UDDI/WS-Context not
DNS/Configuration files)
• At message/web service level and not packet/IP address level
Software-based Service Internet possible as computers “fast”
Familiar from Peer-to-peer networks and built as a software
overlay network defining Grid (analogy is VPN)
SOAP Header contains all information needed for the “Service
Internet” (Grid Operating System) with SOAP Body containing
information for Grid application service
Consequences of Rule of the Millisecond
• Useful to remember critical time scales
–
–
–
–
–
1) 0.000001 ms
– CPU does a calculation
2a) 0.001 to 0.01 ms – Parallel Computing MPI latency
2b) 0.001 to 0.01 ms – Overhead of a Method Call
3) 1 ms
– wake-up a thread or process
4) 10 to 1000 ms – Internet delay
• 2a), 4) implies geographically distributed
metacomputing can’t in general compete with
parallel systems
• 3) << 4) implies a software overlay network is
possible without significant overhead
– We need to explain why it adds value of course!
35
• 2b) versus 3) and 4) describes regions where method
Linking Modules
Closely coupled Java/Python …
Module
B
Module
A
Method Calls
.001 to 1 millisecond

Coarse Grain Service Model
Service
B
Messages
Service
A
0.1 to 1000 millisecond latency
From method based to RPC to message based to event-based
publish-subscribe Message Oriented Middleware
“Listener”
Subscribe
to Events
Service B
Publisher
Post Events
Message Queue in the Sky
Service A
36
What is a High Performance Computer?








We might wish to consider three classes of multi-node computers
1) Classic MPP with microsecond latency and scalable internode
bandwidth (tcomm/tcalc ~ 10 or so)
2) Classic Cluster which can vary from configurations like 1) to 3)
but typically have millisecond latency and modest bandwidth
3) Classic Grid or distributed systems of computers around the
network
• Latencies of inter-node communication – 100’s of milliseconds
but can have good bandwidth
All have same peak CPU performance but synchronization costs
increase as one goes from 1) to 3)
Cost of system (dollars per gigaflop) decreases by factors of 2 at
each step from 1) to 2) to 3)
One should NOT use classic MPP if class 2) or 3) suffices unless
some security or data issues dominates over cost-performance
One should not use a Grid as a true parallel computer – it can link
parallel computers together for convenient access etc.
37
What is a Simple Service?
• Take any system – it has multiple functionalities
– We can implement each functionality as an independent
distributed service
– Or we can bundle multiple functionalities in a single
service
• Whether functionality is an independent service or one of many method calls into a
“glob of software”, we can always make them as Web services by converting
interface to WSDL
• Simple services are gotten by taking functionalities and making as small as possible
subject to “rule of millisecond”
– Distributed services incur messaging overhead of one
(local) to 100’s (far apart) of milliseconds to use message
rather than method call
– Use scripting or compiled integration of functionalities
ONLY when require <1 millisecond interaction latency
• Apache web site has many projects that are multiple functionalities presented as
(Java) globs and NOT (Java) Simple Services
– Makes it hard to integrate sharing common security,38user
•
•
•
•
•
Grids of Grids of Simple Services
Link via methods  messages  streams
Services and Grids are linked by messages
Internally to service, functionalities are linked by methods
A simple service is the smallest Grid
We are familiar with method-linked hierarchy
Lines of Code  Methods  Objects  Programs  Packages
Methods
CPUs
Services
Clusters
Component Grids
Compute
Resource Grids
MPPs
Databases
Sensor
Federated
Databases
Sensor Nets
Data
Resource Grids
Overlay
and Compose
Grids of Grids
39
Component Grids?
• So we build collections of Web Services which we
package as component Grids
–
–
–
–
–
–
–
Visualization Grid
Sensor Grid
Utility Computing Grid
Person (Community) Grid
Earthquake Simulation Grid
Control Room Grid
Crisis Management Grid
• We build bigger Grids by composing component
Grids using the Service Internet
40
Flood CIGrid
…
Electricity
CIGrid
…
Gas Services
and Filters
Flood Services
and Filters
Collaboration Grid
Sensor Grid
Registry
Gas CIGrid
Portals
GIS Grid
Data Access/Storage
Visualization Grid
Compute Grid
Metadata
Core Grid Services
Security
Notification
Workflow
Messaging
Physical Network
Critical Infrastructure (CI) Grids built as Grids of Grids
41
Two-level Programming I


The paradigm implicitly assumes a two-level
Programming Model
We make a Service (same as a “distributed object” or
“computer program” running on a remote computer)
using conventional technologies
• C++ Java or Fortran Monte Carlo module
• Data streaming from a sensor or Satellite
• Specialized (JDBC) database access

Such services accept and produce data from users files
and databases
Service

Data
The Grid is built by coordinating such services
assuming we have solved problem of programming the
service
42
Two-level Programming II




The Grid is discussing the composition of distributed
services with the runtime Service1
Service2
interfaces to Grid as
opposed to UNIX
Service3
Service4
pipes/data streams
Familiar from use of UNIX Shell, PERL or Python
scripts to produce real applications from core programs
Such interpretative environments are the single
processor analog of Grid Programming
Some projects like GrADS from Rice University are
looking at integration between service and composition
levels but dominant effort looks at each level separately
43
What Should One Do?

Grids and Service Oriented Architectures will
• Change landscape in mature areas like enterprise software
• Support new distributed applications in Science,
Government, Education, Business and Community areas
• Encourage trends like outsourcing and globalization in all
activities



Web Service/Grid standards and infrastructure are still
in their infancy but broad principles reasonably clear
Many large scale software development activities are
inconsistent with modern architectures
Development of Application specific (XML-based)
standards is an important “safe” area
44