The Gridbus Toolkit for
Service-Oriented [Cluster/Grid] Computing
“Economic Paradigm for Clusters and Grids-based
Utility Computing”
Rajkumar Buyya
Grid Computing and Distributed Systems (GRIDS) Lab.
The University of Melbourne
Melbourne, Australia
www.gridbus.org
Collaboration:
Nimrod-G Project
WW Grid
Collaborations
Nimrod-G Resource Broker
2
Virtual Lab
Collaborations
Yuko Mizuno-Matsumoto3
Susumu Date
3
Acknowledgements: Co-authors

Nimrod-G

David Abramson, Monash
Jon Giddy, DSTC
Slavisa Garic (nirmod 1.1 update)

Manzur Murshed, Monash (GridSim core 1.0)

Kim Branson, WEHI for Structural Biology

Susumu Date, Osaka University

Wolfgang Gentzsch, Sun Microsystems





GridSim
Virtual Lab
NeuroGrid

Industry Collaborator

Gridbus Project Members

Srikumar Venugopal (Data Grid Scheduler)
Jia Yu (Grid Market Directory)
Martin Placek (Web-based Monitoring and Control Station)
Shoaib Burq (Visual Tool for Parameter Sweep Application Creation)
Alex Burmouta (Grid Bank)
Anthony Sulistio (GridSim 2.0 Visual modeller)
Chee Shin Yeo (GridSim 2.0 Vistual modeller)
Steve Melnikoff (HEP data catalogue and apps parameterization)
Lyle Winton, School of Physics (HEP data catalogue and BASF application help)
Libra 1.0:

CPM/Jxta:









4

Jahanzeb Sherwani, Nosheen Ali, Nausheen Lotia, Zahra Hayat, Lahore University of Management and Science

Fazilah Haron, Chan Huah Yong, Liew Sun, U. Sains Malaysia
Agenda
Grid Challenges and The Project Vision
 Grid Architecture for Computational Economy
 Gridbus and Nimrod-G Technologies
 Scheduling Experiments on the World Wide
Grid testbed
 New Gridbus Tools
 Conclusion

Grid
Grid
Economy
5
Scheduling
Economics
A Sample Grid Computing Environment:
Resource Sharing & Aggregation
Grid Information Service
Grid Resource Broker
R2
2
R5
R3
Application
database
R4
RN
Grid Resource Broker
R6
Grid Information Service
6
R1
Resource Broker
Why Grids ? Large Scale Exploration
needs them—Killer Applications.

Solving grand challenge applications using
computer modeling, simulation and analysis
Aerospace
Internet &
Ecommerce
Life Sciences
CAD/CAM
7
Digital Biology
Military Applications
Grid Characteristics and Challenges
Entities/Issues
Users, Resources,
Owners
Grid Information Service
Geographically
Grid Resource
Broker
Distributed
resources, users, apps
R2
2
R3
Heterogeneous
Application
database
R4
Resource
Varies with time
Availability/Capability
R5
R
N
Policies and
strategies
Grid Resource Broker
R6
QoS requirements
Cost / Price
Grid Information Service
challenges
8
Characteristics
Heterogeneous &
decentralised
R1
Heterogeneous
Resource Broker
Varies: different
resources, users, time
* Resource Management
* Application Construction
Grid Challenges and Technologies
Security
Computational Economy
Uniform Access
Resource Discovery
9
System Management
Data locality
Resource Allocation
& Scheduling
Application Construction
Network Management
Some Grid Projects & Technologies



Australia

Nimrod-G

Gridbus

GridSim

Virtual Lab

DISCWorld

GrangeNet

..new coming up
Europe

UNICORE

Cactus

UK eScience

EU Data Grid

EuroGrid

MetaMPI

XtremeWeb

and many more.
India


Japan



10
Korea...
I-Grid
Ninf
DataFarm
N*Grid

USA












Cycle Stealing & .com Initiatives




Globus
Legion
OGSA
Sun Grid Engine
AppLeS
NASA IPG
Condor-G
Jxta
NetSolve
AccessGrid
and many more...
Distributed.net
SETI@Home, ….
Entropia, UD, Parabon,….
Public Forums





Global Grid Forum
Australian Grid Forum
IEEE TFCC
CCGrid conference
P2P conference
http://www.gridcomputing.com
mix-and-match
Object-oriented
Internet/partial-P2P
Network enabled Solvers
Economic-based Utility
Computing
11
Nimrod-G
The Gridbus Vision: To Enable Service
Oriented Grid Computing & Bus iness!
WW Grid
Gridbus
World Wide Grid!
+ marketplace for
Service-Oriented Computing
12
Gridbus Vision Scales:
Breaks Administrative Barriers
P
E
R
F
O
R
M
A
N
C
E
2100
2100
2100
2100
2100
2100
2100
2100
2100
Administrative Barriers
•Individual
•Group
•Department
•Campus
•State
•National
•Globe
•Inter Planet
•Universe
+
Q
o
S
13
Personal Device
SMPs or
SuperComputers
Local
Cluster
Enterprise
Cluster/Grid
Global
Grid
Inter Planet
Grid
GRIDS Lab @ the U. of Melbourne,
The Gridbus Project: www.gridbus.org

R& D in Distributed Computational Economy and end-to-end
infrastructure for Service-Oriented/Utility Computing: :











Visual Tools for Creation of Distributed Applications
Grid Economy & Scheduling (via Nimrod-G Broker)
GridSim Toolkit: Grid Modeling and Simulation (Java based):
Libra: Economic Cluster Scheduler
Grid Bank: Accounting and Transaction Management
Grid Market Directory and Web Services
World Wide Grid (WWG) – A Global Grid Testbed
Jxta-based P2P Compute Power Market (CPM)
Application Enabler Projects:




14
Architecture, Specification, and “Open Source” Reference Implementation
in Collaboration with interested Global Peers.
Platform: Cluster, Grid, P2P for diverse applications.
Virtual Laboratory Toolset for Drug Design
High-Energy Physics and the Grid Network (HEPGrid)
Brain Activity Analysis on the Grid (NeuroGrid)
Future: GridEmail for Spam Management
—read only if sender pays!
Gridbus: Towards Building a Grid Economy
for Sustained Resource Sharing
15
Offers Incentive and Enable the Creation and Promotion of:
Grid Marketplace (competitive)
ASP
Service Oriented Computing
...
And let users focus on their own work (science, engineering, or commerce)!
Grid Economy: Incentive, Resource Allocation
and Regulation
16
New challenges of Computational
Economy

Resource Owners







Resource Consumers





17
How do I decide prices ? (economic models?)
How do I specify them ?
How do I enforce them ?
How do I advertise & attract consumers ?
How do I do accounting and handle payments?
…..
How do I decide expenses ?
How do I express QoS requirements ?
How I trade between timeframe & cost ?
….
Grids need to support the automation of the process.
GRACE: A Reference Grid Economy
Services Architecture
GRid Architecture for Computational
Economy (GRACE)
GRACE: A Reference
Service-Oriented Grid Architecture for Computational
Economies
Data Catalogue
Grid Bank
Programming
Environments
Applications
Sign-on
Info ?
Grid Explorer
Job
Control
Agent
Grid Market
Services
Health
Monitor
Grid Node N
Secure
Schedule Advisor
QoS
Grid Node1
Pricing
Algorithms
Trade Server
Trade Manager
Trading
…
Deployment Agent
JobExec
Grid Resource Broker
Misc. services
Resource Allocation
Storage
Grid Middleware
Services
Accounting
Resource
Reservation
R1
Grid Consumer
19
Information
Service
R2
…
Rm
Grid Service Providers
Realizing the GRACE Vision: Development
Methodology Used
Applications
Science
Engineering
Commerce
…
Portals
High-level Services and Tools
MPI-G
CC++
…
Parameter Specification
Nimrod-G
Higher Level Resource Aggregators
ActiveSheet
Visual Paramteric
Modeller
Gridbus Scheduler
Grid
Apps.
Grid
Tools
Core Services
MDS
GRAM
GASS
GARA
Jxta
GMD
GBank
GTS
Middleware
PKI-based Grid Security Interface (GSI)
20
JVM
TCP
UDP
Linux
Irix
Solaris
Local
Services
Libra
Condor
LSF
Grid
PBS
SGE
Grid
Fabric
Using Gridbus + Nimrod-G:
Brokering and On Demand Assembly of Grids
Application Code
Explore
data
1
Data Source
Visual Parametric Tool
Data
10
2
(Instruments/dis
tributed sources)
Data Catalogue
Data Replicator
(GDMP)
5
6
(Data) Grid
Resource Broker
3
Grid Info Service
4
ASP Catalogue
9
12
Grid Market Directory
7
8
Grid Service (GS)
(Globus/CPM)
Grid
Service
Bill
GS
Cluster Scheduler
PE
21
Grid Service Provider (GSP)
(e.g., CERN)
CPU
or
PE
GSP
(e.g., IBM)
Cluster Scheduler
PE
GSP
(e.g., UofM)
PE
GTS
GSP
(e.g., VPAC)
11
GridBank
GSP
(Accounting Service)
Gridbus Tools:
Enabling Utility Computing for
Distributed Data Intensive
Applications
Visual Environment for Parameter Application Creation
Grid Market Directory (Portal for GSPs + Webservices for GRBs)
Gridbus Data Grid Scheduler (built on Nimrod-Farming Engine)
Gridbus Monitoring and Control Station (Web-based)
GridBank: Grid Accounting Services Architecture
GridSim Toolkit 2.0
Libra: Cluster Economy Scheduler

Sample P-Sweep/Task/Data
Farming Grid Applications
Bioinformatics:
Drug Design /
Brain Activity Analysis
Ecological Modelling:
Combinatorial
Control Strategies
Optimization:
for Cattle Tick
Meta-heuristic
Data Mining
parameter estimation
Neuroscience
Computer Graphics:
Ray Tracing
High Energy
Physics:
Searching for
Rare Events
Electronic CAD:
Field Programmable
Gate Arrays
VLSI Design:
Finance:
SPICE Simulations
Investment Risk Analysis
Civil Engineering:
Building Design
Automobile:
Crash Simulation
23
Network Simulation
Aerospace:
Wing Design
astrophysics
Drug Design: Data Intensive
Computing on Grid
Molecules
Protein

24
Chemical Databases
(legacy, in .MOL2 format)
It involves screening
millions of chemical
compounds (molecules) in
the Chemical DataBase
(CDB) to identify those
having potential to serve as
drug candidates.
[Collaboration with WEHI for Medical Science, Melbourne]
Gridbus Visual Tool for Parametric
Application Creation (e.g., Docking)
25
NeuroGrid: QoS Requirements driven MEG
Data (Brain Activity) Analysis on the Grid
Analysis All pairs (64x64) of MEG data by
shifting the temporal region of MEG data
over time: 0 to 29750: 64x64x29750 jobs
64 sensors MEG
2
Data Generation
3
1
5
Data Analysis
Grid Resource Broker
(Nimrod-G+Gridbus)
Results
4
Life-electronics laboratory,
AIST
[deadline, budget, optimization preference]
World-Wide Grid
•Provision of MEG (MagnetohyEncephaloGraphy) analysis
26
[Collaboration with Osaka University, Japan]
The Grid Market Directory for Service
Publication and Discovery
Grid Market Directory (GMD)
Web Server (Tomcat)
GMD Portal
Manager
GMD Query
Webservice
Grid Service Info (RDBMS)
Browse
Publish/Manage
Query(SOAP+XML)
Job
submission
Grid Node
Grid Node
Provider (Web Client)
Consumer (Grid Resource Broker)
Grid Node
27
Consumer (Web Client)
Nimrod-G : A Grid Resource Broker



A resource broker for managing, steering, and
executing task farming (parameter sweep/SPMD
model) applications on the Grid based on deadline and
computational economy.
Based on users’ QoS requirements, our Broker
dynamically leases services at runtime depending on
their quality, cost, and availability.
Key Features








28

A single window to manage & control experiment
Persistent and Programmable Task Farming Engine
Resource Discovery
Resource Trading
Scheduling & Predications
Generic Dispatcher & Grid Agents
Transportation of data & results
Steering & data management
Accounting
Nimrod-G and Gridbus Scheduler
Nimrod/G Client
Nimrod/G Client
Gridbus Client
(Default Scheduler or Gridbus Scheduler)
Nimrod/G Engine
Schedule Advisor
Trading Manager
Grid
Store
Grid Dispatcher
Grid Explorer
Grid Middleware
TM
Globus, Legion, Condor, etc.
TS
GE
GIS
Grid Information Server(s)
RM & TS
RM & TS
G
RM & TS
C
L
G
Globus enabled node.
29
L
Legion enabled
node.
RM: Local Resource Manager, TS: Trade Server
G
C L
Condor enabled node.
Deadline
A Nimrod-G
CostMonitor
66 Arlington
Alexandria
Legion hosts
She na nd o a h
Rive r
64
64
81
Ra p p a ha nno c k Po to m a c
Rive r
Rive r
Ja m e s
Rive r
Roanoke
Richmond
Ap p o m a to x
Rive r
Hampton
Norfolk
Virginia Beach
Portsmouth Chesapeake
Newport News
77
VIRGINIA
85
Globus Hosts
Bezek is in both
Globus and Legion Domains
30
G-monitor – Web-based Station
31
Deadline and Budget Constrained
Scheduling Algorithms
32
Algorithm/
Strategy
Execution Time Execution Cost
(Deadline, D)
(Budget, B)
Cost Opt
Limited by D
Minimize
Cost-Time Opt
Minimize when
possible
Minimize
Time Opt
Minimize
Limited by B
Conservative-Time
Opt
Minimize
Limited by B, but all
unprocessed jobs
have guaranteed
minimum budget
Scheduling Experiments on the
World Wide Grid
The World Wide Grid Sites
WW Grid
Cardiff/UK
Portsmoth/UK
Manchester, UK
Cambridge, UK
TI-Tech/Tokyo
ETL/Tsukuba
AIST/Tsukuba
ANL/Chicago
USC-ISC/LA
UTK/Tennessee
UVa/Virginia
Dartmouth/NH
BU/Boston
UCSD/San Diego
Santiago/Chile
34
EUROPE:
ZIB/Germany
PC2/Germany
AEI/Germany
Lecce/Italy
CNR/Italy
Calabria/Italy
Pozman/Poland
Lund/Sweden
CERN/Swiss
CUNI/Czech R.
Vrije: Netherlands
Kasetsart/Bangkok
Singapore
U. Of Melbourne
Monash U.
VPAC/Melbourne
U. Of Sydney
Experiment Setup

Workload:




Deadline: 2 hrs. and budget: 396000 G$
Strategies: 1. Minimise cost 2. Minimise time
Execution:




35
165 jobs, each need 5 minute of CPU time
Optimise Cost: 115200 (G$) (finished in 2hrs.)
Optimise Time: 237000 (G$) (finished in 1.25 hr.)
In this experiment: Time-optimised scheduling run
costs double that of Cost-optimised.
Users can now trade-off between Time Vs. Cost.
Resources Selected & Price/CPU-sec.
36
Resource &
Location
Grid services &
Fabric
Cost/CPU
sec.or unit
No. of Jobs Executed
Linux Cluster-Monash,
Melbourne, Australia
Globus, GTS, Condor
2
64
153
Linux-Prosecco-CNR,
Pisa, Italy
Globus, GTS, Fork
3
7
1
Linux-Barbera-CNR,
Pisa, Italy
Globus, GTS, Fork
4
6
1
Solaris/Ultas2
TITech, Tokyo, Japan
Globus, GTS, Fork
3
9
1
SGI-ISI, LA, US
Globus, GTS, Fork
8
37
5
Sun-ANL, Chicago,US
Globus, GTS, Fork
7
42
4
Time_Opt
Cost_Opt.
Total Experiment Cost (G$)
237000
115200
Time to Complete Exp. (Min.)
70
119
G-monitor Usage: HPC Challenge Demo
@ SC 2002
Australia
North America
GMonitor
MEG
Visualisation
Melbourne+Monash U:
ANL: SGI/Sun/SP2
NCSA: Cluster
Wisc: PC/cluster
NRC, Canada
Many others
Gridbus+
Nimrod-G
VPAC, Physics
Solaris WS
@ SC 2002/Baltimore
Europe
Grid Market
Directory
Asia
AIST, Japan: Solaris
Cluster
Osaka University: Cluster
Doshia: Linux cluster
Korea: Linux cluster
37
WW Grid
Internet
ZIB: T3E/Onyx
AEI: Onyx
CNR: Cluster
CUNI/CZ: Onyx
Pozman: SGI/SP2
Vrije U: Cluster
Cardiff: Sun E6500
Portsmouth: Linux PC
Manchester: O3K
Cambridge: SGI
Many others
Resource Scheduling for DBC Time
Optimization
Condor-Monash
Linux-Prosecco-CNR
Linux-Barbera-CNR
Solaris/Ultas2-TITech
SGI-ISI
Sun-ANL
12
No. of Tasks in Execution
10
8
6
4
2
38
Time (in Minute)
68
72
60
64
52
56
44
48
36
40
28
32
20
24
16
8
12
4
0
0
Resource Scheduling for DBC Cost
Optimization
Condor-Monash
Linux-Prosecco-CNR
Linux-Barbera-CNR
Solaris/Ultas2-TITech
SGI-ISI
Sun-ANL
14
No. of Tasks in Execution
12
10
8
6
4
2
Time (in Minute)
39
10
2
10
8
11
4
96
90
84
78
72
66
60
54
48
42
36
30
24
18
6
12
0
0
The GridSim Toolkit
A Java based tool for Grid Scheduling Simulations
Application, User, Grid Scenario’s Input and Results
Application
Configuration
Resource
Configuration
User
Requirements
Grid
Scenario
...
Output
Grid Resource Brokers or Schedulers
GridSim Toolkit
Application
Modeling
Resource
Entities
Information
Services
Job
Management
Resource
Allocation
Statistics
Resource Modeling and Simulation (with Time and Space shared schedulers)
Single CPU
SMPs
Clusters
Load Pattern
Network
Reservation
Basic Discrete Event Simulation Infrastructure
SimJava
Distributed SimJava
Virtual Machine (Java, cJVM, RMI)
PCs
40
Workstations
SMPs
Clusters
Distributed Resources
Selected GridSim Users!
41
Libra: Economic-based cluster
scheduler
User
Application
+deadline, budget
Cluster Management
System
(e.g., SGE/PBS)
Job
Job
Input
Dispatch
Control
Control
(node 1)
Application
User
Budget
Check
Control
Best
Node
Evaluator
...
(node 2)
Deadline Control
Node Querying
Module
(node N)
Scheduler (Libra)
Server: Master Node
42
Cluster Worker Nodes
with node monitor
(e.g.,pbs_mom)
CPM: P2P Market-Maker Technology
(Jxta based Implementation)
Market
Server
- Discovery
- Membership
User (Consumer)
Market
Repository
CPM Agent
Trader
Bill
Job
Management
Accounting
43
Resources
(Provider)
Summary and Conclusion





44
Grid Computing is emerging as a next generation computing
platform for solving large scale problems.
Resource management is a complex undertaking as systems need
to be adaptive, scalable, competitive,…, and driven by QoS.
Scheduling experiments on the World Wide Grid demonstrate our
brokers ability to dynamically lease services at runtime based on
their quality, cost, and availability and consumers QoS
requirements.
Gridbus tools allow Creation/Composition of Grid Applications with
minimal or no effort. [Grids fail if we ask biologists to create
applications using Globus/MPI—assembly language of grid
computing).
Economic-based resource management and scheduling is essential
for pushing Grids into mainstream computing and creating the
World-Wide Grid Marketplace.
The End, Thank You!
?
For further info:

Visit our home in cyberspace @:





45
http://www.gridbus.org
Visit our Research Exhibit @ SC 2002 for a live demo.
Visit our home on planet earth @ Melbourne, Australia!
W are happy to collaborate with anybody in the universe.
PS: When we deploy our Economic-based GridEmail application, our
Inbox accepts only paid or bartered emails and the rest will be sent to
junk box automatically. This helps in handling Nigerian emails!