An Overview of Grid Computing
and its Adaptation to CSE
Curriculum
Bina Ramamurthy
CSE Department
University at Buffalo (SUNY)
bina@cse.buffalo.edu
http://www.cse.buffalo.edu/gridforce
Partially Supported by NSF DUE CCLI A&I Grant 0311473
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
1
Topics for Discussion
What is “The Grid”?
Motivation to explore the Grid
NSF Supported GridForce Project
Sample grid application
The Grid Conundrm
How to get involved?
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
2
Technology Pipeline
?
……
?
......
Internet
Technology Pipeline
What is in the pipeline?
How is the Internet evolving?
What is next?
Answers: … later
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
3
What is a grid?
Grid is a sophisticated framework that enables
sharing of a variety of resources among distributed
applications.
Open standard
Large scale operations
Automatic
Intelligent
Spontaneous
Interoperable
Service-oriented
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
4
What is a grid? (A formal definition)
Grid specifies a standard architecture, infrastructure,
protocols and application program interface (API) for
building an open enterprise system.
It can provide
 Middleware supporting network of systems to facilitate
sharing, standardization and openness.
 Infrastructure and application model dealing with sharing
of compute cycles, data, storage and other resources.
 A framework for high reliability, availability and security.
 Interoperation of batch-oriented and service-based
architectures.
 Standard service level feature definitions and higher level
concepts for inter and intra-business collaboration.
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
5
Beginnings of the Grid
Beginnings of the grid in Search for Extra Terrestrial
Intelligence (seti@home project)
http://planetary.org/html/UPDATES/seti/index.html

The Wow signal
http://planetary.org/html/UPDATES/seti/SETI@home/
wowsignal.html
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
6
Current Grid Users
A survey of 180 companies last summer by research firm Summit
Strategies found that 4% of respondents had implemented a grid, and 12%
were currently evaluating the technology.
Gartner predicted in 2002 that grid-based distributed systems will return
by 2004-2007.
Oracle Server 10g: g stands for grid. (Oracle 9i: i was for Internet)
Grid middleware from companies such
as DataSynapse and Platform provides users the ability to manage
workloads across the shared resources.
IBM used grid-base infrastructure for 2004 US Open: Enterprise
Networks Aug 2004.
Burlington Coat Factory is investing its IT future on a grid-based,
virtualized architecture: Enterprise Network June 2004.
HR (Human Resource) outsourcer Hewitt Associates put grid to work for
crunching pension calculations.
…
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
7
(Grid takes you) Beyond Search Engines: Enabling
Information Technology (IT) and Scientific
Applications
Television Remote
Simple Search (stateless)
Financial: Build Portfolio
Environment: Plan Forestation
Complex multi-organization applications
5/28/2016
Medicine: Find Cure
Astronomy: Find and
Control Telescopes
On-demand
SIGCSE 2005 Panel, St. Louis, MO
8
Grid Growth
GRID GROWT H (1994-2004)
(Gr ids)
125
115
100
103
105
95
91
75
66
65
55
45
36
# of Grids
85
35
24
25
12
2
2
15
15
7
5
Yea r
5/28/2016
20
04
20
01
20
02
20
03
20
00
99
19
98
19
97
19
96
19
95
19
19
94
-5
Copyright: Mustafa Faramawi 2004
GridForce, University at Buffalo
SIGCSE 2005 Panel, St. Louis, MO
9
Global Grid Share
GLOBAL GRID SHARE (1994-2004)
US
59%
UK
10%
Japan
5%
Greece
Netherla
nds
Australia
Europe
France
Germany
Italy
Sw itzerl
Canada
China
Croatia
Czech
Rep.
Denmark
Finland
Hungary
Iceland
Korea
Poland
Copyright: Mustafa Faramawi 2004
GridForce, University at Buffalo
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
10
Technology Pipeline: Answer
......
……
Internet
Technology Pipeline
What is in the pipeline?
Grid/GS
……
Web/WS
......
Internet
Technology Pipeline
WS: Web Service
GS: Grid Service
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
11
Adaptation of
Grid Technology to CS-Curriculum
Introduce grid technology into the CS undergraduate curriculum.
 Goal: Design and deploy grid services and applications. Study grid
application models.
 Focus on lab exercises to illustrate fundamental grid concepts, and
development of grid services and applications.
Conduct seminars to industry.
 Goal: Overview of grid technology landscape and its alignment to
common technologies and application models.
 Examine case-studies to expose potential uses of grid.
Introduce grid to potential users of grid
 Goal: Publicize the usage models of grid.
 Use grid infrastructure for entry level courses in Sciences and
Engineering.
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
12
GridForce
Our adaptation is evolving into a comprehensive
framework we call GridForce (Grid For
Collaboration and Education):
 Course curriculum,
 Laboratory exercises (labs),
 Infrastructure to support labs,
 Research projects,
 Industrial outreach.
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
13
GridForce: Grid For Research, Collaboration &
Education
Hands-on Labs
Research
Infrastructure
Courses/
Curriculum:
LinuxGrid Globus infrastructure supporting
CSE486/586: Distributed
Systems
CSE487/587: Information
Structures
Sample labs:
Grid Services Lab
secure service oriented architecture
Production Server
Staging Server
Design and implementation of
Grid services with standard capabilities
Cerf
Postel
Mills
Notification
Routable
Dissemination Package:
Syllabus, Lecture Notes, Exams,
Course Evaluations, Pedagogy,
Applications, Lab descriptions,
Solutions, Publications,
and Infrastructure details.
Logging
Grid
Service
Service
Data
OS: FreeBSD
Grid: Globus 3.0.2
Function: fileserver,
firewall
Secure
OS: Solaris 8.0
Grid: Globus 3.0.2
Function:
Debug and test services
Assessment Sample from Fall 2003
CSE486/586
TAX client
IRSServcie
Factory
Effectiveness of Grid
Coverage
Registry
IRS Hosting
Environment
HandleMap
Registry
EMPServcie
Factory
Registry
PERServcie
Factory
Emp Hosting
Environment
EMPService
Number of
Students
IRSService
5/28/2016
HandleMap
BNKServcie
Factory
PER Hosting
Environment
PerService
Registry
BNK Hosting
Environment
BNKService
HandleMap
HandleMap
10
8
6
4
2
0
No. Students
Avg Across
Questions
1
2
3
4
OS: Red Hat Linux 9.2
Grid: Globus3.0.2
Function: Deploy services
CSECCRGrid Collaboration with Center
For Computational Research (CCR); Reusable old Sparcs
offering Condor grid and NSF Middleware Initiative suite.
Virtual Organization (VO) Lab
Grid application design and implementation
Symbolic representation of VO components
Grid-based tax return filer.
Vixen
Collaboration
(SUNY Geneseo)
Labs Illustrating
Grid Use for
Non-CS Majors
Compute nodes:
Data analysis and
graph tools
Gatekeeper: Job
submission
scheduling tools
……………….
Internet :
remote job
submission
5
SIGCSE 2005 Panel, St. Louis, MO
Rating (1-best, 5-worst)
MySQL
Database
14
Goals and Outcomes of GridForce: Grid For Research,
Bina Ramamurthy
http://www.cse.buffalo.edu/gridforce
Collaboration & Education
CSE Department, University at Buffalo
NSF DUE CCLI A&I Grant 0311473
Outcome 4: Multi-platform
Interoperable Research
& Educational Infrastructure & Testbeds
Outcome 1: Successful adaptation of Grid
Dissemination Package:
Syllabus, Lecture Notes, Exams,
Course Evaluations, Pedagogy,
Applications, Lab descriptions,
Solutions, Publications,
and Infrastructure details.
Goal 1: Grid Courses/ Curriculum
CSE486/586: Distributed
Systems
CSE487/587: Information
Structures
1.WSRF.NETGrid:
adapted from
University of Virginia
Outcome 2: Involvement of smaller colleges and industries
2. LinuxGrid Globus infrastructure supporting
secure service oriented architecture
Goal 2: Research in Grid Services
Production Server
Staging Server
collaborations
Applications Internet and Web Standards
Goal 3: Collaboration for
Education & Dissemination
Postel
Outcome 3: Enrollment Strength:
Excellent
Outcome 5: Grid application
Models and grid accessibility
models
5/28/2016
Number of students
IRSServcie
Factory
Industries, colleges and K-12
Mills
OS: FreeBSD
Grid: Globus 3.0.2
Function: fileserver,
firewall
Grid Standards
OS: Solaris 8.0
Grid: Globus 3.0.2
Function:
Debug and test services
Cerf
Vixen
OS: Red Hat Linux 9.2
Grid: Globus3.0.2
Function: Deploy services
TAX client
Compute nodes:
Data analysis and
graph tools
Registry
Gatekeeper: Job
submission
scheduling tools
IRS Hosting
Environment
60
50
40
30
20
10
0
3. CondorGrid
HandleMap
IRSService
……………….
Registry
EMPServcie
Factory
1
2
Semesters
3
4
Registry
PERServcie
Factory
Emp Hosting
Environment
Enrollment
SIGCSE
2005 Panel, St. Louis, MO
EMPService
PerService
HandleMap
HandleMap
Registry
BNKServcie
Factory
PER Hosting
Environment
BNK Hosting
Environment
BNKService
HandleMap
Collaboration with
Center For
Computational
Research (CCR);
Internet :
remote job
submission
15
Courses:
CSE4/586 Distributed Systems
Learning outcome: fundamental concepts of distributed
systems and grid.
Lab exercises to support concepts:
 Three-tier client server system using Web Services.
 A simple grid framework.
 Design and implementation of a grid service.
Text: Distributed Systems: Concepts and Design (3rd Edition)
by George Coulouris, Jean Dollimore, Tim Kindberg,
Addison-Wesley Inc., 2000.
Prerequisites: Data structures and algorithms, object-oriented
design and development, working knowledge of Java.
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
16
Courses:
CSE 4/586: Lab Exercises
Exercise
Topic
Learning Objective
Lab1
Webservices
Lab2
Grid
Infrastructure
Lab3
Grid
Design and implement a grid-base
Programming
service using Globus 3.0.2
5/28/2016
To understand the alignment of the
grid technology to Web Services,
WS Definition Language (WSDL)
and service description using
WSDL.
A Webservices based grid.
SIGCSE 2005 Panel, St. Louis, MO
17
Courses:
CSE4/587:Information Structures
Learning outcome: understand grid infrastructure and grid
architecture, design and deploy grid services and grid
applications.
Lab exercises support:
 Enterprise application using Java 2 Enterprise Edition.
 Grid application in high performance area.
 Service-oriented grid application.
Text: “The Grid 2: Blueprint for a New Computing
Infrastructure” by Ian Foster , Carl Kesselman, MorganKauffmann, 2004.
Prerequisites: Data structures and algorithms, object-oriented
design and development, working knowledge of Java,
fundamentals of client/server architectures.
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
18
Courses:
CSE4/587: Lab Exercises
Exercise
Topic
Learning Objective
Lab1
Commercial
Application
Study requirements of a
commercial domain and
implement an application.
Lab2
High performance
Application.
Study requirements of
scientific/business domain and
implement compute intensive
application.
Lab3
Defining a highlevel grid
service
Workflow service, a
business process, improvements to
QoS
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
19
Services and Resources
Services: scientific services,
business services and
personal services.
Resources: data, storage
bandwidth
MyApplication
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
20
Grid Application using the services and resources
Services: scientific services,
business services and
personal services.
Resources: data, storage
bandwidth
MyApplication
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
21
Grid-based Stock Portfolio Building Application: Design Diagram By
Faramawi, a Student in CSE4/586 Course
StockBrokerService
StockExchangeService
subscribe()
createPortfolio()
authenticate()
buy()
sell()
depositFunds()
withdrawFunds()
buy()
sell()
transactions
users
portfolios
Markowitz
PortfolioAdvisoryService
Call &
Listen
StockClient
history
advise()
console/gui
Notify
Listen
random
Notify
StockMonitorService
HTML
Data feed (HTTP)
authenticate()
monitor()
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
22
Grid-based Stock Portfolio Building Application: Design Diagram By
Faramawi, a Student in CSE4/586 Course
StockBrokerService
StockExchangeService
subscribe()
createPortfolio()
authenticate()
buy()
sell()
depositFunds()
withdrawFunds()
buy()
sell()
transactions
users
portfolios
Markowitz
PortfolioAdvisoryService
Call &
Listen
StockClient
history
advise()
console/gui
Notify
Listen
random
Notify
StockMonitorService
HTML
Data feed (HTTP)
authenticate()
monitor()
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
23
Screen Shots from the Stock Monitoring Application:
Ross Goddard, CSE UG Student
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
24
Screen Shots from the Stock Monitoring Application:
Ross Goddard, CSE UG Student
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
25
OGSA
Grid
Mappings
Applications
Applications
Written to run over the network: e-mail, FTP,
order-service, HR, SMTP, IIOP, CORBA
Written for users operating within VO environment
Collective
OGSA Architectured Services
Services with associated protocols
OGSI
Directory, Brokering, Monitoring,
Diagnostics, Data replication,
Policy, Membership
Implemented by GT3 & OGSI.net
Web Services Technology
SDKs with associated APIs
Workflow, Discovery & Registry
Java, C/C++, .Net
UDDI, ebXML Registries, IBM WSFL, MSXLang
Resource
Service Description Language
Internet (OSI)
Presentation
Protocol and data conversion,
transport-independent policies:
encryption, QoS routing, caching,
Security (SSL), CORBA, Data Rep.
Session
Session Mgmt, checkpointing, reliable delivery:
HTTP, SSL, SQL, NFS, RPC
Transport
Flow control, error checking, guaranteed delivery:
WSDL, WSCL (XML)
Information
Management
TCP, UDP
Service Invocation & Messaging
SOAP (XML)
Connectivity
Transport
Communication
HTTP, FTP
Authentication
Security
Workflow
Database
Logical->physical address translation,
route and congestion management IP, ATM
Data Link
Fabric (OGSA-enabled resources)
Logical
Network
Packets-to-bits conversion, data frames:
Mac, LLC, ATM, PPP
File System Directory Messaging
Physical
Physical Compute Nodes
5/28/2016
Servers
Storage
Network
Sensors
SIGCSE 2005 Panel, St. Louis, MO
Transmits raw bitstream over physical cable:
Ethernet band signalling, ISDN
26
The Grid Conundrum
Grid
Flux
Technology
Internet
State of the technology
Solid
System level
Protocol level
operation
operation
Where are the opportunities for CSE educators and students?
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
27
How to get involved?
Opportunities exist in:





Shaping the standards
Specifying accessibility models
Specifying application models
Redefining some older concepts such as trust and identity in the context of
grid
Defining newer concepts such as resource use metering and service level
agreement
Read online literature
Introduce it in your courses
Course curriculum, project descriptions, solutions and lecture material
are available online at www.cse.buffalo.edu/gridforce
Major challenge is lab infrastructure: GridForce has testbeds in various
platforms available for use.
5/28/2016
SIGCSE 2005 Panel, St. Louis, MO
28