Grid: The Social Imperative
B. Ramamurthy
CSE Department
University at Buffalo (SUNY)
bina@cse.buffalo.edu
http://www.cse.buffalo.edu/gridforce
Partially Supported by NSF DUE CCLI A&I Grant
0311473
5/28/2016
Erir Community College
1
Topics for Discussion
What does the title mean?
Software trends
What is Grid?
Gridforce Project
The Internet Revolution
Evolution of Request/Response Model
Grid Service
Grid Concepts
Case Studies: Applications we implemented
Infrastructure
Demos
5/28/2016
Erir Community College
2
Software Trends
scale
Application
complexity
Multi-tier
Server-side
Component
programming
Ex: J2EE
Client-server
Classes
Grid Computing
Virtualization
Federation
Provisioning
Object-oriented
programming
monolithic
Structured
programming
1970
5/28/2016
1980
1990
Erir Community College
2000
Time (years)
3
Grid Technology
Emerging enabling technology.
Natural evolution of distributed systems and the Internet.
Middleware supporting network of systems to facilitate
sharing, standardization and openness.
Infrastructure and application model dealing with sharing
of compute cycles, data, storage and other resources.
Promoted by NSF through its Network Middleware
Initiative (NMI version 4).
Publicized by prominent industries as on-demand
computing, utility computing, etc.
Move towards delivering “computing” to masses similar
to other utilities (electricity and voice communication).
5/28/2016
Erir Community College
4
GridForce: Grid For Research, Collaboration &
Education
Hands-on Labs
Research
Infrastructure
Courses/
Curriculum:
LinuxGrid Globus infrastructure supporting
CSE486/586: Distributed
Systems
CSE487/587: Information
Structures
Sample labs:
Grid Services Lab
secure service oriented architecture
Production Server
Staging Server
Design and implementation of
Grid services with standard capabilities
Cerf
Postel
Mills
Notification
Routable
Dissemination Package:
Syllabus, Lecture Notes, Exams,
Course Evaluations, Pedagogy,
Applications, Lab descriptions,
Solutions, Publications,
and Infrastructure details.
Logging
Grid
Service
Service
Data
OS: FreeBSD
Grid: Globus 3.0.2
Function: fileserver,
firewall
Secure
OS: Solaris 8.0
Grid: Globus 3.0.2
Function:
Debug and test services
Assessment Sample from Fall 2003
CSE486/586
TAX client
IRSServcie
Factory
Effectiveness of Grid
Coverage
Registry
IRS Hosting
Environment
HandleMap
Registry
EMPServcie
Factory
Registry
PERServcie
Factory
Emp Hosting
Environment
EMPService
Number of
Students
IRSService
5/28/2016
HandleMap
BNKServcie
Factory
PER Hosting
Environment
PerService
Registry
BNK Hosting
Environment
BNKService
HandleMap
HandleMap
10
8
6
4
2
0
No. Students
Avg Across
Questions
1
2
3
4
OS: Red Hat Linux 9.2
Grid: Globus3.0.2
Function: Deploy services
CSECCRGrid Collaboration with Center
For Computational Research (CCR); Reusable old Sparcs
offering Condor grid and NSF Middleware Initiative suite.
Virtual Organization (VO) Lab
Grid application design and implementation
Symbolic representation of VO components
Grid-based tax return filer.
Vixen
Collaboration
(SUNY Geneseo)
Labs Illustrating
Grid Use for
Non-CS Majors
Compute nodes:
Data analysis and
graph tools
Gatekeeper: Job
submission
scheduling tools
……………….
Internet :
remote job
submission
5
Erir Community College
Rating (1-best, 5-worst)
MySQL
Database
5
The Internet Revolution
Began as an experimental network for
collaboration among scientists.
Enabling technology for the information age.
Commercial success.
Major impact on society and economy of the
world.
Fundamental concept: Request-response
model.
5/28/2016
Erir Community College
6
Request-Response
When you access a web page you request information at a location
specifying its address using http (protocol).
Response from the server to the request is typically in the form of a
html document.
Browser parses the html document and displays it for you.
Now turn html tags into meaningful tags Extensible Markup
Language (XML)
Turn the information request into {operation, followed by operands}
Simple Object Request Protocol (SOAP)
Standard for SOAP over XML: Web Services (WS)
WS is currently a very popular technology for application development.
For example: Java 2 Enterprise Environment specification includes Web
Services standard.
Summarizing:
html over http XML, SOAP over http WS  ?
5/28/2016
Erir Community College
7
Grid Service Framework
Every grid service is a web service!
with additional standard capabilities by
Open Grid Services Architecture (OGSA)
Open Grid Services Interface (OGSI)
specifies capabilities defined in OGSA
Globus Toolkit (GT3.0.2) is an
implementation of OGSI.
5/28/2016
Erir Community College
8
Grid Service
Logger object; Levels of logging:
Info, .. Warn, Error, Fatal
Filtering and redirecting to file, console
Logging
For asynchronous messaging
Secure
…
Notification
Web
Service
Service
Data
Routable
Services with ability
to migrate
Stores service properties and
States; for discovery, monitoring,
negotiations, etc.
Persistent
Permanent services such as naming service that
get activated and terminated with the container
5/28/2016
Erir Community College
9
Grid Concepts
Virtualization
Virtual Organization (VO)
Federation
Provisioning
Service Oriented Architectures (SOA)
5/28/2016
Erir Community College
10
CSE486:Lab1: Web Services
National Weather
Service Web Site
IP Socket
API
http
RMI
Client
1
RMI Weather
Server
Weather Web
5
Service
Container/Server
5/28/2016
RMI
LAN
2
Relational
Database 4
Erir Community College
3
Oracle 9i
SOAP
XML
http
Weather 6
Web Service
Web Client
11
CSE487:Lab 3: Tax Return Filer
IRSServcie
Factory
Registry
IRS TAX Filer
Hosting
Environment
HandleMap
IRSService
EMPServcie
Factory
EMPService
Registry
EMPLOYEMENT
Hosting
Environment
HandleMap
PERServcie
Factory
TAX client
PERSONAl
Hosting
Environment
Registry
PerService
BNKServcie
Factory
BANK Hosting
Environment
Registry
BNKService
HandleMap
HandleMap
Concepts illustrated: Virtual organization (VO) called IRS/Tax Filer that brings together virtualized capabilities
of physical organizations of banking, personal profiles, and employment.
Grid service handle (GSH) and Grid service reference (GSR), registry and handlemap, discovery of services, index
services, application of
notification, logging.
.
5/28/2016
Erir Community College
12
Infrastructure: LinuxGrid
Goal: To facilitate development of service-oriented
applications for the grid.
Two major components: Staging server and
Production grid Server.
Grid application are developed and tested on staging
server and deployed on a production server.
Production grid server:
 Three compute nodes with Red Hat Linux and
Globus 3.0.2 instance.
 One utility gateway node with Free BSD and
Globus 3.0.2.
5/28/2016
Erir Community College
13
Development Environment
Production Server
Staging Server
Postel
OS: Solaris 8.0
Grid: Globus 3.0.2
Function:
Debug and test services
5/28/2016
Mills
OS: FreeBSD
Grid: Globus 3.0.2
Function: fileserver,
firewall
Erir Community College
Cerf
Vixen
OS: Red Hat Linux 9.2
Grid: Globus3.0.2
Function: Deploy services
14
Infrastructure: SparcGrid
Goal: To run jobs submitted in a distributed manner
on a Condor-based computational cluster Condor.
Composed of 50 Sun recyclable used Sparc4
machines, which form computational nodes, headed
by a front-end Sun server.
The installation scripts are custom-written facilitating
running of jobs in a distributed manner.
Partially supported by Center for Computational
Research (CCR).
5/28/2016
Erir Community College
15
CSECCR Grid
Compute nodes:
Data analysis and
graph tools
Gatekeeper: Job
submission
scheduling tools
Students work on computationally
intensive application. Gene expression
analysis; Markovitz model for financial
portfolio picking; Students also learn
About Condor, jon submission,
scheduling, building grid infrastructure out
of discarded computers.
……………….
Internet :
remote job
submission
5/28/2016
MySQL
Database
Erir Community College
16
SparcGrid Monitor (Ganglia)
5/28/2016
Erir Community College
17
Summary
We will conclude the talk with some demos.
In this talk, we covered salient features of the
emerging grid technology.
At University at Buffalo (UB) we have a established a
comprehensive framework for grid education.
A major social impact of grid will be computing
accessible to the masses.
Grid will enable a computational pipeline similar to
how Internet enabled an information pipeline.
5/28/2016
Erir Community College
18