|epcc|
Overview of a Grid
Scheduling Paradigm
NeSC Workshop
Open Issues in Grid Scheduling
Ali Anjomshoaa
EPCC, University of Edinburgh
Tuesday, 21 October 2003
|epcc| Grid Scheduling

Why is scheduling so important for Grids?

In order to make coherent and coordinated
use of ubiquitous and heterogeneous Grids,
Grid management is essential.

Grid management is not an option.

“Scheduling is all about management.”
2
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Scheduling is about Management

Grids are complex multivariate environments, which are
made up of numerous Grid entities that need to be
managed.

These Grid entities fulfill various roles, which places
them within the remit of different management
environments.

Grid entities and their roles are interdependent on each
other and their environments.

The management of these Grid entities is, for the most
part, the undertaking of Grid scheduling.
3
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Scheduling is about Management

Have introduced the concepts of:




There is a mapping:


Grid entities
Grid entity roles
Grid management environments
entity roles ↔ management environments
In addition, there are interdependencies:



Between the Grid entities
Between the Grid entity roles
Between the Grid management environments
4
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Entities

So what are these Grid entities?

A Grid entity is defined by its roles.

Grid entity roles can include:





Resource
Account
Contract
User




Job
Policy
Workflow
…
A Grid entity can have one or more roles.
5
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Entity Roles

A Grid entity can fulfill many roles. For example:



it can have both a Job role and a Resource role,
it can have both a Job role and a Workflow role,
or indeed any other combination of roles.

A Grid entity role is defined by a set of role specific
attributes.

A Grid entity should be managed within different
management environments, depending on the various
roles that it fulfills.

Could say that the interest of Grid scheduling is in the
role management of the various roles of Grid entities.
6
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Entity Definition

A Grid entity is defined by its roles.

A Grid entity role is defined by a set of role specific
attributes.

“A Grid entity is defined by the sets of role specific
attributes of its roles.”

The definition of a Grid entity could be captured in
a description document as sets of role attributes.

The description document will then allow the Grid
entity to be managed according to its various roles,
within a Grid.
7
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Entity Definition
Grid
Entity
8
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Entity Definition
Resource
Role
Job
Role
…
Grid
Entity
…
9
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Entity Definition
Resource
Attribute
Set
Job
Attribute
Set
…
Grid
Entity
…
10
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Entity Definition
Resource
Attribute
Set
Job
Attribute
Set
Description
Document
…
Grid
Entity
…
11
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Summary of Concepts

Previously introduced the concepts of:

Grid entities
 Grid entity roles
 Grid management environments

Have now introduced two new concepts of:


Grid entity role specific attributes
Grid entity description documents
12
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Entity Roles and Management

Different roles require different management
environments.

“The management environment within which a Grid entity
is being managed, will depend on the particular role of
that Grid entity which is being managed.”

A role type can not be managed in more than one
management environment.

A management environment can manage more
than one type of role.
13
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Entity Roles and Management

Role interdependencies need to be managed also.

This will mean that the boundaries of management
environments will need to be crossed.

Common factorisations of methods and protocols,
from various management environments, need to
be made, in order to allow the management of role
interdependencies.
14
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Management Environments

Resource Management Environment



Computing Resource Management Environment
Database Resource Management Environment
Network Resource Management Environment

Job Management Environment

User Management Environment

Policy Management Environment

Account/Finance Management Environment

Security Management Environment
15
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Management Environments

A Grid management environment sets the
rules for the management of a set of Grid
entity role types.

The types of Grid entity roles managed in a
particular management environment, can
not be managed by any other.
16
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Entity Roles and Management
Grid
Entity
17
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Entity Roles and Management
Resource Management Environment
Job Management Environment
Resource
Resource
Role
Grid
Entity
Job
Job
Role
18
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Entity Roles and Management
Resource Management Environment
Job Management Environment
Grid
Entity
Resource
Attribute
Set
Job
Attribute
Set
19
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Management Environments

Different management environments should allow
for custom management models, methods,
protocols, languages, and semantics.

However, due to the interdependencies between
these management environments, they must also
share a common set of languages, semantics, and
ontologies in an interoperability layer to allow them
to interoperate.

Without this common set of interoperability
features, it would not be possible to manage role
and environment interdependencies.
20
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Management Environments
Security
Resource
Policy
Job
User
Account
21
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Grid Management Environments
Security
Resource
Policy
Interoperability
Layer
Job
User
Account
22
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| The Result?

What is the advantage of these concepts and
abstractions?

Discrete roles and management environments
allows them to be developed in a highly
independent way.

This framework is easily extendable to include new
roles and management environments.

This framework allows role definition and attribute
sets to be extended or updated, without breaking
the framework.
23
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Meta-Scheduling

Scheduling should allow the management
of roles within their respective management
environments.

Meta-scheduling should handle the
interdependencies between roles and their
environments, both within Grid entities, and
between Grid entities.
24
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Summary

Have introduced the concepts of:


Grid entities
Grid entity roles



Grid entity role specific attributes
Grid entity description documents
Grid management environments

Roles are managed within their respective
management environments.

Role interdependencies must be managed.
25
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| Conclusions

Managing discrete Grid entity roles in respective
management environments allows for the development of
highly independent pieces of the Grid management puzzle.

This framework enables manageable extensions and
upgrades of the various pieces of the Grid management
puzzle.

This framework gives flexibility in defining Grid entities
through the attribution of an extensible set of roles.

This framework allows independent management of discrete
roles, while enabling management of role interdependencies
through the interoperability layer.

This framework should allow the consolidation of disparate
scheduling efforts in various collaborations.
26
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling
|epcc| The Holy Grail!

Interoperable Grid architectures based on
heterogeneous infrastructure layers,
managing heterogeneous computing
environments.

Need common interoperability layers:
protocols, languages, semantics, ontologies
27
Tuesday, 21 October 2003
NeSC Workshop - Open Issues in Grid Scheduling