High Performance Systems Group
Performance-responsive
Scheduling for
Grid Computing
Dr Stephen Jarvis
High Performance Systems Group
University of Warwick, UK
High Performance Systems Group
Context
• Funded by / collaborating with
–
–
–
–
–
UK e-Science Core Programme
IBM (Watson, Hursley)
NASA (Ames)
NEC Europe
Los Alamos National Laboratory
• Integrate established performance tools
into emerging grid middleware
What do we mean by ‘scheduling’
• Users view
– Jobs run somewhere on the Grid
– Notion of deadline
– Execution is single domain (includes pre-staging)
• Resource providers view
– Don’t mind which jobs are run where
– As long as resources are well/evenly used
– Maintaining customers deadlines is important
• System view
– Jobs can run anywhere
– Resources are heterogeneous
– Throughput is important, as are scheduling overheads
High Performance Systems Group
Managing through Middleware
High Performance Systems Group
Managing through Middleware
•Determine what resources
are required (predict)
•Determine what resources
are available (discover)
•Map requirements to
available resources
(schedule)
•Maintain contract of
performance (QoS)
High Performance Systems Group
Performance Services
• Intra-domain
– Lab- / department-based
– Shared resources under
local administration
• Multi-domain
– Campus- / country-based
– Wide-area resource and
task management
– Cross domain
High Performance Systems Group
Performance Services
• Intra-domain
– Lab- / department-based
– Shared resources under
local administration
• Multi-domain
– Campus- / country-based
– Wide-area resource and
task management
– Cross domain
High Performance Systems Group
Performance Services
• Intra-domain
– Lab- / department-based
– Shared resources under
local administration
• Multi-domain
– Campus- / country-based
– Wide-area resource and
task management
– Cross domain
High Performance Systems Group
Performance Prediction
• Performance prediction tools
• Aim to predict
– Execution time
– Communication usage
– Data and resource requirements
• Provides best guess as to how an
application will execute on a given
resource
High Performance Systems Group
PACE
Application
Resource
User
High Performance Systems Group
PACE
Application
Application
Model
Resource
Model
Resource
User
High Performance Systems Group
PACE
Application
Application
Model
Model parameters
Evaluation
Engine
Resource
Model
Resource
Resource config.
User
High Performance Systems Group
PACE
Application
Application
Model
Model parameters
Evaluation
Engine
Resource
Model
Resource
Resource config.
User
High Performance Systems Group
• Scaling properties
50
45
sweep3d
40
35
30
fft
improc
25
20
15
10
closure
jacobi
memsort
cpi
16
13
10
7
4
5
0
1
Running TimeRun-time
on SGIOrigin2000 (sec)
Why is prediction useful?
T he Number of Processors
• Compare runtime
options with
– deadline
– available resources
– priority / other
jobs
– etc.
Allows runtime scenarios to be explored before deployment
High Performance Systems Group
1. Intra-Domain Co-Scheduling
• Augment Condor scheduler with additional
performance information
• Scheduler driver, or co-scheduler (called Titan)
• Use predictive data for system improvement
– Time to complete tasks / utilisation of resources
– QoS – ability to meet deadlines
• Handle predictive and non-predictive tasks
High Performance Systems Group
Intra-Domain Co-Scheduling
• Non-predictive tasks
REQUESTS FROM
USERS OR OTHER
DOMAIN SCHEDULERS
PACE
PORTAL
PREEXECUTION
ENGINE
SCHEDULE
QUEUE
MATCHMAKER
GA
Titan
CLUSTER
CONNECTOR
CLASSADS
CONDOR
RESOURCES
High Performance Systems Group
Intra-Domain Co-Scheduling
• Non-predictive tasks
REQUESTS FROM
USERS OR OTHER
DOMAIN SCHEDULERS
PACE
PORTAL
PREEXECUTION
ENGINE
SCHEDULE
QUEUE
MATCHMAKER
GA
Titan
CLUSTER
CONNECTOR
CLASSADS
CONDOR
RESOURCES
High Performance Systems Group
Intra-Domain Co-Scheduling
• Non-predictive tasks
• Tasks with prediction data
REQUESTS FROM
USERS OR OTHER
DOMAIN SCHEDULERS
PACE
PORTAL
PREEXECUTION
ENGINE
SCHEDULE
QUEUE
MATCHMAKER
GA
Titan
CLUSTER
CONNECTOR
CLASSADS
CONDOR
RESOURCES
High Performance Systems Group
Intra-Domain Co-Scheduling
• Non-predictive tasks
• Tasks with prediction data
REQUESTS FROM
USERS OR OTHER
DOMAIN SCHEDULERS
PACE
PORTAL
PREEXECUTION
ENGINE
SCHEDULE
QUEUE
MATCHMAKER
GA
Titan
CLUSTER
CONNECTOR
CLASSADS
CONDOR
RESOURCES
High Performance Systems Group
Intra-Domain Co-Scheduling
• Non-predictive tasks
• Tasks with prediction data
REQUESTS FROM
USERS OR OTHER
DOMAIN SCHEDULERS
PACE
PORTAL
PREEXECUTION
ENGINE
SCHEDULE
QUEUE
MATCHMAKER
GA
Titan
CLUSTER
CONNECTOR
CLASSADS
CONDOR
RESOURCES
High Performance Systems Group
Intra-Domain Co-Scheduling
• Non-predictive tasks
• Tasks with prediction data
REQUESTS FROM
USERS OR OTHER
DOMAIN SCHEDULERS
PACE
PORTAL
PREEXECUTION
ENGINE
SCHEDULE
QUEUE
MATCHMAKER
GA
Titan
CLUSTER
CONNECTOR
CLASSADS
CONDOR
RESOURCES
High Performance Systems Group
Intra-Domain Co-Scheduling
• Non-predictive tasks
• Tasks with prediction data
REQUESTS FROM
USERS OR OTHER
DOMAIN SCHEDULERS
PACE
PORTAL
PREEXECUTION
ENGINE
SCHEDULE
QUEUE
MATCHMAKER
GA
Titan
CLUSTER
CONNECTOR
CLASSADS
CONDOR
RESOURCES
High Performance Systems Group
Intra-Domain Deployment
Without co-scheduler
Time to complete = 70.08m
With co-scheduler
Time to complete = 35.19m
High Performance Systems Group
2. Multi-Domain Management
• Publish intra-domain perf. data through
global information services (MDS)
• Augment service with agent system
– One agent per domain / VO
• When a task is submitted
– Agents query IS, and negotiate to discover best
domain to run task
• Scheme is tested on a 256-node exp. Grid
– 16 resource domains; 6 arch. types
High Performance Systems Group
Multi-Domain Management
time
High Performance Systems Group
Multi-Domain Management
time
High Performance Systems Group
Multi-Domain Management
time
High Performance Systems Group
Multi-Domain Management
Time to complete = 2752s
High Performance Systems Group
Multi-Domain Management
Time to complete = 467s;
an improvement of 83%
High Performance Systems Group
Multi-Domain Management
Time to complete = 467s;
an improvement of 83%
High Performance Systems Group
QoS: Ability to Meet Deadline
active
inactive
High Performance Systems Group
Resource usage
active
inactive
High Performance Systems Group
Other work
• OGSA compatibility
• Prediction
– Accuracy
– Other prediction techniques
• Workflow (CCGrid 2003)
• Reservation
• V. 1.1, Condor/GT2-based
– www.dcs.warwick.ac.uk/~hpsg
– Documented at HPDC-12/GGF-8, FGCS