High Performance Systems Group
University of Warwick, UK

• Grid research at Warwick.
• Grid workload management and related middleware services
• Performance modelling & prediction
• Workflow
• OGSA Integration
2

• e -Science project to develop performance-aware middleware services.
• Focus on the performance behaviour of existing grid technologies, and the performance effects of mapping workload to particular grid resources.
• Activities include:
– Study performance behaviour of the MDS in Globus
– Refinement of existing performance modelling tools
– Developing scheduling and request-routing mechanisms
– Developing workload management tools that orchestrate work-flows end-to-end.
3

• e -Science project to develop performance-aware middleware services.
• Focus on the performance behaviour of existing grid technologies, and the performance effects of mapping workload to particular grid resources.
• Activities include:
– Study performance behaviour of the MDS in Globus
– Refinement of existing performance modelling tools
– Developing scheduling and request-routing mechanisms
– Developing workload management tools that orchestrate work-flows end-to-end.
4

• The usual WLM constraints apply:
– Make efficient use of resources
– Provide guaranteed qualities of service
• However, there are specific grid issues:
– Grids span administrative domains
– Large-scale
– Dynamic (in resources, users & configuration)
• The environment dictates a distributed approach with clear separations of concern.
5

• GWLM must typically identify resource(s) that are suitable for a particular application.
– General users are unlikely to want to make such a choice
“by hand”
– If there is a choice, how is one system selected over another?
– Some decisions are straightforward
• Accept or reject based on architecture, configuration, etc.
– Others are more complex
• No clear favourite exists.
• Current tasks obscure selection.
– Partial solution can be obtained by understanding the anticipated application performance
6

• Performance models (at a reasonable level of fidelity) can be effective at driving resource management decisions.
• It is beneficial (from a WLM standpoint) if there is some separation between architecture and application parameters.
• A suitable tool, PACE, aims to predict:
– execution time
– communication usage
– data & resource requirements
• Allows a “best guess” of how a task will behave with respect to performance characteristics
7

Application
Resource 1 Resource 2 Resource 3
8

Source Code
Analysis
Object Editor
Performance
Library
Performance scripting language
Compiler
Resource 1 Resource 3 Resource 2
9

Source Code
Analysis
Object Editor
Performance
Library
Performance scripting language
Compiler
Resource 1
Compiler
Hardware Model
CPU Comm Cache
Resource 3
10

• Expected time predicted by a modeling tool for a given workload on a given resource.
– Fast evaluation produces run-times for one or more systems scenarios.
• Pre-execution evaluation
– Scalability analysis (sizing an application appropriately)
• Modular components for heterogeneous resources
– Not limited to a single “performance point”
– Applicable to Grids
• Can drive workload management decisions & scheduling systems.
11

• QoS is inherently user-focussed (are my jobs being completed within my deadline?).
• From the system perspective, it may be more appropriate to reject a task rather than to accept a task and then fail to meet an agreement.
• Performance prediction can reveal “workable” solutions to workload management - it allows service levels to be specified with some confidence that the workload will execute to the user’s requirement.
• “Deliver what was promised”
12

Application 1
Service Policy mA
1
Application 2
Service Policy
Application 2
Service Policy mA
2 mA
3
Hosts
Cluster
Interface
QoS
Reporting

Application 1
Service Policy mA
1
Application 2
Service Policy
Application 2
Service Policy mA
2 mA
3
Hosts
Cluster
Interface
QoS
Reporting

Application 1
Service Policy mA
1
Application 2
Service Policy
Application 2
Service Policy mA
2 mA
3
Hosts
Cluster
Interface
QoS
Reporting

• A workload management system has been developed at Warwick that makes significant use of performance models.
• It is a layered architecture that divides the problem into multi-domain, local-domain and resource tiers.
• TITAN does not schedule itself - it is a grid
“enhancement service” that can control cluster RMs such as Condor or the Globus run manager.

• Tasks are presented to brokers that communicate within a pre-arranged structure (typically a singlerooted hierarchy).
• Each broker is tied to a local scheduler that acts on its behalf - accepting and publishing tasks to other brokers.
• Tasks are “offered” to multiple brokers who respond with an execution-time statement obtained by performance model evaluation. The task is then routed appropriately.

• TITAN currently executes tasks using Condor.
• Condor is a high-throughput scheduler
– Historically used in “cycle-stealing” mode
– Increasingly used with Globus in dedicated mode
– Submit fil e is produced
– Creates ClassAds for Condor
– Task passed to Condor
• TITAN monitors the Condor queue
– Feedback performance of finished tasks
– Notify scheduler of chances in the resources
18

70 mins end-to-end
19

36 mins end-to-end
20

• Recent work has been focussed on managing flows of related work presented to the brokers.
• This fits with the trend in grid computing to develop small functional components linked together to form an application (WebServices).
• This architecture is beneficial from the performance modeling standpoint - small tasks are run frequently and are closed from the user.
• Each component can be modeled / timed prior to user execution and parameterised appropriately.
21

• TITAN considers work flow descriptions at both the multi and local level
– Multi: break the work flow into sub-flows based on an estimation of communication dependencies.
– Local: honour local flow - performance model use to guide interleaving.
• Workflow is described in a simple XML document
(Apache ANT-like syntax) to provide a list of dependent actions.
• We are currently looking at combing performance models with different workflow description techniques.
22

• The TITAN local scheduler uses a genetic algorithm to perform task placement.
• This has been adapted to factor workflow descriptions when building schedule solutions.
• The GA is able to “interleave” different sub-flows and utilise the idle time between neighbouring processes.
• The local scheduler is able to commit to a deadline for an entire flow using performance models. This can be used in partnership with the brokers.
23

• Two simple flows of heterogeneous tasks (with different performance behaviours) are submitted along with a number of single tasks.
24

• Using the performance models,
TITAN determines processor mappings that meet the task deadlines.
25

• Flows are interleaved with other jobs, whilst maintaining task deadlines.
TIME
26

• We are currently comparing mapping flows using
TITAN against Condor DAG-based equivalents.
• Initial results mimic single task results (i.e. reduction in makespan, improvement in the ability to meet deadlines).
27

• To support grid middleware integration, TITAN is now being developed in line with the OGSA (GT3 containers).
• This includes the development of a performance service that is able to provide predictive data ondemand using SOAP.
• TITAN functions as a self-contained Grid service able to offer scheduling functions to local and remote users.
28

• Performance-aware services make a valuable contribution to grid middleware
– good use of available resources
– steer tasks to appropriate architectures
– addresses SLA issues
• Can commit to single task and flows of tasks - this is crucial as the impact of failure is greater.
• Work is currently focused on multi-domain workflow management (dividing the sub-flows) and developing the OGSA integration.
29

30