Appendix A
Desktop Grid
Availability
Predictor and
Machine
Recommender
Rohaya binti Latip
Mohammad Yaser Shafazand
Sazlinah Hasan
Network, Parallel and Distributed Computing, Faculty of
Computer Science and Information Technology, Universiti
Putra Malaysia, Malaysia
The supporting document is for the application of the
Desktop Grid CPU Availability Predictor and Machine
Recommender software copyright
Desktop Grid
CPU
Availability
Predictor and
Machine
Recommender
(DGAPMR)
Abstract
Desktop grid computing has emerged from the concept of providing relatively large amounts of
computing power for little cost using typical desktop machines. Approaches have been taken to
utilize the idle CPU cycles of desktop machines using desktop grid computing. However,
efficiently using resources has complications due to their volatile state. The scheduler, being
responsible for appropriate job dissemination, needs accurate job runtime estimations for an
efficient resource management. Distance, high diversity, distributed resource ownership and
intermittent availability of resources make it more complicated for predictions in desktop grids.
Though there has been many successful work on resource state predictions, but the overhead of
these systems has been less considered in the architecture design.
Another issue for the desktop grid scheduler is to appropriately match a suitable desktop
machine (resource provider) to a job. Machines that have enough available resources to satisfy
the job requirements and which are predicted to finish the job as soon as possible, must be
selected.
As a solution to these problems we introduce Desktop Grid CPU Availability Predictor and
Machine Recommender (DGAPMR) which uses a novel design for low overhead prediction and
a genuine mechanism to use recommender systems for desktop grids for more efficient
scheduling.
1. Introduction
Desktop grids are a form of computational grids in which each resource is a desktop machine. It
is a paradigm that uses heterogeneous, distributive computing resources (e.g., desktops
machines, notebooks) connected through the Internet or Intranet and owned by an organization
to provide computing power needed by projects. It has greatly enabled the use of vast amount of
computational capacities by stealing cycles while the computer is in idle state. However, utilizing
this computation power has complications due to the distance and high diversity of computing
nodes, not to mention the distributed resource ownership and intermittent availability of
resources.
One of the current focuses in desktop grids are to estimate if a resource or machine is available
long enough for a job to finish and return its results. The resource allocation or scheduling
problem is known as the problem to efficiently assign resources to jobs. This is a well-known
challenge in all categories of grid computing.
Thus, researches focus on mechanisms to predict properties and availabilities of resources. To
have a better utilization, it is important for the scheduler to be able to match jobs to the
appropriate machines. To address the issues discussed in desktop grid resource availability
prediction and job scheduling we introduce Desktop Grid CPU Availability Predictor and Machine
Recommender (DGAPMR). This design addresses two main issues in desktop grid computing.
The first is to predict the CPU availability of computing machines with low overhead. The second
is the assign appropriate machines to jobs in the scheduler. For both of these main subjects we
have come up with novel designs.
Our novel solution to the problem of predicting resource availability with low overhead is a
design which first discovers availability clusters of machines based on the mean availability of
machines throughout a time span on the server. It then creates mathematical models of each
cluster availability behavior. This mathematical model is sent from the server with its parameters
to all machines in the relevant cluster. The machines use this received model to predict their
everyday availability themselves (on the client side). This design has an update strategy and
distributes the prediction load on clients while considering accuracy and low overhead.
Our other novel solution is for the scheduling problem using a recommender system that
recommends machines to jobs based on job requirements while having a revising feature to
reevaluate the machine resources based on availability. This recommender would have no
interference with the schedulers work; it will just create suggestion lists with better anticipated
machines for a job.
Page 1 of 4
2.
Architecture of DGAPMR
The DGAPMR is actually composed of two parts: the Desktop Grid Availability Predictor (DGAP)
part and the Desktop Grid Machine Recommender (DGMR). The DGAP architecture is shown in
Figure 1. The Availability Monitor in the client side gathers the availability monitored data and
sends it to the server upon its request. The server classifies the clients based on their mean
availability into clusters. It calculates the mathematical model of availability behavior of each
cluster and sends the mathematical model to its relevant machines in the cluster. The Next Day
Availability Predictor then predicts the next day availability by filling the parameters in the
mathematical model based on its properties. This cycle iterates whenever the model needs to be
updated which is minimum once per a month.
DGAP Server Module
Mathematical
Model
Mathematical Model
Identifier
DGAP Client
Module
Next Day
Availability
Predictor
Clustering Process
Monitored
Availability Info
Availability
Monitor
Figure 1 DGAP Architecture
The general algorithm used in the Clustering process is described in Figure 2. We have used the
k-means function for simplicity while any other clustering functions would also be appropriate.
Figure 2 Clustering Process Algorithm
The algorithm used for our mathematical model identifier is also shown in Figure 3. We have
used the Box-Jenkins function of creating ARIMA models. We must note that other mathematical
functions that result in well-defined mathematical models could also be used.
Page 2 of 4
Figure 3 Mathematical Model Identifier Algorithm
The DGMR architecture is shown in Figure 4. The DGMR intervenes the resource list given by
the resource manager and replaces the list with a recommended one. We must note that the
components in the desktop grid server would be left intact while a more efficient list of resources
are created and recommended for each job.
DGMR Server
Module
Desktop Grid Server
Resource Manager
Machine
Recommender
Desktop Grid
Scheduler
Resource Availability
Predictor
Figure 4 DGMR architecture
The flow is as following (refer to Figure 5): The DGMR receives a Job with all its requirements. It
then retrieves copies of the resource manager’s resource list (a list of machines) which align with
the job resource needs and prioritizes the list based on that. The time each machine could finish
the job would be calculated. The predicted values for the CPU availability of each of the
machines in the list are then gathered and the time each machine could finish the job would be
recalculated using the availability factors. The list would be then revised and reordered. The
revised list is then recommended and sent to the scheduler.
Page 3 of 4
Figure 5 Recommender Process Flow
We must note that the Revise phase has the ability to receive other availability factors such as
machine and bandwidth availability. We have used and tested CPU availability records in our
studies.
Page 4 of 4