Proactive Prediction Models for Web
Application Resource Provisioning in the Cloud
_______________________________
Samuel A. Ajila & Bankole A. Akindele
Presentation Outline










Introduction to Problem Area
Motivation
Goals and Scope
Contributions
Related work
Machine learning algorithms
Implementation setup
Evaluation metrics
Selected results
Conclusion
2
Introduction



Cloud computing refers to both the applications delivered
as services over the Internet and the hardware and
systems software in the data centers that provide those
services (SaaS, PaaS and IaaS)
Challenges include: Data security threats, performance
unpredictability and prompt (quick) resource scaling
Accurate Virtual machine (VM) resource provisioning:
•

under and over-provisioning
Current techniques like Control theory, Constraint
programming and Machine learning
3
Motivation





VM instantiation (scaling) takes time and ranges from 5 to 12
minutes
Challenges that can result from the instantiation duration:
• Possibility of Service Level Agreement (SLA) violations Cloud providers
• Poor customer’s Quality of Experience (QoE) – Cloud client
• Reputation loss – Cloud client/providers
Presently, monitoring metrics made available to clients are
limited to CPU, Memory and Network utilization
The motivation is to “predict resource usage so that cloud
providers can make adequate provisioning ahead of time”
To extend the monitoring metrics by including Response time
and Throughput
4
Goals and Scope





Design and develop a cloud client prediction model for cloud
resource provisioning in a Multitier web application
environment
The model would be capable of forecasting future resource
usage to enable timely VM provisioning
To achieve this goal, SVM, NN and LR learning techniques are
analysed using the Java implementation of TPC-W (workload)
The scope of this work is limited to IaaS
Prediction model is built around the web server tier. It is
possible to extend to other tiers
5
Contributions




Design and development of a cloud client prediction model that
uses historical data to forecast future resource usage
The evaluation of the resource usage prediction capability of
SVM, NN and LR using three benchmark workloads from TPC-W
The extension of the prediction model to include Throughput
and Response time, thus providing wider and better scaling
decision options for cloud clients
The comparison of the prediction capability of SVM, NN and LR
models under random and steady traffic patterns
6
Related works
Table 1: Auto-scaling techniques
Technique
Class
Comments
Threshold/Rule
based
Reactive
Reacts to system changes but do not
anticipate them
Control theory
Reactive/Proactive Except where used with a proactive
approach it suffers reactive issues
Queuing theory
Reactive
Reinforcement
learning
Reactive/Proactive Good but converging at optimal
policy can be unfeasibly long (state
–action pairs)
Time series analysis
Proactive
As complexity of system grows, the
analytic formulas become difficult
Good and promising
7
Machine learning algorithms
Linear Regression
𝑝
𝑗=1 𝑋𝑗 𝛽𝑗

The linear regression model has the form: 𝑓 𝑋 = 𝛽0 +

𝑋𝑗 is the input data

𝛽0 and 𝛽𝑗 ’s are the unknown parameters estimated from the training
data. Estimation is done using the least squares method.
The coefficients 𝛽 are picked to minimize the residual (difference
between the actual and predicted value) sum of squares :
(𝑦𝑖 − 𝑓 𝑋 )2
𝑦𝑖 is the actual value

8
(1)
Machine learning algorithms (cont’d)
Neural Network
 A two-stage regression or
classification model represented
by a network diagram
 Derived features 𝑍𝑚 are created
from linear combinations of the
input after which the target 𝑌𝑘 is
modeled as a linear combination
of 𝑍𝑚
 Like Linear regression, unknown
parameters called weights are
sought to make the model fit the
training data well
Figure 1 : Single hidden layer, feed forward neural network
9
Machine learning algorithms (cont’d)
Support Vector Regression
 The goal of SVR is to find a function 𝑓 𝑋 that has at most 𝜀 (the
precision by which the function is to be approximated) deviation from
the actual obtained target for all training data

Mathematically, 𝑓 𝑋 = 𝑤. ∅ 𝑥

Input data 𝑥 is mapped to higher dimensional feature space via the
kernel function, then a linear regression is performed
Goal is to find optimal weights 𝑤 and threshold 𝑏

+𝑏
10
Architecture
Figure 2 Implementation architecture
11
Implementation setup (cont’d)
 Total length of experiment was about 10 hours
 Selected experimental workload mix
Time (minutes)
1-7
56-63 154-161 350-357 490-497 504-511
Shopping mix
users
84
168
16
180
248
160
Browsing mix
users
52
112
36
320
192
160
Ordering mix
users
52
108
28
224
268
160
Total user
Requests
188
388
80
724
708
480
12
Evaluation metrics
Metric
MAPE (Mean
Percentage Error)
Performance metrics and their calculations
Calculation
Absolute 1 𝑛 | 𝑎 𝑖 −𝑝 𝑖 | where 𝑎 and 𝑝 are the actual and predicted values
𝑛
𝑖=1
𝑎𝑖
𝑖
𝑖
respectively
𝑛
2
𝑖=1( 𝑎 𝑖 −𝑝 𝑖 )
RMSE (Root Mean Square
Error)
MAE (Mean Absolute Error)
√
PRED 25
No. of observations with relative error ≤ 25% / No. of observation


𝑛
1 𝑛
|𝑝
𝑛 𝑖=1 𝑖
− 𝑎𝑖 |
Evaluation is done on the 60% training and 40% held out test
dataset
The held out dataset is used to forecast to a maximum interval
of 12 minutes (VM instantiation time reported by other
authors)
13
CPU Training & Testing Performance
CPU Utilization Training and Testing Performance Metric for LR, NN, and SVR
LR (Linear Regression) Model
LR Training
LR Testing
NN (Neural Network) Model
NN Training
NN Testing
SVR(Support Vector Regression) Model
SVR Training
SVR Testing
MAPE
113.31
36.19
MAPE
105.63
50.46
MAPE
107.80
22.84
RMSE
14.70
22.13
RMSE
14.08
31.08
RMSE
15.48
11.84
MAE
11.11
15.98
MAE
9.48
19.82
MAE
10.09
8.74
PRED(25)
0.51
0.36
PRED(25)
0.59
0.34
PRED(25)
0.64
0.64
14
Selected results (cont’d)
CPU utilization test performance metric
Model
LR
NN
SVR
MAPE
36.19
50.46
22.84
RMSE
22.13
31.08
11.84
MAE
15.98
19.82
8.74
PRED(25)
0.36
0.34
0.64
Figure 5 CPU Utilization Actual and Predicted test model results
15
Selected results (cont’d)
Throughput test performance metric
Model
LR
NN
SVR
MAPE
24.62
38.90
22.07
RMSE
3.72
6.12
3.22
MAE
2.87
4.46
2.41
PRED(25)
0.63
0.47
0.67
Figure 6 Throughput Actual and Predicted test model results
16
Selected results (cont’d)
Response time test performance metric
Model
LR
NN
SVR
MAPE
12.35
17.84
9.92
RMSE
1.39
2.02
1.21
MAE
1.11
1.64
0.87
PRED(25)
0.91
0.75
0.93
Figure 7 Response time Actual and Predicted test model results
17
CPU - Comparison of Prediction Models
18
Throughput – Comparison of Prediction
Models
19
Sensitivity Measurement Using Little’s Law
Data Consistency Measurement Using Little’s Law
1-7
56-63 154350490Time (minute)
161
357
497
55
11
103
101
Average total user 27
(188/7) (388/7) (80/7) (724/7) (708/7)
requests
10.52 2.47
15.92 10.00
Average Throughput 5.29
(Requests/second)
5.23
4.45
6.47
10.01
Average time spent 5.10
(seconds)
4.92
9.92
7.53
9.06
Measured time spent 4.66
6.31
55.14 14.08 10.49
Time variance (%) 9.44
498503
95
(664/7)
12.23
504511
69
(480/7)
13.27
7.77
5.2
9.01
13.76
9.45
44.97
20
Conclusion

SVR displayed superior prediction accuracy over both LR and
NN in a typically nonlinear, not defined a-priori workload by:
In the CPU utilization prediction model, SVR
outperformed LR and NN by 58% and 120% respectively

For the Throughput prediction model, SVR again
outperformed LR and NN by 12% and 76% respectively;
and finally,

The Response time prediction model saw SVR
outperforming LR and NN by 26% and 80% respectively.
 Based on this experimental results SVR may be accepted as
the best prediction model in a nonlinear, not defined a-priori
system

21
Future works



SVR and other machine learning algorithms are good for
forecasting, however
• training and retraining is a challenge
• Parameter selection is still empirical
Combination of SVR and other predicting techniques may
mitigate this challenge
Other future direction include
• Inclusion of database tier for a more robust scaling
decision
• Resource prediction on other non-web application
workload
22
Questions
Thank You
23