vii
TABLE OF CONTENTS
CHAPTER
TITLE
DECLARATION
ii
DEDICATION
iii
ACKNOWLEDGEMENT
iv
ABSTRACT
v
ABSTRAK
vi
TABLE OF CONTENTS
vii
LIST OF TABLES
x
LIST OF FIGURES
xi
LIST OF ABBREVIATIONS
1
PAGE
xiii
INTRODUCTION
1
1.1
Introduction
1
1.2
Background Study
3
1.3
Problem Statement
7
1.4
Research Goal
9
1.5
Research Objective
9
1.6
Research Scope
9
1.6
Significance of The Study
10
viii
2
3
LITERATURE REVIEW
11
2.1
Introduction
11
2.2
Autoregressive Integrated Moving Average (ARIMA)
11
2.3
Artificial Neural Network
13
2.4
Least Square Support Vector Machine
17
2.5
Self Organizing Map
20
2.6
Hybrid Model
24
2.7
Input Determination
25
2.8
Summary
26
RESEARCH METHODOLOGY
28
3.1
Introduction
28
3.2
River Flow Forecasting
25
3.2.1
29
River Flow Forecasting Model
3.3
Autoregressive Integrated Moving Average Model
29
3.3.1
ARIMA Tentative Model
31
3.3.1.1
The ARIMA (p, 0, 0) Model
31
3.3.1.2
The ARIMA (0, 0, q Model
33
3.3.1.3
The ARIMA (p, d, q) Model
35
3.3.1.4
The SARIMA (p, d, q) x (P, D,Q)s) Model
36
3.3
Artificial Neural Network Model
37
3.3.1
FeedForward Network
38
3.3.1.1
39
Multilayer Perceptron
3.4.2
Backpropagation Algorithm
40
3.4.3
The ANN Architecture
41
3.4.3.1
The Number of Input Nodes
41
3.4.3.2
The Number of Hidden Layers and
Hidden Nodes
41
3.4.3.3
The Number of Output Nodes
42
3.4.3.3
Activation Function
42
ix
3.5
Least Square Support Vector Machine Model
44
3.5.1
Karush-Kuhn-Tucker
45
3.5.2
Validation Technique
46
3.5.2.1
46
3.5.3
LSSVM Algorithm
46
3.5.1
Kernel
50
3.5.2.1
3.6
4
Kernel Function
50
Self Organizing Map (SOM) Model
52
3.6.1
Train The Network
53
3.6.1.1
Input Determination
53
3.6.1.2
Weight Vector
54
3.6.1.3
Learning Rate
55
3.6.1.4
Determining The Winning Node
55
3.6.1.5
Topological Neighbourhood
56
3.6.1.6
Neighbourhood Function
57
3.6.1.7
Updating The Weight
58
3.6.2
3.7
Cross Validation
SOM Algorithm
Framework of The Study
58
60
DATA COLLLECTION AND ANALYSIS
63
4.1
Introduction
63
4.2
Data Source
63
4.2.1
Case Study 1 (Muda River)
65
4.2.2
Case Study 2 (Selangor River)
66
4.2.3
Case Study 3 (Bernam River)
68
4.3
Statistical Measure
69
4.4
Input Determination
71
4.4.1
Stepwise Regression Analysis
71
4.4.2
Step In Stepwise Regression Analysis
72
x
5
4.4.3
Input determination For Muda River
74
4.4.3
Input determination For Selangor River
76
4.4.3
Input determination For Bernam River
77
EXPERIMENT AND RESULT ANALYSIS
79
5.1
Introduction
79
5.2
Experiment Setup
79
5.3
Fitting The ARIMA Model To The Monthly River Flow
80
5.3.1
Case Study 1 – Muda River
81
5.3.1.1 Identification Step
81
5.3.1.2 Estimation and Diagnostic
Checking Step
5.3.1.3 Forecasting Step
5.3.2
85
89
Case Study2 – Selangor River
90
5.3.2.1 Identification Step
91
5.3.2.2 Estimation and Diagnostic
Checking Step
5.3.3
94
5.3.2.3 Forecating Step
98
Case Study 3 – Bernam River
99
5.3.3.1 Identification Step
100
5.3.3.2 Estimation and Diagnostic
Checking Step
5.3.3.3 Forecasting Step
5.4
5.5
Fitting The ANN Model To The Monthly River Flow
104
104
109
5.4.1
Case Study 1 – Muda River
110
5.4.2
Case Study 2 – Selangor River
113
5.4.3
Case Study 3 – Bernam River
116
Fitting The LSSVM Model To The Monthly River Flow 119
5.5.1
Case Study 1 – Muda River
120
5.5.2
Case Study 2 – Selangor River
121
5.5.3
Case Study 3 – Bernam River
123
xi
5.6
Fitting The SOM-LSSVM Model To The Monthly
River Flow
6
7
125
5.6.1
Case Study 1 – Muda River
126
5.6.2
Case Study 2 – Selangor River
128
5.6.3
Case Study 3 – Bernam River
131
COMPARISON AND DISCUSSION
134
6.1
Introduction
134
6.2
Muda River
134
6.3
Selangor River
139
6.4
Bernam River
143
CONCLUSION AND RECOMMENDATION
148
7.1
Introduction
148
7.2
Conclusion
148
7.3
Contributions
150
7.4
Suggestions For Future Research
151
REFERENCES
153
Appendices A-I
166
xii
LIST OF TABLES
TABLE NO.
TITLE
PAGE
4.1
List of River Name
64
4.2
The Input Data Structures For Muda River
75
4.3
The Input Data Structures For Selangor River
77
4.4
The Input Data Structures For Bernam River
78
5.1
List of MSE and Tentative ARIMA Models for Muda River
85
5.2
The Performance Evaluation for The Tentative Models
86
5.3
List of MSE and Tentative ARIMA Models for Selangor River
94
5.4
The Performance Evaluation for The Tentative Models
95
5.5
List of MSE and Tentative ARIMA Models for Bernam River
104
5.6
The Performance Evaluation for The Tentative Models
105
5.7
The Network Architecture of ANN For Muda River
( n =input nodes)
5.8
The Result for The Training and Testing for
Muda River Using ANN Model
5.9
115
The Network Architecture of ANN for Bernam River
( n =Input Nodes)
5.12
114
The Result for The Training and Testing for
Selangor River Using ANN Model
5.11
112
The Network Architecture of ANN for Selangor River
( n =Input Nodes)
5.10
110
117
The Result for The Training and Testing Bernam
River Using ANN Model
118
xiii
5.13
The Result for The Training and Testing for
Muda River Using LSSVM Model
5.14
The Result for The Training and Testing for
Selangor River Using LSSVM Model
5.15
122
The Result for The Training And Testing for
Bernam River Using LSSVM Model
5.16
120
124
The Result for The Training and Testing Using
A Hybrid Model of SOM-LSSVM With Different
of Map Sizes for Muda River
5.17
127
The Result for The Training And Testing Using
A Hybrid Model of SOM-LSSVM With Different
of Map Sizes for Selangor River
5.18
129
The Result for The Training and Testing Using
A Hybrid Model of SOM-LSSVM With Different
of Map Sizes for Bernam River
6.1
132
The Comparative Performance of ARIMA, ANN,
LSSVM and SOM-LSSM Models for
Muda River.
6.2
135
The Comparative Performance of ARIMA, ANN,
LSSVM and SOM-LSSM Models for
Selangor River.
6.3
139
The Comparative Performance of ARIMA, ANN,
LSSVM and SOM-LSSM Models for
Bernam River.
144
xiv
LIST OF FIGURES
FIGURE NO.
2.1
TITLE
PAGE
Architecture Of A Three-Layer Back Propagation
Neural Network
15
2.2
The Self Organizing Maps Architecture
21
3.1
The Feed Forward Network
38
3.2
Multilayer Perceptron Architecture
40
3.3
The Sigmoid Function
44
3.4
The Linear Function
44
3.5
An Illustrated Of The Proposed Model
60
3.6
The Framework Of Study
62
4.1
Monthly River Flow For Muda River From
January 1962 – December 1989
4.2
Monthly River Flow For Selangor River
From January 1960 – December 2006
4.3
67
Monthly River Flow For Bernam River
From January 1966 – December 2008
5.1
66
69
Non-Stationary Time Series Plot of Muda River
From January 1962 – December 1989
81
5.2(a)
The ACF of Monthly Time Series for Muda River, Kedah
81
5.2(b)
The PACF of Monthly Time Series for Muda River, Kedah
83
5.3
The Stationary Time Series Plot for Muda River After
Differencing From January 1962 – December 1989
5.4 (a)
83
The ACF After Differencing for Monthly Time Series
for Muda River, Kedah
84
xv
5.4(b)
The PACF After Differencing for Monthly Time Series
for Muda River, Kedah
5.5(a)
84
The ACF of Residuals for ARIMA (0, 1, 2) x (3, 0, 2)12
Without Constant for Monthly Time Series
for Muda River, Kedah
5.5(b)
88
The PACF of Residuals for ARIMA (0, 1, 2) x (3, 0, 2)12
Without Constant for Monthly Time Series
for Muda River, Kedah
5.6
The Observed and Predicted ARIMA (0, 1, 2) x (3, 0, 2)12
for Monthly River Flow of Muda River
5.7
88
89
Non-Stationary Time Series Plot of Selangor River
From January 1960 – December 2006
90
5.8(a)
The ACF of Monthly Time Series for Selangor River, Selangor
92
5.8(b)
The PACF of Monthly Time Series for Selangor River, Selangor
92
5.9
The Stationary Time Series Plot of Selangor River After
Differencing From January 1960 – December 2006
5.10(a)
The ACF After Differencing for Monthly Time Series
for Selangor River, Selangor
5.10(b)
93
The PACF After Differencing for Monthly Time Series
for Selangor River, Selangor
5.11(a)
93
94
The ACF of Residuals for ARIMA (1, 0, 2) x (0, 1, 1) 12
Without Constant for Monthly Time Series
for Selangor River, Selangor
5.11(b)
97
The PACF of Residuals for ARIMA (1, 0, 2) x (0, 1, 1) 12
Without Constant for Monthly Time Series
for Selangor River, Selangor
5.12
The Observed and Predicted ARIMA (1, 0, 2) x (0, 1, 1) 12
for Monthly River Flow of Selangor River
5.13
98
99
Non-Stationary Time Series Plot of Bernam River
From January 1966 – December 2008
100
5.14(a)
The ACF of Monthly Time Series for Bernam River, Selangor
101
5.14(b)
The PACF of Monthly Time Series for Bernam River, Selangor
102
xvi
5.15
Stationary Time Series Plot of Bernam River After
Differencing From January 1966 – December 2008
5.16(a)
The ACF After Differencing for Monthly Time Series
for Bernam River, Selangor
5.16(b)
103
The PACF After Differencing for Monthly Time Series
for Bernam River, Selangor
5.17(a)
102
103
The ACF of Residuals for ARIMA (2, 0, 0) x (2, 0, 2)12
Without Constant for Monthly Time Series
for Bernam River, Selangor
5.17(b)
107
The PACF of Residuals for ARIMA (2, 0, 0) x (2, 0, 2)12
Without Constant for Monthly Time Series
for Bernam River, Selangor
5.82
The Observed and Predicted ARIMA (2, 0, 0) x (2, 0, 2)12
for Monthly River Flow of Bernam River
5.19
128
The Observed and Predicted SOM-LSSVM for Monthly
River Flow of Selangor River
5.27
125
The Observed and Predicted SOM-LSSVM for Monthly
River Flow of Muda River
5.26
123
The Observed and Predicted LSSVM for Monthly
River Flow of Bernam River
5.25
121
The Observed and Predicted LSSVM for Monthly
River Flow of Selangor River
5.24
119
The Observed and Predicted LSSVM for Monthly
River Flow of Muda River
5.23
116
The Observed and Predicted ANN for Monthly
River Flow of Bernam River
5.22
113
The Observed and Predicted ANN for Monthly
River Flow of Selangor River
5.21
108
The Observed and Predicted ANN for Monthly
River Flow of Muda River
5.20
107
130
The Observed and Predicted SOM-LSSVM for Monthly
River Flow of Bernam River
133
xvii
6.1(a)
The Observed and Predicted River Flow During Testing
Period By ARIMA, For Muda River
6.1(b)
The Observed and Predicted River Flow During Testing
Period By ANN For Muda River
6.1(c)
146
The Observed and Predicted River Flow During Testing
Period By SOM-LSSVM For Bernam River
6.6
145
The Observed and Predicted River Flow During Testing
Period By LSSVM For Bernam River
6.5 (d)
145
The Observed and Predicted River Flow During Testing
Period By ANN For Bernam River
6.5 (c)
143
The Observed and Predicted River Flow During Testing
Period By ARIMA, For Bernam River
6.5 (b)
142
The Boxplot for Predicted River Flow Using ARIMA,
ANN, LSSVM And SOM-LSSVM For Selangor River
6.5 (a)
141
The Observed and Predicted River Flow During Testing
Period By SOM-LSSVM, For Selangor River
6.4
141
The Observed and Predicted River Flow During Testing
Period By LSSVM, For Selangor River
6.3 (d)
140
The Observed and Predicted River Flow During Testing
Period By Ann, For Selangor River
6.3 (c)
138
The Observed and Predicted River Flow During Testing
Period By ARIMA, For Selangor River
6.3 (b)
137
The Boxplot For Predicted River Flow Using ARIMA,
ANN, LSSVM And SOM-LSSVM For Muda River
6.3 (a)
137
The Observed and Predicted River Flow During Testing
Period By SOM-LSSVM For Muda River
6.2
136
The Observed and Predicted River Flow During Testing
Period By LSSVM For Muda River
6.1 (d)
136
146
The Boxplot for Predicted River Flow Using ARIMA,
ANN, LSSVM And SOM-LSSVM For Bernam River
147
xviii
LIST OF SYMBOLS
p
-
The non seasonal auto regression
d
-
The regular differencing
q
-
The non seasonal moving average
P
-
The seasonal auto regression
D
-
The seasonal differencing
Q
-
The seasonal moving average
S
-
The length of season
xt
-
The current value of the time series

-
The constant
at
-
The random errors
1 ,  2 ,....,  p
-
The AR parameters of the model
1 ,  2 ,  q
-
The MA parameters of the model

-
The mean of time series
 t 1 ,  t  2 ,...,  t  q
-
The errors of the model
yt
-
The output layers,
xt i
-
The input of the network,
wj
-
The connection weights between nodes of input and
hidden layers
wi
-
The connection weights between nodes of hidden and
output layers,
g (.) and f (.)
-
The activation function.
 (.)
-
Map from input space to feature space
xix
J
-
The loss function
w, e
-
The output layer vector or parameter vector in
primal space
wT
-
The transpose of w

-
The regularization constants
b
-
Bias term
L
-
The Lagrangian
 (t )
-
The learning rate
 (t  1)
-
updated learning rate
j
-
The updated neighbourhood function
|| X  W j ||
-
Represent Euclidean distance
 (t )
-
The bandwidth of the Gaussian RBF
 (t )
-
The learning rate at time t
W j (t )
-
The old weight vector.
h ji
-
The neighbourhood function
W j (t  1)
-
The The adjusted weight vector at time (t  1) .
xx
LIST OF ABBREVATIONS
ACF
-
Autocorrelation Function
ANN
–
Artificial Neural Network
AR
–
Autoregressive
ARIMA
–
Autoregressive Integrated Moving Average
BP
-
Backpropagation
FFN
-
Feed Forward Network
LSSVM
–
Least Square Support Vector Machine
MA
–
Moving Average
MLP
-
Multilayer Perceptron
MAE
-
Mean Absolute Error
MSE
-
Mean Square Error
PACF
-
Partial Autocorrelation Function
R
-
Correlation Coefficient
RMSE
-
Root Mean Square Error
SARIMA
-
Seasonal Autoregressive Integrated Moving Average
SOM
–
Self Organizing Map
SVM
–
Support Vector Machine
SVR
-
Support Vector Regression
xxi
LIST OF APPENDICES
APPENDIX
TITLE
PAGE
A1
Monthly River Flow Data For Muda River
166
A2
Monthly River Flow Data For Selangor River
168
A3
Monthly River Flow Data For Bernam River
171
B1
Input Determination For Muda River (K7)
174
B2
Input Determination For Selangor River (L7)
180
B3
Input Determination For Bernam River (M7)
190
C1
Input Determination For Selangor River (L8)
199
C2
Input Determination For Bernam River (M8)
208
D1
Comparison Between The Observations And Predictions Using
ARIMA For Monthly River Flow In Muda River
D2
Comparison Between The Observations And Predictions Using
ANN For Monthly River Flow In Muda River
D3
220
Comparison Between The Observations And Predictions Using
ANN For Monthly River Flow In Selangor River
E3
219
Comparison Between The Observations And Predictions Using
ARIMA For Monthly River Flow In Selangor River
E2
218
Comparison Between The Observations And Predictions Using
SOM-LSSVM For Monthly River Flow In Muda River
E1
217
Comparison Between The Observations And Predictions Using
LSSVM For Monthly River Flow In Muda River
D4
216
221
Comparison Between The Observations And Predictions Using
LSSVM For Monthly River Flow In Selangor River
222
xxii
E4
Comparison Between The Observations And Predictions Using
SOM-LSSVM For Monthly River Flow In Selangor River
F1
Comparison Between The Observations And Predictions Using
ARIMA For Monthly River Flow In Bernam River
F2
I
231
The Predicted Monthly River Flow In Selangor River
Using ARIMA
H3
230
The Predicted Monthly River Flow In Muda River
Using ARIMA
H2
229
The Estimated Parameter For Tentative Models For
Bernam River
H1
228
The Estimated Parameter For Tentative Models For
Selangor River
G3
227
The Estimated Parameter For Tentative Models For
Muda River
G2
226
Comparison Between The Observations And Predictions Using
SOM-LSSVM For Monthly River Flow In Bernam River
G1
225
Comparison Between The Observations And Predictions Using
LSSVM For Monthly River Flow In Bernam River
F4
224
Comparison Between The Observations And Predictions Using
ANN For Monthly River Flow In Bernam River
F3
223
232
The Predicted Monthly River Flow In Bernam River
Using ARIMA
233
The List of Publication
234