See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/225155100 Improved Water Level Forecasting Performance by Using Optimal Steepness Coefﬁcients in an Artiﬁcial Neural Network Article in Water Resources Management · August 2011 DOI: 10.1007/s11269-011-9824-z CITATIONS READS 34 695 4 authors: Muhammad Sulaiman Ahmed El-Shafie Universiti Malaysia Pahang Universiti Kebangsaan Malaysia 2 PUBLICATIONS 35 CITATIONS 131 PUBLICATIONS 3,173 CITATIONS SEE PROFILE SEE PROFILE Othman A Karim Hassan Basri Universiti Kebangsaan Malaysia Universiti Kebangsaan Malaysia 63 PUBLICATIONS 909 CITATIONS 164 PUBLICATIONS 2,694 CITATIONS SEE PROFILE Some of the authors of this publication are also working on these related projects: Intelligent Solid Waste monitoring and Management for SmartCity/SmartVillage View project Phytoremediation of Arsenic using Terrestrial Plant View project All content following this page was uploaded by Muhammad Sulaiman on 21 September 2015. The user has requested enhancement of the downloaded file. SEE PROFILE Water Resour Manage (2011) 25:2525–2541 DOI 10.1007/s11269-011-9824-z Improved Water Level Forecasting Performance by Using Optimal Steepness Coefficients in an Artificial Neural Network Muhammad Sulaiman · Ahmed El-Shafie · Othman Karim · Hassan Basri Received: 20 August 2010 / Accepted: 4 April 2011 / Published online: 24 May 2011 © Springer Science+Business Media B.V. 2011 Abstract Developing water level forecasting models is essential in water resources management and flood prediction. Accurate water level forecasting helps achieve efficient and optimum use of water resources and minimize flooding damages. The artificial neural network (ANN) is a computing model that has been successfully tested in many forecasting studies, including river flow. Improving the ANN computational approach could help produce accurate forecasting results. Most studies conducted to date have used a sigmoid function in a multi-layer perceptron neural network as the basis of the ANN; however, they have not considered the effect of sigmoid steepness on the forecasting results. In this study, the effectiveness of the steepness coefficient (SC) in the sigmoid function of an ANN model designed to test the accuracy of 1-day water level forecasts was investigated. The performance of data training and data validation were evaluated using the statistical index efficiency coefficient and root mean square error. The weight initialization was fixed at 0.5 in the ANN so that even comparisons could be made between models. Three hundred rounds of data training were conducted using five ANN architectures, six datasets and 10 steepness coefficients. The results showed that the optimal SC improved the forecasting accuracy of the ANN data training and data validation when compared with the standard SC. Importantly, the performance of ANN data training improved significantly with utilization of the optimal SC. Keywords Artificial neural networks · Sigmoid function · Steepness coefficient · Water level forecasting M. Sulaiman (B) Faculty of Civil Engineering and Natural Resources, University Malaysia Pahang, Pahang, Malaysia e-mail: [email protected] A. El-Shafie · O. Karim · H. Basri Department of Civil Engineering, Faculty of Engineering, National University of Malaysia, Bangi, Malaysia 2526 M. Sulaiman et al. 1 Introduction River flow forecasting is essential in water resources management because it can facilitate the management of water resources, thereby optimizing the use of water. The ability to forecast river flow also helps predict the occurrence of future flooding, enabling better preparation to avoid the loss of lives and minimize property damage. Forecasting studies normally require a series of historical datasets so that future events can be predicted based on past events. It is vital for local agencies such as the water authority to maintain good quality river flow data to facilitate reliable river flow forecasting. The artificial neural network (ANN) is a computing model that can solve nonlinear problems from a series of sample datasets. ANN computing is based on the way the human brain processes information. The ANN can define hidden patterns within sample datasets and forecast based on new dataset inputs. Thus, in many areas of study such as forecasting river flow, the ANN does not require defined physical conditions of the subject, which in this case is the river. Other problems such as regression, classification, prediction, system identification, feature extraction and data clustering can also be solved through ANN computing. ANNs have been widely applied in many areas including the financial, mathematical, computer, medicinal, weather forecasting and engineering fields. In water resources studies, ANNs are employed to forecast daily river flow (Atiya et al. 1999; Coulibaly et al. 2000; Ahmed and Sarma 2007; El-Shafie et al. 2008; Wu et al. 2009), water levels (Bustami et al. 2007; Leahy et al. 2008), flood events (Tareghian and Kashefipour 2007; Kerh and Lee 2006), rainfall runoff patterns (Chiang et al. 2004; Agarwal and Singh 2004; Rahnama and Barani 2005), reservoir optimization (Cancelliere et al. 2002; Chandramouli and Deka 2005) and sedimentation (Cigizoglu and Kisi 2006; Rai and Mathur 2008). For a general review of the application of ANN in hydrology, refer to ASCE (2000a, b). The objective in studies of ANN forecasting is to identify the best forecasting model that can provide the most accurate forecasting results possible. This is achieved by modifying key components in the ANN during data training. These essential parameters are the number and type of data inputs, number of hidden layers and neurons, activation of transfer function, and optimization method to identify the weight in neurons. In river flow forecasting, data inputs can be generated from historical river flow data, rainfall, precipitation and sedimentation. El-Shafie et al. (2007, 2008, 2009) conducted river flow forecasting of the Nile River using river flow data obtained from a single station. Turan and Yurdusev (2009) used a multiple upstream river flow station to forecast river flow. Chiang et al. (2004) and Rahnama and Barani (2005) included rainfall and runoff data when conducting river flow forecasting, and Zealand et al. (1999) included precipitation, rainfall and flow data in their forecasting study. Fernando et al. (2005) suggested several methods to identify the proper inputs to a neural network. Other studies (Alvisi et al. 2006; Toth and Brath 2007) have investigated the effects of the number and type of inputs on the ANN performance. Once data inputs have been determined, selection of the number of hidden layers and neurons plays a vital role in achieving the best forecasting performance. Determination of the number of layers and neurons used in forecasting studies has generally been based on a trial-and-error approach (Coulibaly et al. 2000; Improved Water Level Forecasting Performance 2527 Joorabchi et al. 2007; Solaimani and Darvari 2008; Turan and Yurdusev 2009). Many studies have shown that one hidden layer is sufficient. Indeed, Hornik et al. (1989) found that multilayer feed forward networks with one hidden layer were capable of approximating to any desired degree of accuracy provided sufficient hidden units were available. Zhang et al. (1998) reviewed published studies of ANN and found that a single hidden layer is most popular and widely used in the layer selection. Two hidden layers are also able to produce the best forecasting performance in certain problems (Barron 1994). However, selection of the number of ANN neurons is still based on trial and error (Chauhan and Shrivastava 2008). Activation transfer function (ATF) is the main computing element of ANN and plays an important role in achieving the best forecasting performance. The most common type of activation transfer function is the sigmoid function (Zhang et al. 1998). However, several studies have used different types of ATFs within the ANN to improve the forecasting performance. Shamseldin et al. (2002) used logistic, bipolar, hyperbolic tangent, arc-tan and scaled arc-tan to explore the potential improvement of ANN forecasting. Joorabchi et al. (2007) applied log-sigmoid and hyperbolic tangent sigmoid transfer functions to produce their output. Han et al. (1996) introduced optimization of the variant sigmoid function using a genetic algorithm to optimize the ANN convergence speed and generalization capability. Many researchers have studied the exterior architecture of the ANN using a trial and error approach. Others have investigated the interior architecture of the ANN, but have been limited to testing different types of ATF in the ANN architecture. The sigmoid function, which is the most commonly used computing function for ATF, has been widely used in the ANN because of its ability to influence the performance of the ANN. This study was conducted to evaluate the effectiveness of the steepness coefficient in a sigmoid function for improving ANN data training and forecasting in a river flow study. Additionally, the effectiveness of the optimal steepness coefficient approach for the exterior architecture in river flow forecasting was compared with that of the traditional approach based on trial and error. The improved performance of the ANN water level forecasting could assist water authorities in managing water resources. 2 Methodology In this study, sigmoid functions with different steepness coefficients were evaluated as activation transfer functions in an ANN to improve the forecasting water level at Rantau Panjang Station, Johor Baru, Malaysia. The performance of the ANN data training and data validation were measured using the statistical index Nash-Sutcliffe efficiency coefficient (NS). The Root Mean Square Error (RMSE) was also used to measure the accuracy of the data forecasting performance. 2.1 Artificial Neural Network The ANN is a non-linear mathematical computing model that can solve arbitrarily complex non-linear problems such as time series data forecasting. The ANN can identify hidden patterns in sample datasets and forecast based on these patterns. ANN computing is based on the way human brains process information. Indeed, 2528 M. Sulaiman et al. the ANN is a non-linear solver that needs to compute constant terms or unknown parameters before the x’s or data input can be computed. The general ANN nonlinear equation can be defined as follows: y1 , . . . yk = f (x1 , . . . xm , . . . c1 , . . . cn ) (1) where y is the forecasted output, k is the number of forecasted outputs, x is the input, m is the number of data input, c is a constant term and n is the number of constant terms. The constant term c is referred to as the weight in ANN modeling. y can be computed if m, the c values and n are known. The x values are data input. To determine the m, c values and n, it is necessary to understand the ANN architecture. The architecture of ANN is dependent on the type of ANN that is used. In this study, a multi-layer perceptron back-propagation (MLP-BP) neural network, which is the most common type of ANN used in forecasting studies, was employed. The reason for the popularity of the MLP-BP is its simplicity, easy implementation and demonstrated success in forecasting studies. Figure 1 provides an example of the MLP-BP architecture. There are three layers in MLP-BP, an input layer, a hidden layer and an output layer. There can be only one input layer and one output layer; however, there can be more than one hidden layer. Each layer contains stored neurons, with the total number of neurons being determined by the user. The number of neurons in the input layer is equivalent to the number of data inputs selected in a study. The number of output neurons in the output layer is generally one, resulting in one forecasted output. The number of neurons in the hidden layer is subject to user selection. Most often, the number of input neurons and hidden neurons are determined based on trial and error. An additional dummy neuron known as a bias neuron that holds a value of 1 is added to the input layer and each hidden layer. The bias neuron acts as a threshold value so that the value of the forecasted output falls between 0 and 1. The neurons are interconnected between layers as shown in the figure. The direction of the link is from the input layer to the output layer, and these links represent the computational process within the ANN architecture. The actual computational process occurs in the neuron, in which the activation transfer function assigned to the neuron is used to compute the incoming value and produce an output. In the input neuron, the activation transfer functions relay the incoming data input as output. In the hidden and output neurons, the activation transfer function computes the incoming value and produces an output. In the input neurons, the activation transfer function employed is a linear transfer function. In hidden and Fig. 1 Multi-layer perceptron back-propagation neural network in the study Improved Water Level Forecasting Performance 2529 Fig. 2 An active neuron with sigmoid function and threshold input output neurons, the activation transfer function that is commonly used is a sigmoid function. Figure 2 shows a neuron with a sigmoid function that computes incoming data and produces an output. The data received by the neuron is the summation of the output of the previous neuron based on its weight. Finally, the weight c values can be determined by data training. Data training can be accomplished when the number of neurons in each layer is defined, an activation transfer function is assigned to the neurons in each layer, and training data are available. The data training process involves two parts of computation, feed-forward and back propagation. In the feed-forward computation, computing starts from the input layer and proceeds to the output layer based on the links and ATF in the ANN architecture. The output of the feed-forward computation is the forecasted value. In the data training process, performance is evaluated based on the model output value with respect to the observed value. If the measured performance does not achieve the target performance, a back-propagation computation takes place. The backpropagation computation is a process of adjusting the weights in the architecture based on the gradient descent method. The process of the weight adjustment starts from the output layer and proceeds backward toward the input layer. The process of feed-forward and back-propagation continues until the performance target is achieved. Once the data training process is completed, the weights of the c values are determined. Then, the forecasting process proceeds based on the single feed-forward computation utilizing the input data as shown in the model architecture. The results are then evaluated by applying performance measures to the observed data versus the model output values. 2.2 Sigmoid Function The activation transfer function (ATF) forces incoming values to range between 0 to 1 and 1 to −1 depending on the type of function used. The most commonly used ATF in the MLP-BP is the sigmoid function. The sigmoid function is a differentiable function in which the gradient method can be applied to the ANN to adjust the ANN weights so that the model output and observed data reach a target performance value during data training. The sigmoid function is defined as follows: y= 1 1 + e−kx (2) 2530 M. Sulaiman et al. where y is the sigmoid value, k is the sigmoid steepness coefficient and x is the data or incoming values. As in the case of the neural network, the incoming values are the summation of the input and weight values. Additional input with a value of 1 and its weights are added as a threshold value so that the computed sigmoid function will result in an activation value between 0 and 1. 2.3 Case Study Area and Dataset The case study area in the present study was Rantau Panjang station, which is located along the bank of the Johor River (Fig. 3). More than 12 flood events have occurred in Rantau Panjang since 1963. All occurrences of flooding were caused by river flow from the upper stream and heavy rainfall during the Northeast monsoon, which is between December and January (except in 1964). The Johor River, which is located at the central part of southern Johor, originates from Mount Gemuruh (109 m) and Mount Belumut (1,010 m). The river length is about 122.7 km with a drainage area of 2,636 km2 . The river flows through the southern part of Johor and discharges into the Straits of Johor (0 m). The main tributaries of the river are the Linggiu and Sayong rivers. Data for this study were provided by the Department of Irrigation and Drainage, Ampang, Selangor. The data consist of 45 years of historical hourly water level data obtained from 1963 to 2007. The collected data were divided into two categories; training data and validation data. Due to missing daily data in the historical records, the training and validation data were divided into smaller groups of continuous datasets. Six water level datasets were extracted from 1963 to 1986 for data training and a single dataset with 1,144 daily water level data points extracted from 1994 to Fig. 3 The Johor River basin Improved Water Level Forecasting Performance 2531 2007 for data validation. The description of the training and validation datasets is shown in Table 1. The datasets in this study were normalized based on the following equation: N= Oi − Omin Omax − Omin (3) where N is the normalized value, O is the observed value, Omax is the maximum observed value and Omin is the minimum observed value. 2.4 Implementation A total of 10 sigmoid functions based on 10 different steepness coefficients (Fig. 4; Table 2) were used as the activation transfer functions in the hidden and output neurons. The steepness coefficients ranged from 0.025 to 1. The steepness coefficient with a value of one, which is commonly used as the default steepness coefficient in sigmoid functions, was used as the Standard Steepness Coefficient (SSC). The other nine steepness coefficients ranging from 0.025 to 0.7 were referred to as Milder Steepness Coefficients (MSC) and numbered from 1 to 9. As an example, MSC_1 refers to a steepness coefficient of 0.7. Pre-analysis of a steepness coefficient greater than one shows poor performance in data training and validation of the water level; thus, it is not described in this paper. Previous studies (Atiya et al. 1999; Solaimani and Darvari 2008) have revealed that a single layer is sufficient for forecasting river flow. Therefore, in this study, an MLP-BP with one hidden layer was selected. In the ANN model tested in this study, 3, 4, 5, 6, and 7 input data values were investigated. The number of input data refers to the number of multi-lead days ahead of the forecasting day. The number of input neurons was the same as the number of data input, while the number of output neurons was one, referring to 1 day ahead of water level forecasting. The number of neurons in the hidden layer was equal to the number of input neurons. Based on these findings, five MLP-BP architectures ranging from Net1 to Net5 were developed (Table 3). The MLP-BP architecture used in this study is shown in Fig. 1. The goal of ANN data training is to determine the weights in the ANN architectures by processing sets of data input from historical data through iteration. Generally, the weights in the network model are initialized with random values between value 0 and 1. This method helps achieve better forecasting performance when compared with conditions in which the initial weights are initialized with fixed values. However, to enable equal data forecasting comparisons between different network models, the initial weights in all network models in this study were initialized Table 1 Training and forecasting datasets used in the study Dataset Date from Date to Water level (days) Purpose DS_1 DS_2 DS_3 DS_4 DS_5 DS_6 DS_7 05-08-1963 16-12-1971 12-05-1976 15-02-1979 27-11-1979 20-03-1981 16-02-2004 15-07-1970 01-03-1976 08-12-1977 13-11-1979 09-03-1981 24-11-1986 02-05-2007 2,264 1,539 576 272 470 1,276 1,144 Data training Data training Data training Data training Data training Data training Data forecasting 2532 M. Sulaiman et al. Fig. 4 Sigmoid function with ten different steepness coefficients. MSC milder steepness coefficient with a fixed value of 0.5. By doing so, each data training process started with the same position of weight values, thereby enabling even comparisons of the forecasting performance of different network models. Another, parameter that needs to be fixed is the epoch so that equal forecasting performance of the different network models can be made. Thus, an early stopping procedure (ASCE 2000a), which is the best approach to stopping data training, was not implemented in this study. In summary, 300 data training models based on six training datasets, 10 steepness coefficients and five ANN architectures were tested. The ANN weights were initialized with a fixed value of 0.5, and 2000 epochs were used for the data training. The model performance was tested based on a single validating dataset generated from the best known training performances of the SSC and the optimal MSC of the six training datasets. Comparisons of the observed and forecasting water levels were charted to show the results of the data validation. 2.5 Performance Evaluation Data training performance in this study was evaluated using the Nash-Sutcliffe coefficient of efficiency (NS). The efficiency NS is a statistical index widely used Table 2 The ten steepness coefficients in the sigmoid function Steepness coefficient Value Sigmoid function SSC MSC_1 MSC_2 MSC_3 MSC_4 MSC_5 MSC_6 MSC_7 MSC_8 MSC_9 1.00 0.70 0.50 0.35 0.25 0.17 0.100 0.075 0.050 0.025 1/(1 + e−x ) 1/(1 + e−0.7x ) 1/(1 + e−0.5x ) 1/(1 + e−0.35x ) 1/(1 + e−0.25x ) 1/(1 + e−0.17x ) 1/(1 + e−0.1x ) 1/(1 + e−0.075x ) 1/(1 + e−0.05x ) 1/(1 + e−0.025x ) Improved Water Level Forecasting Performance Table 3 The five ANN architectures 2533 ANN architecture Input neuron Hidden neuron Output neuron Weights Net1 Net2 Net3 Net4 Net5 3 4 5 6 7 3 4 5 6 7 1 1 1 1 1 16 25 36 49 64 to describe the forecasting accuracy of hydrological models. An NS with a value of one indicates a perfect fit between the modeled and observed values. In addition, an NS with a value of greater than 0.9 indicates satisfactory performance results, while a value greater than 0.95 indicates good performance results between the two data series. The NS is defined as: N NS = 1 − i=1 N (Oi − Fi )2 (4) (Oi − O)2 i=1 where O is the observed value, F is the forecasted value and N is the number of data being evaluated. The model performance was measured using the NashSutcliffe’s coefficient of efficiency and root mean square error (RMSE). The RMSE describes the average magnitude of error of the observed and forecasted values and is defined as: N RMSE = (Oi − Fi )2 i=1 N (5) where the terms above are the same as in Eq. 4. The units for RMSE in this study are in millimeters. 3 Results and Discussion The data training and data validation results for the five ANN architectures trained with six datasets, 10 different steepness coefficients in sigmoid function and 2000 epochs are presented herein. Figure 5 shows the data training performance for the five ANN architectures in which each of the six datasets were trained with 10 steepness coefficients. All datasets trained with MSC in the ANN architectures generally had better performance than datasets that were trained with the SSC. Milder MSC values such as MSC_3 to MSC_9 tended to produce better performance than SSC, MSC_1 and MSC_2. However, this does not necessarily indicate that milder steepness coefficients produce better training performance. Figure 6 shows an enlargement of Fig. 5e for data training performance greater than 0.9. The figure shows a variation in dataset performance above 0.9 for Net5, which is also indicated for Net1 to Net4. Dataset DS_1 shows decreasing performance starting from MSC_3, which demonstrates that a milder steepness coefficient is not necessarily better. Datasets DS_2, DS_4 and 2534 M. Sulaiman et al. (a) Net1 (c) Net3 (b) Net2 (d) Net4 (e) Net5 Fig. 5 Data training performances of the six datasets with five different network models and ten steepness coefficients DS_6 indicate that the best performance occurred when a steepness coefficient of MSC_9, which was the steepest MSC tested, was used. In fact, the optimal MSC could occur at or between MSC_3 to MSC_9. As shown in Fig. 6, datasets DS_1, DS_3 and DS_6 showed strong data training performance where the performance for the SSC training had NS values close to or greater than 0.9, while datasets DS_2, DS_4 and DS_5 had poor data training performance when the NS values were below 0.85, and occasionally when the values were lower than 0.7. However, for the poor data training performance, the NS values improved to greater than 0.85, Improved Water Level Forecasting Performance 2535 Fig. 6 Enlargement of data training performance for Net5. DS dataset. MSC milder steepness coefficient and most to greater than 0.9, as the steepness coefficient in the range of MSC_4 and MSC_9 was used. The results shown in Figs. 5 and 6 indicate that an optimal steepness coefficient existed in all ANN architectures and datasets tested, which gave optimal data training performance and better data training results when compared with the standard steepness coefficient. Indeed, even bad training datasets improved significantly to an NS value above 0.9, which can be considered satisfactory. Figure 7 shows a summary of data training performances from Fig. 5 of five sets of ANN architectures that were trained with SSC and another five sets of ANN architectures that were trained with the optimal MSC for each of the datasets. The Fig. 7 Comparison of data training performances based on the standard steepness coefficient (SSC) and optimal milder steepness coefficients (MSC) 2536 M. Sulaiman et al. optimal MSC refers to the best data training performance among the nine MSCs for the specific dataset and ANN architecture. For data training with the SSC, the performance of the six datasets varied depending on the ability to achieve high NS values using the same ANN architecture. For example, for dataset DS_1, the range of NS varied from 0.893 to 0.922. In the trial and error method, if the ANN architecture is to be selected based on dataset DS_1 for data validation, then the ANN architecture Net3 will be used. This differs from data training using the optimal MSC, in which the data training performance for different ANN architectures are within 0.940. Thus, any ANN architecture trained with the optimal MSC can be used for data validation. Another example is dataset DS_2, for which the data training performance varies from 0.100 to 0.790. ANN architecture Net5 will be selected for validation if in the trial and error method, but for data training with the optimal MSC, any ANN architecture can be used in data validation since the results for different ANN architectures is within 0.908. These findings indicate that different ANN architectures do not affect the optimal data training performance. Instead, the optimal steepness coefficient that seems to exist in all ANN architectures strongly influences the data training performance results. The same conclusion is shown for dataset DS_3 to DS_6. The figures and tables show that the optimal MSC is a better approach than the trial and error method and that it produces a better result than SSC. These results or findings may help to determine future studies in ANN in which use of the optimal steepness coefficient method could be a better approach than trial and error for identifying the optimal data training performance. Table 4 shows the data training performance for the six datasets with the best SSC and optimal MSC based on data training using six ANN architectures and 10 steepness coefficients. It could be observed from Fig. 7 that the proposed modifications in MSC enhance the performance for all data sets over SSC. For more details, from Table 4, for data sets resultant in relatively good performance (DS_1, DS_3 and DS_6) utilizing SSC an improvement NS ranged between 0.9% and 2% could be observed. On the other hand, for the data sets experienced relatively poor performance using SSC, significant enhancement for NS between 12% and 14% has been reached. In fact, such improvements in the model performance are significant since achieving NS more than 0.9 is considered as satisfactory performance. Table 5 shows the model performance based on the best SSC and optimal MSC of data training models using validation dataset(DS_7). While examining DS_7, it can be depicted from Table 5 “column 2” that the model performance values NS were found to be consistent between training (DS_1 to DS_6), as shown in Table 4 “column 2” utilizing SSC architecture. On the other hand, for dataset Table 4 Data training performances (NS) of the six training datasets Dataset Best SSC Best optimal MSC Improvement (%) DS_1 DS_2 DS_3 DS_4 DS_5 DS_6 0.922 (Net1) 0.790 (Net5) 0.938 (Net1) 0.794 (Net1) 0.805 (Net1) 0.938 (Net5) 0.942 (Net5:MSC_3) 0.915 (Net5:MSC_9) 0.952 (Net5:MSC_4) 0.921 (Net5:MSC_9) 0.940 (Net5:MSC_7) 0.947 (Net5:MSC_9) 2.0 12.5 1.4 12.7 13.5 0.9 Improved Water Level Forecasting Performance 2537 Table 5 Forecasting performances (NS) of dataset DS_7 Dataset Best SSC Best optimal RMSE (mm) Improvement (%) DS_1 DS_2 DS_3 DS_4 DS_5 DS_6 0.913 (Net1) 0.873 (Net5) 0.923 (Net1) 0.615 (Net1) 0.867 (Net1) 0.930 (Net5) 0.938 (Net5:MSC_3) 0.928 (Net5:MSC_9) 0.938 (Net5:MSC_4) 0.924 (Net5:MSC_9) 0.924 (Net5:MSC_7) 0.938 (Net5:MSC_9) 344 370 344 381 382 344 2.5 5.5 1.5 30.9 5.7 0.8 DS_2 and DS_5, which are based on the SSC, the performance was better than the data training. Model performance based on the SSC range varied widely from 0.615 to 0.930; however, the forecasting performance of relatively good performance datasets (DS_1, DS_3 and DS_6) based on the optimal MSC had similar values of 0.938. Conversely, the relatively poor performance datasets (DS_2, DS_4 and DS_5) showed consistent performance values of 0.921 to 0.928. These findings indicate that the optimal forecasting performance was achieved by different good performance datasets (DS_1, DS_3 and DS_6) using the optimal MSC approach. Figure 8 shows the model performance of DS_7 based on data training using dataset DS_4 and SSC, while Fig. 9 shows the model performance of DS_7 based on data training using dataset DS_4 and the optimal MSC (MSC_9). In fact, the use of DS_4 as a training dataset is because it provides the worst data forecasting performance based on the SSC, but improved drastically while using the optimal MSC. The figures show a comparison of the observed and forecasted water level and a scatter plot of the performance of the data forecasting. The scatter plot presented in Fig. 8 shows that most data move away from the equal or middle line, while the scatter plot in Fig. 9 shows that most data are centered around the middle line. These findings show that the data forecasting based on the SSC has an NS of 0.615, while the data forecasting based on the optimal MSC has an NS of 0.924. These results demonstrate that the optimal MSC greatly improved the poor forecasting dataset. Figure 10 shows the forecasting performance of DS_7 based on data training using dataset DS_6 and SSC. Figure 11 shows the forecasting performance of DS_7 based on data training using dataset DS_6 and the optimal MSC, which is MSC_9. The use of DS_6 as a training dataset produced the best data forecasting performance based Fig. 8 Model performance of ANN based model using DS4, Net1 and standard steepness coefficient 2538 M. Sulaiman et al. Fig. 9 Model performance of ANN based model using DS4, Net5 and milder steepness coefficient MSC_9 on the SSC and optimal MSC. The scatter plots of the forecasting performances of the two models could be similar, except that in the scatter plot in Fig. 11 showed more data centered around the middle line than the plot shown in Fig. 10 specially for the observed data less than 6,000 mm. This accuracy is reflected by the performance of the SSC and optimal MSC, which had NS values of 0.93 and 0.938, respectively. The results shown in both scatter plots were slightly better than those shown in the scatter plot in Fig. 9, which had an NS value of 0.924. These findings confirm the existence of an optimal MSC in the sigmoid function that produced the optimal forecasting performance. Based on these findings, the MSC are able to provide good forecasting results; however, the mechanism by which MSC produces sigmoid values between 0 and 1 at the output neuron is unclear since it appears to have a limited range of sigmoid values (Fig. 4). For example, MSC_9 appears to produce sigmoid values limited to around 0.5. This is because the sigmoid functions shown in Fig. 4 based on the xvalues between 4 and −4. Figure 4 is suited to view sigmoid values between 0 and 1 for the SSC. However, if expanded to x-values between 100 and −100, then the sigmoid values between 0 and 1 are visible for all MSCs. For MSC 9, the sigmoid value is 0.1 if the x-value is −88, while it is 0.9 if the x-value is 88. For the SSC, to produce a sigmoid value of 0.1 and 0.9, the x-value must be −2.2 and 2.2 respectively. Thus, this explanation shows that all MSC can produce sigmoid values between 0 and 1 if appropriate x-values are passed to the sigmoid function. The mechanism that enables the summation of w and x values passed to the hidden and output neurons to have a high value is also unclear. Based on the two parameter w and x values Fig. 10 Model performance of ANN based model using DS6, Net5 and standard steepness coefficient Improved Water Level Forecasting Performance 2539 Fig. 11 Model performance of ANN based model using DS6, Net5 and milder steepness coefficient MSC_9 in the summation, only a high value of weights can increase the summation. This is because the x values will always fall between 0 and 1. Figure 12 shows an example of the bias weights of SSC and MSC_9 that are passed to the output neuron during data training. As shown in the figure, the weight for MSC_9 increased dramatically when compared with the SSC during the data training process. The weight generated by SSC was only 2.48424, but that of MSC_9 was 86.12270 after 2000 epochs. It should be noted that both bias weights were initialized with a value of 0.5 at the start of the data training process. Thus, these findings confirmed that the weights in the ANN model influence the output of MSC between 0 and 1. Linear transfer function has been successfully applied to the output layer to improve the forecasting performance of the ANN model (Toth and Brath 2007; Hornik et al. 1989). However, in our case study, based on the bias weight explanation and its effects on the summation value that is passed to output neurons, it is not possible to have a linear transfer function in the output layer and an MSC in the hidden layer. This is due to the rigidity of the back propagation algorithm to re-adjust the weights of hidden layer and output layer in case a considerable difference in the weight values is experienced. Actually, the use of MSC in hidden neurons will present a wide range different in the weight values in hidden and output layer, which turn into unfeasibility to use linear transfer function in the output neuron. However, there are several optimized algorithm rather than back propagation such as genetic algorithm and particle swarm optimization that could help overcome this drawback. Fig. 12 Weight of bias connected to output neuron (a) using standard steepness coefficient, (b) using milder steepness coefficient MSC_9 2540 M. Sulaiman et al. 4 Conclusions The results of this study demonstrated that the optimal steepness coefficient of the sigmoid function effectively improved the ANN data training performance when compared with ANN trained using the standard steepness coefficient. Based on this study, models with a steepness coefficient between 0.17 and 0.025 performed better than those with a steepness coefficient between 1 and 0.35. These results could also be applicable to other studies; therefore, it is suggested that additional studies employing steepness coefficients between 0.17 to 0.025 be used to evaluate the ANN data training and forecasting performance. The results of the present study also show that the optimal steepness coefficient method is more efficient for producing the best training performance than trial and error. Importantly, the optimal steepness coefficient significantly improved the data training of poor performance training datasets. The improved data training can help improve the accuracy of data forecasting at Rantau Panjang station and thus assist in monitoring for the possible future occurrence of flood. For future research in applying linear transfer function at output neurons and MSC in hidden neurons, it is highly recommended to find better and more flexible optimization technique that can handle problem of big difference of weights adjustment in hidden and output neurons due to the drawback of back propagation algorithm. Acknowledgements The authors thank the Environmental Research Group at the Department of Civil Engineering, Faculty of Engineering, University Kebangsaan Malaysia for a research grant (UKM-GUP-PLW-08-13-308) provided to the second and third authors. In addition, the authors appreciate the Department of Irrigation of Selangor and Johor for providing data and assisting with the background study of the Johor River. References Agarwal A, Singh RD (2004) Runoff modelling through back propagation artificial neural network with variable rainfall-runoff data. Water Resour Manage 18:285–300 Ahmed JA, Sarma AK (2007) Artificial neural network model for synthetic stream flow generation. Water Resour Manage 21(6):1015–1029 Alvisi S, Mascellani G, Franchini M, Bardossy A (2006) Water level forecasting through fuzzy logic and neural network approaches. Hydrol Earth Syst Sci 10(1):1–17 ASCE Task Committee on the application of ANN in Hydrology (2000a) Artificial neural networks in hydrology. I: preliminary concepts. J Hydrol Eng 5(2):115–123 ASCE Task Committee on the application of ANN in Hydrology (2000b) Artificial neural networks in hydrology. II: hydrological applications. J Hydrol Eng 5(2):124–137 Atiya AF, El-Shoura SM, Shaheen SI, El-Sherif MS (1999) A comparison between neural network technique—case study: river flow forecasting. IEEE Trans Neural Netw 10(2):402–409 Barron AR (1994) Approximation and estimation bounds for artificial neural networks. Mach Learn 14:115–133 Bustami RA, Bessaih N, Bong C, Suhaili S (2007) Artificial neural network for precipitation and water level predictions of Bedup River. IAENG International Journal of Computer Science 34(2):228–233 Cancelliere A, Gıuliano G, Ancarani A, Rossi G (2002) A neural networks approach for deriving irrigation reservoir operating rules. Water Resour Manage 16:71–88 Chandramouli V, Deka P (2005) Neural network based decision support model for optimal reservoir operation. Water Resour Manage 19:447–464 Chauhan S, Shrivastava RK (2008) Performance evaluation of reference evapotranspiration estimation using climate based methods and artificial neural networks. Water Resour Manage 23:825– 837 Improved Water Level Forecasting Performance 2541 Chiang YM, Chang LC, Chang FJ (2004) Comparison of static-feedforward and dynamic-feedback neural networks for rainfall–runoff modeling. J Hydrol 290:297–311 Cigizoglu HK, Kisi O (2006) Methods to improve the neural network performance in suspended sediment estimation. J Hydrol 317:221–238 Coulibaly P, Anctil F, Bobee B (2000) Daily reservoir inflow forecasting using artificial neural networks with stopped Training Approach. J Hydrol 230:244–257 El-Shafie A, Reda Taha M, Noureldin A (2007) A neuro-fuzzy model for inflow forecasting of the Nile River at Aswan High Dam. Water Resour Manage 21(3):533–556 El-Shafie A, Noureldin AE, Taha MR, Basri H (2008) Neural network model for Nile river inflow forecasting based on correlation analysis of historical inflow data. J Appl Sci 8(24):4487–4499 El-Shafie A, Abdin AE, Noureldin A, Taha MR (2009) Enhancing inflow forecasting model at Aswan high dam utilizing radial basis neural network and upstream monitoring stations measurements. Water Resour Manage 23(11):2289–2315 Fernando TMKG, Maier HR, Dandy GC, May RJ (2005) Efficient selection of inputs for artificial neural network models, Proc. of MODSIM 2005 International Congress on Modelling and Simulation: Modelling and Simulation Society of Australia and New Zealand, December 2005/Andre Zerger and Robert M. Argent (eds) 1806–1812 Han J, Moraga C, Sinne S (1996) Optimization of feedforward neural networks. Eng Appl Artif Intell 9(2):109–119 Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2:359–366 Joorabchi A, Zhang H, Blumenstein M (2007) Application of Artificial Neural Networks in Flow Discharge Prediction for Fitzroy River, Australia. J Coast Res SI 50:287–291 Kerh T, Lee CS (2006) Neural networks forecasting of flood discharge at an unmeasured station using river upstream information. Adv Eng Softw 37:533–543 Leahy P, Kiely G, Gearóid C (2008) Structural optimisation and input selection of an artificial neural network for river level prediction. J Hydrol 355:192–201 Rahnama MB, Barani GA (2005) Application of rainfall-runoff models to Zard River catchment’s. Am J Environ Sci 1(1):86–89 Rai RK, Mathur BS (2008) Event-based sediment yield modeling using artificial neural network. Water Resour Manage 22(4):423–441 Shamseldin AY, Nasr AE, O’Connor KM (2002) Comparison of different forms of the multi-layer feed-forward neural network method used for river flow forecasting. Hydrol Earth Syst Sci 6(4):671–684 Solaimani K, Darvari Z (2008) Suitability of artificial neural network in daily flow forecasting. J Appl Sci 8(17):2949–2957 Tareghian R, Kashefipour SM (2007) Application of fuzzy systems and artificial neural networks for flood forecasting. J Appl Sci 7(22):3451–3459 Toth E, Brath A (2007) Multistep ahead streamflow forecasting: role of calibration data in conceptual and neural network modeling. Water Resour Res 43:W11405. doi:10.1029/2006WR005383 Turan ME, Yurdusev MA (2009) River flow estimation from upstream flow records by artificial intelligence methods. J Hydrol 369:71–77 Wu CL, Chau KW, Li YS (2009) Methods to improve neural network performance in daily flows prediction. J Hydrol 372:80–93 Zealand CM, Burn DH, Simonovic SP (1999) Short term stream flow forecasting using artificial neural networks. J Hydrol 214:32–48 Zhang G, Patuwo BE, Hu MY (1998) Forecasting with artificial neural networks: the state of the art. Int J Forecast 14:35–62 View publication stats