Model Evaluation
The runoff at Tillegara was modelled using rainfall and evaporation data, recorded from the catchment.
It was proposed that the Australian Water Balance Model (AWBM) was an appropriate method to model
this runoff. The AWBM is a conceptual model which simulates the rainfall-runoff for a catchment using 8
parameters; K, A1, A2, A3, S1, S2, S3, BFI (Sharma, 2011). To evaluate whether the AWBM is an
appropriate for modelling the runoff at Tillegara, two and a half years of correct runoff data (1096
points), along with rainfall-evaporation data for this time period was analysed.
AWBM Verification
The comparison of AWBM output data and the correct data can be seen below in figure 1;
1000
AWBM Verification
Logarithmic Runoff (mm)
100
Real
Runoff
(mm)
10
1
0.1
Model
Runoff
(mm)
0.01
0.001
0.0001
0.00001
1
101 201 301 401 501 601 701 801 901 1001
Time (days)
Figure 1: Logarithmic Plot of AWBM Vs Correct Runoff
It can be seen in Figure 1 that the AWBM models the general trends of the real runoff data. The AWBM
plot is more linear in nature than the real data and acts similar to a ‘line of best fit’ for the real runoff
values. Between approximately days 150-400 and 850-1096, the AWBM models the runoff very well and
from this graph appears to be an appropriate method of modelling. In Appendix { }, figure {}, a graph of
the difference (Real-Model) has been plotted, and apart from the outliers, the majority of the data
points plot near to zero, which indicates the model represents the real runoff data well.
A boxplot of the data was also
constructed to provide graphical
comparison. In figure 2, the similarities
between shape of the data can be seen
however the AWBM model appears to
‘dampen’ the large runoff events. While
the real runoff boxplot has three values
above 100mm, the AWBM’s maximum
value is approximately 92mm. While the
AWBM appears to be an appropriate
method when modelling for small rainfall
events (0-5mm), it appears to induce a
‘dampening’ error when attempting to
simulate large rainfall events (>30mm).
Figure 2: Boxplot of Real Vs Model
This dampening effect will induce under-estimation errors in reservoir inflow calculations
Statistical analysis conclusively proves that the AWBM is an appropriate representation of the real
runoff for Tillegara. Refer to Appendix….for full analysis. Using a hypothesis test with a null hypothesis,
Ho: mean model=mean real, and a level of significance (α) of 0.01, a P-value of 0.9677 results. This gives
us significant evidence to accept the null hypothesis that both the means are equal and verify the
AWBM as an appropriate model for Tillegara.
All rainfall-runoff models have some related errors and the AWBM is no exception to this rule. However,
It can be concluded from these different methods of analysis, that whilst some errors are induced
especially in modelling large rainfall events, the AWBM is an appropriate model for Tillegara runoff and
is appropriate for use in storage calculations.
Historical Data
30 years of daily rainfall and evaporation data was provided, however small sections of the evaporation
data was missing and had to be approximated. This data was approximated using three different
methods to attempt to provide the most accurate representation and are described below;



If the missing data point was an isolated occasion, then the average of the previous day and
following day was taken.
If 4-10 data points were missing in one month, then the average evaporation for the month was
calculated using the available data and the missing data was assigned the average.
If a whole month was missing, which was the case in 1990, and from November 1994 to
December 1995, then the average for the corresponding month, two years previous and two
years following was calculated and then the whole month was assigned this average. For
instance, if November 1990 was missing then data for November from 1988 to 1992 was
gathered, and averaged. This was then assigned to the missing 1990 month.
Explanation of missing historical data approximation method (can go into appendix)
Three methods of approximation were used to attempt to provide the most accurate representation of
real data. When an isolated point was found, it was decided that the most accurate way to approximate
this would be to take the average of the surrounding day. This was done because weather patterns are
variable, thus the missing value would be much more influenced by the surrounding days’ weather
compared with the monthly average. When a whole month was missing, one option was the collect
averages for each month using the 30 years of data, however with such climatic events as El nino and La
Nina, sampling just 5 years of data surrounding the missing month was deeming more accurate as it
better accounted for climatic ‘seasons’ and fluctuations.
Reference
Sharma, A , 2011– CVEN3031-3731 Notes Weeks 1-6, Australian Water Balance Model Pg1
Appendix
A1)
Difference (mm) [Real-Model]
100
80
Difference (mm)
60
40
20
0
Difference
(mm) [RealModel]
-20
-40
-60
-80
-100
1
101 201 301 401 501 601 701 801 901 1001
Time (days)
Figure …, Plot of the difference (real minus model) in the output data.
A2)
Statistical Analysis
To analyse the 2 data sets and determine whether the AWBM model is a valid representation of the real
runoff values, a Hypothesis test has been carried out. This statistical analysis will provide a conclusive
way of proving an assumption (in this case that the mean of the real runoff is equal to the mean of the
AWBM model), is true or false.
Assumptions
-Real runoff has a mean of μ and a standard deviation of σ
-Model runoff has a mean of ‘mean-x’ and a standard deviation of ‘std-x’
-The number of data points ‘n’ for the 2.5 years of data is 1096. This means that the central limit
theorem applies and a z-test can be carried out on the data.
- α = Level of significance = 0.01
Using matlab and the following script, the hypothesis test was carried out.
Qhat= model data points, Q= Real runoff data points
[h, p, ci ] = ztest(qhat,mean(Q),std(Q),0.01)
Given that ‘mean-x’= 1.438 and using a null hypothesis Ho: μ=mean-x, the statistical analysis gives a Pvalue of 0.9677 and a Confidence Interval ranging from [0.8556, 2.0203]. As the P-value is greater than
the assumed α value of 0.01, we can accept the null hypothesis and conclude that the AWBM is an
appropriate representation of the real runoff data