CHAPTER 01: FUNDAMENTALS OF TIME SERIES ANALYSIS
Forecasting: Forecasting is the process of predicting future values or events based on the analysis
of past and present data or event.
Steps in forecasting:
1. Defining the purpose or objective of forecasting,
2. Selecting the required variables which are going to be forecasted,
3. Determining the periods of time for forecasting the series,
4. Collecting and preparing or managing data to make fit for analysis,
5. Selecting appropriate forecasting model,
6. Developing the forecasting model that best explains the data and fitting the developed model,
6. Generating forecast for future time periods.
Methods of forecasting:
1. Time series analysis: Autoregressive Integrated Moving Average (ARIMA), Seasonal ARIMA,
Autoregressive
Conditional
Heteroscedasticity
(ARCH),
Generalized
Autoregressive
Conditional Heteroscedasticity (GARCH) model.
2. Econometrics: Vector Autoregressive (VAR) model, Generalized Method of Moments (GMM),
and Generalized Estimating Equation (GEE).
3. Regression analysis: Linear Mixed Effect Model (LMM), Two-stage Least Square (2SLS),
Instrumental Variable (IV) Regression model.
4. Machine learning techniques: Decision tree, Random Forest, Artificial Neural Network (ANN).
5. Input-output analysis.
6. Numerical analysis using extrapolation technique.
Time series data: Time series data refers to the ordered sequence of observations collected and
recorded at regular intervals of time such as hourly, daily, weekly, monthly, quarterly, annually.
Example of time series data in business field:
1. Monthly revenue and sales figures of a company for the last 60 months,
2. Daily closing stock prices for the last 365 days,
3. Yearly return on investment (ROI) of bank for the last 20 years.
4. Net profit margin of a company for the last 25 years.
Time series analysis: Time series analysis comprises methods for analyzing time series data to
extract meaningful statistics and other characteristics of time series data.
Uses of time series analysis in the business field:
1. Revenue and sales,
2. Production planning,
3. Budget analysis,
4. Resource allocation,
5. Financial market analysis,
6. Risk management,
7. Inventory management,
8. Cash flow,
9. Performance of firms.
Time series forecasting: Time series forecasting refers to the use of suitable time series model to
predict future values or events based on previously observed values in the series.
Objective of time series analysis: The objectives of time series analysis are:
1. to understand the pattern and trend of the historical data,
2. to forecast the future values of the historical series.
Components of time series:
1. Secular trends,
2. Seasonal movements,
3. Cyclical movements,
4. Irregular or random fluctuations.
1. Secular trends: The secular trend is the main component of a time series which results from
long term effects of the series. A trend can be increasing or decreasing depending on whether the
time series exhibits an increasing long-term pattern or a decreasing long-term pattern. Secular
trend allows us to understand the pattern or trend of the original series as well as permits to project
the pattern or trend to the future.
Example: (i) Gross domestic product, production of a firm, and money supply may show
increasing trend; (ii) Population growth, death rate may show decreasing trend.
8
8
7
7
6
6
3
Series (Yt)
9
Series (Yt)
9
2
2
1
1
5
5
4
4
3
0
0
1
2
3
4
5
6
Time (t)
7
8
9
1
Figure 1(a): Linear increasing trend
2
3
4
5
6
Time (t)
7
8
9
Figure 1(b): Linear decreasing trend
16
14
14
12
10
10
8
8
12
6
Series (Yt)
18
16
Series (Yt)
18
4
4
2
2
6
0
0
1
2
3
4
5
6
Time (t)
7
8
9
Figure 1(c): Nonlinear increasing trend
1
2
3
4
5
6
Time (t)
7
8
9
Figure 1(d): Nonlinear increasing trend
2. Seasonal components: Seasonal components are the short-term movement occurring the same
fluctuations in each repetitive calendar periods. In order to detect the seasonal component, the
series must be recorded in a small unit of time (seasons) viz., hours, days, week, months. Seasonal
variation arises as a result of natural changes, habits, customs, or festivals in the seasons (small
unit of time) during the year (regular time interval).
Example: (i) Sales of cloths is the highest in the month of Eid; (ii) Consumption of ice-cream is
higher during summer and lower in winter.
Figure 2. Seasonal movements
3. Cyclical movements: Cyclical movements refers to the long-term oscillation (wave like pattern)
that are apparent over a number of time periods. The period of time having wave like pattern is
called cycle and each cycle may longer from 5 to 10 or more time periods. The cyclical movements
are more frequent in the business cycle and contains four well-defined phases, namely (i)
prosperity, (ii) Decline, (iii) depression, and (iv) improvement.
Example: (i) Economic recession and inflation, (ii) Long-term product demand cycle.
Figure: Cyclical movement of a time series from 1980-2022 having four business cycles.
4. Irregular fluctuation: Irregular fluctuations refers to the sudden change (variation) in the time
series which does not repeat in definite pattern. This may happen due to the catastrophic events
such as floods, fires, earthquakes, revolutions, war, epidemics, strikes etc. and this variation cannot
be explained by trend, seasonal or cyclic movements.
Figure: Irregular fluctuation in the time series.
Before forecasting a time series, the actual behavior of time series should be identified using
suitable methods. In other words, the measurement of trend component, seasonal variation,
cyclical component may be essential for ensuring appropriate time series forecasting model. Time
series plot is one of the most suitable tools to provide a general indication of trend, seasonal
and other components. Time series plot refers to a two-dimensional curve of original series
against time points.
Example of time series plot showing different patterns and components:
Source: Rob J. Hyndman & George Athanasopoulos. (2018). Forecasting Principles and Practice.
i)
The monthly housing sales (top left) show strong seasonality within each year, as well as some
strong cyclic behavior with a period of about 6–10 years. There is no apparent trend in the
data over this period.
ii) The US treasury bill contracts (top right) show results from the Chicago market for 100
consecutive trading days in 1981. Here there is no seasonality, but an obvious downward
trend. Possibly, if we had a much longer series, we would see that this downward trend is
actually part of a long cycle, but when viewed over only 100 days it appears to be a trend.
iii) The Australian monthly electricity production (bottom left) shows a strong increasing trend,
with strong seasonality. There is no evidence of any cyclic behavior here.
iv) The daily change in the Google closing stock price (bottom right) has no trend, seasonality or
cyclic behavior. There are random fluctuations which do not appear to be very predictable,
and no strong patterns that would help with developing a forecasting model.
However, the following are some methods for measuring the components of time series:
1. a) The measurement of linear trend:
i) Free hand smooth curve
ii) Moving average method
iii) Semi average method
iv) Least square method.
1. b) The measurement of nonlinear trend
i) Free hand smooth curve
ii) Moving average method
iii) Polynomial equation
iv) Exponential curve
v) Growth curve fitted by Modified exponential, Gompertz, and Logistic model.
2. Measurement of seasonal variation:
i) Method of simple average
ii) Ratio-to-trend method
iii) Ratio-to-moving average method
iv) Link relative method.
3. Measurement of cyclical variation:
i) Residual method
ii) Direct method.
Now we are going to estimate the trend component of time series by the moving average and semiaverage method.
Estimating trend component using moving average method: A m-order moving average of a
time series data refers to the arithmetic mean of that series within m-time points. Mathematically,
1
𝑦̂𝑡 = 𝑚 ∑𝑘𝑗=−𝑘 𝑦𝑡+𝑗
(01)
Where m = 2k + 1 and t = k + 1. For 3-point moving average (3-MA), the value m is 3 and
consequently the value of k is 1 and t is 2. Using these values, the mathematical equation will be:
1
1
𝑦̂2 = 3 ∑1𝑗=−1 𝑦𝑡+𝑗 = 3 (𝑦2−1 + 𝑦2+0 + 𝑦2+1 ) =
𝑦1 +𝑦2 +𝑦3
(02)
3
Mathematical problem: The following table shows the volume of electricity sold to residential
customers in South Australia each year from 1989 to 2008. Estimate the trend component of the
annual electricity sales to residential customers using 3-year moving average method. Construct a
trend line comparing with annual electricity sales to residential customers.
Year
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007 2008
Sales (GWh)
2569
2575
2763
2845
3001
3108
3358
3076
3181
3222
3176
3431
3527
3638 3655
Solution: The first trend value is estimated for the year 1995 using the average of annual electricity
sales of the first three years (1994-1996); the second trend value is estimated by the average of the
electricity sales for 1995-1997; and so on. Therefore, there are no values for either first or the last
year. Therefore, there is no trend value for the first year as well as the last year. The trend values
of yearly electricity sales are presented in the following table:
Year
1994
1995
1996
1997
1998
Volume of electricity sales
2569
2575
2763
2845
3001
Trend value using 3-MA
2635.67
2727.67
2869.67
2984.67
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
3108
3358
3076
3181
3222
3176
3431
3527
3638
3655
3155.67
3180.67
3205.00
3159.67
3193.00
3276.33
3378.00
3532.00
3606.67
The annual electricity sales volumes and their associated trend values are plotted separately against
time and produces the following lines:
Volume of electricity sales
4000
3500
3000
Volume of electricity sales
Trend value using 3-MA
2500
2000
1993
1995
1997
1999
2001
Year
2003
2005
2007
2009
Estimating trend component using semi-average method: Semi-average refers to the average
of two equally divided series from an observed time series. If the series have odd number of years,
then the middle year is ignored and two equal parts are formed. The trend values are estimated
using the gradual increase or decrease of these two averages. In graphical presentation, the trend
line is obtained by connecting these averages to a straight line.
Mathematical problem: The following table shows the volume of electricity sold to residential
customers in South Australia each year from 1989 to 2008. Estimate the trend component of the
annual electricity sales to residential customers using semi-average method. Construct a trend line
comparing with annual electricity sales to residential customers.
Year
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007 2008
Sales (GWh)
2569
2575
2763
2845
3001
3108
3358
3076
3181
3222
3176
3431
3527
3638 3655
Solution: Since there are 15 years, we will compute the average of sales for first seven years (19942000) and for last seven years (2002-2008). The averages represent the trend value of electricity
sales for the year 1997 and 2005 respectively. The difference between these averages produces the
increment of sales volume for seven years. In order to get the gradual increase (increment for each
year), we will divide the difference by 7.
Here, the trend value for 1997 =
the trend value for 2005 =
2569 + 2575 + 2763 + 2845 + 3001 + 3108 +3358
7
3181 + 3222 + 3176 + 3431 + 3527 + 3638 +3655
7
The increment of trend for each year =
Year
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
3404.29 − 2888.43
Volume of electricity sales
2569
2575
2763
2845
3001
3108
3358
3076
3181
3222
3176
3431
3527
3638
3655
7
= 2888.43
= 3404.29
= 73.69
Semi-average
2888.43
3404.29
Trend value using semi-average
(2741.04 – 73.69) = 2667.35
(2814.73 – 73.69) = 2741.04
(2888.43 – 73.69) = 2814.73
= 2888.43
(2888.43 + 73.69) = 2962.12
(2962.12 + 73.69) = 3035.82
(3035.82 + 73.69) = 3109.51
(3256.90 – 73.69) = 3183.20
(3330.59 – 73.69) = 3256.90
(3404.29 – 73.69) = 3330.59
= 3404.29
(3404.29 + 73.69) = 3477.98
(3477.98 + 73.69) = 3551.67
(3551.67 + 73.69) = 3625.37
Volume of electricity sales
4000
3500
3000
Volume of electricity sales
2500
2000
1993
Trend line using Semi-average
1995
1997
1999
2001
Year
2003
2005
2007
2009
Time series forecasting techniques
Forecasting when there is no seasonality: Based on the indication of trend, the following
forecasting techniques are widely used to forecast a time series data when there is no seasonality:
i) Exponential smoothing: The are three different exponential smoothing methods available for
forecasting a time series based on the variation in the trend component. These are:
a) Simple exponential smoothing (SES): This method is suitable for forecasting data with no
clear trend or seasonal pattern. In this method, forecasts are calculated using weighted
averages, where the weights decrease exponentially as observations come from further in
the past i.e., the smallest weights are associated with the oldest observations.
b) Holt’s linear trend method: Holt (1957) extended simple exponential smoothing to allow
the forecasting of data with a trend. The forecasts generated by this method display a
constant trend (increasing or decreasing) indefinitely into the future.
c) Damped trend method: Empirical evidence indicates that the series with constant trend tend
to over-forecast (specially for longer time periods). Gardner & McKenzie (1985) introduce
a parameter that “dampens” the trend to a flat line in some time to the future. This method
has proven to be very effective for forecasting a series having fluctuation in trend.
ii) Autoregressive integrated moving average (ARIMA (p,d,q)) model: Exponential smoothing
and ARIMA models are the two most widely used approaches to time series forecasting, and
provide complementary approaches to the problem. While exponential smoothing models are
based on a description of the trend and seasonality in the data, ARIMA (p,d,q) models aim to
describe the autocorrelations in the data.
In an autoregressive (AR) model of order p, we forecast the variable of interest using a model
against the past values (p-lagged values) of that variable. In moving average (MA) model of order
q, we forecast the variable of interest using a model against the past values of forecast error (qordered forecasting errors). It should be noted that, a moving average model is used for forecasting
the future values, while moving average smoothing is used for estimating the trend-cycle of past
values.
Forecasting when there is seasonality: The following forecasting techniques are widely used to
forecast a time series data when there is seasonality:
i) Holt Winter seasonal method: Holt (1957) and Winters (1960) extended Holt’s method to
capture seasonality. The additive Holt Winter method is preferred when the seasonal variations
are roughly constant through the series, while the multiplicative method is preferred when the
seasonal variations are changing proportional to the level of the series.
ii) Holt Winter Damped method: Damping is possible with both additive and multiplicative HoltWinters’ methods. A method that often provides accurate and robust forecasts for seasonal data
is the Holt-Winters method with a damped trend and multiplicative seasonality.
iii) Seasonal Autoregressive Integrated Moving Average (SARIMA) model: ARIMA models
are also capable of modelling a wide range of seasonal data. A seasonal ARIMA model is
formed by including additional seasonal terms in the ARIMA models.
CHAPTER 02: TEST OF HYPOTHSIS
Statistical Hypothesis: Any statement, description, or specification about population parameter
or population is called hypothesis.
Two types of hypotheses are used to test the statistical significance of population parameters:
1. Null hypothesis: Null hypothesis refers to the statement about no difference between parameter
and statistic. According to R. A. Fisher, null hypothesis is tested for possible rejection under the
assumption that it is true. Null hypothesis is denoted by H0.
2. Alternative hypothesis: The alternative hypothesis is a logical opposite statement of null
hypothesis. Alternative hypothesis is denoted by HA or H1. If the null hypothesis is rejected, then
the statement of parameter for the alternative hypothesis is considered true. The statement of the
parameter in alternative hypothesis indicates the type of test i.e., one-tailed or two-tailed test.
Note: The decision in a test of hypothesis is drawn based on the null hypothesis and type of test
(one-tailed or two-tailed). Specifically, the decision on the statement of parameter is always
carried out in terms of null hypothesis. If the null hypothesis is not rejected, then the statement of
parameter stated in null hypothesis is considered true. On the other hand, if the null hypothesis is
rejected, then the statement of parameter stated in alternative hypothesis is considered true.
Type-I error: Type-I error refers to the error of rejecting null hypothesis when it is true. In short,
type-I error occurs when null hypothesis is wrongly rejected.
Type-II error: Type-II error refers to the error of not rejecting null hypothesis when it is false. In
short, type-II error occurs when the null hypothesis is not rejected wrongly.
Note: Type-I error is more serious than type-II errors. Although the probability of committing
these errors is never been 0 but it is desired to keep the errors at minimum level. In empirical test
of hypothesis, the probability of committing type-I error is kept fixed at the initial stage of testing
process.
Level of significance: The probability of rejecting null hypothesis when it is true, is called the
level of significance. In short, the level of significance refers to the probability of committing a
type-I error. It is also known as “size of test” and denoted by α. In addition, the level of confidence
is considered at 1 – α. Therefore, the empirical study tests the hypothesis at 100×α% level of
significance or 100×(1– α)% level of confidence.
Power of a test: The probability of rejecting null hypothesis when it is false, is called the power
of a test. In short, the power of a test refers to probability of rightly rejecting null hypothesis. It is
denoted by 1 – β.
Test statistic: Test statistic refers to the function of sample observation which is used to provide
the evidence about the rejection or not rejection of null hypothesis. The calculated or observed
value of test statistic is computed using sample estimates from the empirical datasets.
One-tailed test: One-tailed test indicates that the statement of parameters in alternative hypothesis
is stated in one direction (less than or greater than) only. Specifically, the value of parameters in
alternative hypothesis is stated either less than the testable value (left-tailed) or greater than the
testable value (right-tailed). The rejection (critical) region in this test is distributed in either left
tail or in the right tail of the probability distribution of test statistic.
Two-tailed test: Two-tailed test indicates that the statement of parameters in alternative
hypothesis is stated in both direction (not equal to). Specifically, the value of parameters in
alternative hypothesis is stated not equal to the testable value which implies that the parameters
can be both less than and greater than to the testable value. The rejection (critical) region in this
test is divided equally in the left and right tails of the probability distribution of test statistic.
Critical (Rejection) region and Acceptance region: Critical region refers to the set of possible
values of test statistic which can contradict the null hypothesis and can support the alternative
hypothesis. On the other hand, the set of values of test statistic that support the null hypothesis is
termed as acceptance region.
Note: The critical region may lead to invalid conclusion regarding the parameter of concern.
Specifically, critical region refers to the values test statistic for which we can rejected the null
hypothesis when it is true. For this region, the critical region is identified using the smaller and
fixed level of significance.
Left-tailed critical value (-Zα) and critical region (α) Right-tailed critical value (+Zα) and critical region (α)
Two-tailed critical values and critical region (α)
Tabulated (critical) value of test statistic: The value of test statistic that separates the acceptance
and rejection region is called the tabulated value of test statistic. The tabulated or critical value of
test statistic is computed from the distribution function of sampling distribution (can be extracted
from the published statistical tables or can be computed using statistical software). The decision
regarding the null hypothesis is taken comparing the calculated (observed) and the tabulated
(theoretical) value of test statistic.
p-value: p-value indicates the probability of having enough evidence to reject or not to reject the
null hypothesis. In short, p-value refers to the smallest value of level of significance at which the
null hypothesis can be rejected. In other words, p-value is the evidence against a null hypothesis.
The smaller the p-value, the stronger the evidence that we should reject the null hypothesis.
The conclusions about the hypothesis test are drawn when the p-value of a test is compared against
the level of significance, which plays the role of a benchmark. The most typical levels of
significance are 0.10, 0.05, and 0.01. The level of significance of 0.05 is considered conventional
and the most commonly used.
In empirical test of hypothesis:
(i) If p ≤ 0.01, then we reject the null hypothesis at 1% level of significance.
(ii) If 0.01 < p ≤ 0.05, then we reject the null hypothesis at 5% level of significance.
(iii) If 0.05 < p ≤ 0.10, then we reject the null hypothesis at 10% level of significance.
(iv) If p > 0.10, then we do not reject the null hypothesis at 10% level of significance.
The following test of hypothesis are essential for making informed business decisions:
1. Test regarding the equality of proportion (Single proportion, double proportion)
2. Test regarding the equality of variances
3. Test regarding the equality of mean (Single mean, double mean (independent), paired mean,
several mean)
4. Test regarding the significance of correlation
5. Test of association or test of independence
6. Test regarding the regression coefficients
CHAPTER 03: TEST REGARDING PROPORTION
3.1 Single proportion test
The single proportion (one sample binomial) test is used to determine whether the population
proportion (P) of a response is equal to the hypothesized population proportion (P0).
Hypotheses: The null and alternative hypotheses are as follows:
H0 : P = P0 i.e., the proportion of a categorical response (e.g., male gender) is equal to P0
H1 : P ≠ P0 i.e., the proportion of a categorical response (e.g., male gender) is not equal to P0
Test statistic: 𝑍 =
𝑝−𝑃0
𝑃 (1−𝑃0 )
𝑛
√ 0
𝑟
Where, p = 𝑛 = Sample proportion (r number of occurrences out of n numbers)
P0 = Testable or hypothesized proportion
n = Total number of sample (sample size)
Critical region (CR): The two-sided critical region at 5% level of significance (α = 0.05) is:
{ Z ≤ ─ Zα/2 ,
Z ≥ Z1 - α/2 } or
| Z | > | Zα/2 |
{ Z ≤ ─ Z0.025 , Z ≥ Z0.975 } or
| Z | > | Z0.025 |
{ Z ≤ ─ 1.96 , Z ≥ 1.96 }
| Z | > | 1.96 |
or
Decision: If the calculated value of | Zcal | is greater than the tabulated value of | Ztab | (1.96), then
we reject the null hypothesis at 5% level of significance, otherwise we do not reject the null
hypothesis.
Conclusion: If we reject the null hypothesis, then we conclude that the proportion of a categorical
response is not equal to P0.
Note: The 95% confidence interval (CI) (α = 0.05) is:
𝑃 (1−𝑃 )
𝑃 (1−𝑃0 )
𝑛
= 𝑝 ± 𝑍0.025 × √ 0 𝑛 0
𝑝 ± 𝑍𝛼 × √ 0
2
𝑃 (1−𝑃 )
Lower bound, LB = 𝑝 − 1.96 × √ 0 𝑛 0
and
𝑃 (1−𝑃 )
= 𝑝 ± 1.96 × √ 0 𝑛 0
𝑃 (1−𝑃 )
Upper bound, UB = 𝑝 + 1.96 × √ 0 𝑛 0
Therefore, the 95% CI is: { LB , UB }
For a practical example: Browse the web page
3.2 Double proportion test
The double proportion test is used to determine whether the population proportion of two
independent responses are equal or not. In other words, double proportion test is used to determine
whether two proportions are coming from the same population.
Hypotheses: The null and alternative hypotheses are as follows:
H0 : P1 = P2 that means, P1 – P2 = 0, i.e., there is no significant difference between the P1 and P2
H1 : P1 ≠ P2 that means P1 – P2 ≠ 0, i.e., there is significant difference between the P1 and P2
𝑝1 −𝑝2
Test statistic: 𝑍 =
1
1
1
2
√𝑝(1−𝑝)×(𝑛 + 𝑛 )
Where,
p1 & p2 = Proportions obtained from 1st and 2nd sample responses.
n1 & n2 = Total number of samples in 1st and 2nd sample responses.
p=
𝒓𝟏 +𝒓𝟐
𝒏𝟏 +𝒏𝟐
= Overall sample proportion
Critical region (CR): The two-sided critical region at 5% level of significance (α = 0.05) is:
{ Z ≤ ─ Zα/2 ,
Z ≥ Z1 - α/2 } or
| Z | > | Zα/2 |
{ Z ≤ ─ Z0.025 , Z ≥ Z0.975 } or
| Z | > | Z0.025 |
{ Z ≤ ─ 1.96 , Z ≥ 1.96 }
| Z | > | 1.96 |
or
Decision: If the calculated value of | Zcal | is greater than the tabulated value of | Ztab | (1.96), then
we reject the null hypothesis at 5% level of significance, otherwise we do not reject the null
hypothesis.
Conclusion: If we reject the null hypothesis, then we conclude that there is significant difference
in the proportion of responses in the 1st and 2nd population.
Note: The 95% confidence interval (CI) (α = 0.05) is:
(𝑝1 − 𝑝2 ) ± 𝑍𝛼 × √𝑝(1 − 𝑝) × (
2
1
1
+ )
𝑛1 𝑛2
= (𝑝1 − 𝑝2 ) ± 𝑍0.025 × √𝑝(1 − 𝑝) × (
1
1
+ )
𝑛1 𝑛2
1
1
1
2
= (𝑝1 − 𝑝2 ) ± 1.96 × √𝑝(1 − 𝑝) × (𝑛 + 𝑛 )
1
1
1
1
1
2
1
2
LB = (𝑝1 − 𝑝2 ) − 1.96 × √𝑝(1 − 𝑝) × (𝑛 + 𝑛 ) & UB = (𝑝1 − 𝑝2 ) + 1.96 × √𝑝(1 − 𝑝) × (𝑛 + 𝑛 )
Therefore, the 95% CI is: { LB , UB }
For more practical example: Browse the web page -1
Browse the web page -2
CHAPTER 04: TEST REGARDING MEAN
4.1 Single mean test
Single mean test is used to determine whether the population mean (µ) of a variable is different
from the hypothesized/testable population mean (µ0).
Hypotheses: The null and alternative hypotheses are as follows:
H0 : µ = µ0 i.e., the mean of a continuous variable is not different from µ0
H1 : µ ≠ µ0 i.e., the mean of a continuous variable is significantly different from µ0
Case-1: When the population standard deviation (σ) is known
Test statistic: 𝑍 =
𝑥̅ −𝜇0
𝜎
√𝑛
Where, 𝑥̅ = Sample mean
µ0 = Testable or hypothesized mean
σ = Known population standard deviation
n = Total number of sample (sample size)
Critical region (CR): The two-sided critical region at 5% level of significance (α = 0.05) is:
{ Z ≤ ─ Zα/2 ,
Z ≥ Z1 - α/2 } or
| Z | > | Zα/2 |
{ Z ≤ ─ Z0.025 , Z ≥ Z0.975 } or
| Z | > | Z0.025 |
{ Z ≤ ─ 1.96 , Z ≥ 1.96 }
| Z | > | 1.96 |
or
Case-2: When the population standard deviation (σ) is unknown and estimated from sample
Test statistic: 𝑡 =
𝑥̅ −𝜇0
𝑆
√𝑛
Where, 𝑥̅ = Sample mean
µ0 = Testable or hypothesized mean
1
1
𝑆 2 = 𝑛−1 ∑(𝑥𝑖 − 𝑥̅ )2 = 𝑛−1 [∑ 𝑥𝑖2 − 𝑛𝑥̅ 2 ] = Estimated sample variance
n = Total number of sample (sample size)
Critical region (CR): The two-sided critical region at 5% level of significance (α = 0.05) is:
{ t ≤ ─ tα/2, n-1
,
t ≥ t1 - α/2, n-1 } or
| t | > | tα/2, n-1 |
{ t ≤ ─ t0.025, n-1
,
t ≥ t0.975, n-1 } or
| t | > | t0.025, n-1 |
Decision: If the calculated value of | Zcal | or | tcal |is greater than the tabulated value of | Ztab | (1.96)
or | ttab at n-1 df |, then the test value falls in the critical region and we reject the null hypothesis at 5%
level of significance, otherwise we do not reject the null hypothesis.
Conclusion: If we reject the null hypothesis, then we conclude that the mean of a continuous
variable is significantly different from µ0.
Construct a 100×(1-α)% confidence interval (CI)
Case-1: When the population standard deviation is known:
95% CI (α = 0.05)
𝝈
𝝈
̅ ± 𝒁𝜶 ×
=𝒙
𝒏
𝟐
𝝈
̅ ± 𝒁𝟎.𝟎𝟓 ×
=𝒙
𝒏
√
𝟐
𝝈
̅ − 𝟏. 𝟗𝟔 ×
Lower bound, LB = 𝒙
&
𝒏
√
̅ ± 𝟏. 𝟗𝟔 ×
=𝒙
𝒏
√
√
𝝈
̅ + 𝟏. 𝟗𝟔 ×
Upper bound, UB = 𝒙
𝒏
√
Case-2: When the population standard deviation is unknown and estimated from sample
95% CI (α = 0.05)
𝑺
𝑺
̅ ± 𝒕𝜶,𝒏−𝟏 ×
=𝒙
𝒏
𝟐
̅ ± 𝒕𝟎.𝟎𝟓,𝒏−𝟏 ×
=𝒙
𝒏
√
𝑺
̅ − 𝒕𝟎.𝟎𝟐𝟓,𝒏−𝟏 ×
Lower bound, LB = 𝒙
𝒏
√
𝟐
&
√
𝑺
̅ + 𝒕𝟎.𝟎𝟐𝟓,𝒏−𝟏 ×
Upper bound, UB = 𝒙
𝒏
√
Therefore, the 95% CI is: { LB , UB }
Mathematical problems (Single mean test)
Problem-1: Suppose Laraki, a car company based in Morocco, is launching a brand-new model.
Laraki claims that the new model is the most fuel-efficient vehicle with an estimated mean of 0.047
liters/km. The company uses a random sample of 20 cars that yielded a mean of 0.051 liters/km
and a standard deviation of 0.028 liters/km.
Required:
a. Does the actual gas mileage for this model deviate significantly from the Laraki’s claim? (Use
5% significance level).
b. Construct a 95% confidence interval for the mean mileage of the new model of Laraki’s car.
Problem-2: Armani claims that any range of their fragrances may last more than 8 hours on
average. Dr. Jamal, a market analyst, wants to verify the claim. He uses a random sample of 18
consumers of Armani’s fragrances, and the mean time that an Armani fragrance lasts when applied
is 15.25 hours with a standard deviation of 16.92 hours.
Required:
a. At the 0.05 level of significance, is there evidence that the population mean that Armani’s
fragrances last more than 8 hours?
b. Construct a 95% confidence interval for the population mean of Armani’s fragrance duration.
Problem-3: The following data represent the amount of soft drink filled in a sample of 50
consecutive 2-liter bottles. The results, listed horizontally in the order of being filled, were:
2.109 2.086 2.066 2.075 2.065 2.057 2.052 2.044 2.036 2.038 2.031 2.029 2.025
2.029 2.023 2.020 2.015 2.014 2.013 2.014 2.012 2.012 2.012 2.010 2.005 2.003
1.999 1.996 1.997 1.992 1.994 1.986 1.984 1.981 1.973 1.975 1.971 1.969 1.966
1.967 1.963 1.957 1.951 1.951 1.947 1.941 1.941 1.938 1.908 1.894
Required:
a. At the 0.05 level of significance, is there evidence that the mean amount of soft drink filled is
different from 2.0 liters?
b. Construct a 95% confidence interval for the mean amount of soft drinks filled in a 2-liter bottle.
4.2 Double mean test (Two independent samples)
The double mean test or independent sample mean test is used to determine whether the population
mean of two independent population are equal or not. In other words, double mean test is used to
determine whether the population means of two independent sample are significantly different or
not.
Hypotheses: The null and alternative hypotheses are as follows:
H0 : µ1 = µ2 that means, µ1 – µ2 = 0, i.e., there is no significant difference between µ1 and µ2
H1 : µ1 ≠ µ2 that means µ1 – µ2 ≠ 0, i.e., there is significant difference between µ1 and µ2
Case-1: When the population standard deviations (σ1 and σ2) are known
𝑥̅1 −𝑥̅2
Test statistic: 𝑍 =
𝜎2 𝜎2
𝑛1 𝑛2
√ 1+ 2
𝑥̅1 & 𝑥̅2 = Sample means obtained from 1st and 2nd sample.
Where,
𝜎12 & 𝜎12 = Known population variances
n1 & n2 = Sample sizes.
Critical region (CR): The two-sided critical region at 5% level of significance (α = 0.05) is:
{ Z ≤ ─ Zα/2 ,
Z ≥ Z1 - α/2 } or
| Z | > | Zα/2 |
{ Z ≤ ─ Z0.025 , Z ≥ Z0.975 } or
| Z | > | Z0.025 |
{ Z ≤ ─ 1.96 , Z ≥ 1.96 }
| Z | > | 1.96 |
or
Case-2: When the population standard deviations (σ1 and σ2) are unknown and estimated
from samples (S1 and S2) (assumed equal)
𝑥̅1 −𝑥̅2
Test statistic: 𝑡 =
1
1
+
𝑛1 𝑛2
𝑆𝑝 ×√
Where,
𝑥̅1 & 𝑥̅2 = Sample means obtained from 1st and 2nd sample.
n1 & n2 = Sample sizes.
𝑆𝑃 =
(𝑛1 −1)𝑆12 +(𝑛2 −1)𝑆22
𝑛1 +𝑛2 −2
= Pooled variance of 1st and 2nd sample.
1
1
1
1
2
𝑆12 = 𝑛 −1 ∑(𝑥1𝑖 − 𝑥̅1 )2 = 𝑛 −1 [∑ 𝑥1𝑖
− 𝑛𝑥̅1 2 ] = 1st sample variance
𝑆22 =
1
𝑛2 −1
∑(𝑥2𝑖 − 𝑥̅2 )2 =
1
𝑛2 −1
2
[∑ 𝑥2𝑖
− 𝑛𝑥̅2 2 ] = 2nd sample variance
Critical region (CR): The two-sided critical region at 5% level of significance (α = 0.05) is:
{ t ≤ ─ tα/2, n1+n2-2
,
t ≥ t1 - α/2, n1+n2-2 }
or
| t | > | tα/2, n1+n2-2 |
{ t ≤ ─ t0.025, n1+n2-2
,
t ≥ t0.975, n1+n2-2 }
or
| t | > | t0.025, n1+n2-2 |
Decision: If the calculated value of | Zcal | or | tcal | is greater than the tabulated value of | Ztab |
(1.96) or | ttab at n1+n2-2 df |, then the value of test statistic falls in the critical region and thus we reject
the null hypothesis at 5% level of significance, otherwise we do not reject the null hypothesis.
Conclusion: If we reject the null hypothesis, then we conclude that there is significant difference
between µ1 and µ2.
Construct a 100×(1-α)% confidence interval (CI)
Case-1: When the population standard deviations (σ1 and σ2) are known:
95% CI (α = 0.05)
𝝈𝟐
𝝈𝟐
𝟏
𝟐
𝝈𝟐
𝝈𝟐
𝟏
𝟐
𝝈𝟐
𝝈𝟐
𝟏
𝟐
̅𝟏 − 𝒙
̅𝟐 ) ± 𝒁𝜶 √ 𝟏 + 𝟐
= (𝒙
𝒏
𝒏
𝟐
̅𝟏 − 𝒙
̅𝟐 ) − 𝟏. 𝟗𝟔 × √ 𝟏 + 𝟐
Lower bound, LB = (𝒙
𝒏
𝒏
&
̅𝟏 − 𝒙
̅𝟐 ) + 𝟏. 𝟗𝟔 × √ 𝟏 + 𝟐
Upper bound, UB = (𝒙
𝒏
𝒏
Case-2: When the population standard deviations are unknown and estimated from samples
95% CI (α = 0.05)
𝟏
𝟏
𝟏
𝟐
𝟏
𝟏
𝟏
𝟐
𝟏
𝟏
𝟏
𝟐
̅𝟏 − 𝒙
̅𝟐 ) ± 𝒕𝜶, 𝒏𝟏 +𝒏𝟐 −𝟐 × 𝑺𝒑 √ +
= (𝒙
𝒏
𝒏
𝟐
̅𝟏 − 𝒙
̅𝟐 ) − 𝒕𝟎.𝟎𝟐𝟓, 𝒏𝟏 +𝒏𝟐 −𝟐 × 𝑺𝒑 √ +
Lower bound, LB = (𝒙
𝒏
𝒏
̅𝟏 − 𝒙
̅𝟐 ) + 𝒕𝟎.𝟎𝟐𝟓, 𝒏𝟏 +𝒏𝟐 −𝟐 × 𝑺𝒑 √ +
Upper bound, UB = (𝒙
𝒏
𝒏
&
Therefore, the 95% CI is: { LB , UB }
Mathematical problems (Double mean test)
Problem-1: A survey of 90 teenage boys found that, on average, they spent 3.6 hours per week
chatting with friends over the internet. The standard deviation was 1.2 hours. A similar survey of
50 teenage girls found an average of 3.9 hours, with standard deviation 2.1 hours.
Required:
a. Is there is any significant difference between boys’ and girls’ chatting behavior? (Use 5%
significance level).
b. Construct a 95% confidence interval for their average chatting behavior.
Problem-2: You and some friends have decided to test the validity of an advertisement by a local
pizza restaurant, which says it delivers to the dormitories faster than a local branch of a national
chain. Both the local pizza restaurant and national chain are located across the street from your
college campus. You define the variable of interest as the delivery time, in minutes, from the time
the pizza is ordered to when it is delivered. You collect the data by ordering 10 pizzas from the
local pizza restaurant and 10 pizzas from the national chain at different times. Following data
represents their delivery times.
Local
Chain
16.8
18.1
22.0 19.5
11.7
14.1
15.2 17.0
15.6
21.8
18.7 19.5
16.7
13.9
15.6 16.5
17.5
20.8
20.8 24.0
Required:
a. At the 0.05 level of significance, is there evidence that the mean delivery time for the local
pizza restaurant is less than the mean delivery time for the national pizza chain?
b. Construct a 95% confidence interval for the population mean difference of delivery time
between local pizza restaurant and national pizza chain.
4.3 Double mean test (Two paired samples)
The paired mean test or paired t-test is used to determine whether the population mean of two
related or dependent or paired population are equal or not. In other words, paired mean test is used
to determine whether the population means of two paired or dependent sample are significantly
different or not. In addition, the dependent or related or paired sample illustrates the before and
after case of the same subject. In the business sector, the sales volume before and after the
promotion of a product, marks or test score before and after training program, blood pressure
before and after taking drug etc. can be categorized as a paired sample. In most cases, this test
should be a one-tailed or right-tailed test to test whether a product's advertisement or the training
programs are significant.
Hypotheses: The null and two tailed alternative hypotheses are as follows:
H0 : µ1 = µ2 that means, µ1 – µ2 = 0, i.e., there is no significant difference between µ1 and µ2
H1 : µ1 ≠ µ2 that means µ1 – µ2 ≠ 0, i.e., there is significant difference between µ1 and µ2
Test statistic: 𝑡 =
𝑑̅
𝑠𝑑
√𝑛
1
Where, 𝑑̅ = 𝑛 ∑𝑛𝑖=1 𝑥𝑖 − 𝑦𝑖 = Sample means obtained by subtracting 2nd sample from 1st sample.
1
𝑠𝑑 = √𝑛−1 ∑𝑛𝑖=1(𝑑𝑖 − 𝑑̅) = Sample standard deviation obtained from the differenced sample.
Critical region (CR): The two-sided critical region at 5% level of significance (α = 0.05) is:
{ t ≤ ─ tα/2, n-2 , t ≥ t1 - α/2, n-2 }
{ t ≤ ─ t0.025, n-2 ,
or
| t | > | tα/2, n-2 |
t ≥ t0.975, n-2 } or
| t | > | t0.025, n-2 |
Decision: If the calculated value of | tcal | is greater than the tabulated value of | ttab | or | ttab at n-2 df
|, then the value of test statistic falls in the critical region and thus we reject the null hypothesis at
5% level of significance, otherwise we do not reject the null hypothesis.
Conclusion: If we reject the null hypothesis, then we conclude that there is significant difference
between µ1 and µ2.
Construct a 100×(1-α)% confidence interval (CI)
𝒔𝒅
̅ ± 𝒕𝜶
= 𝒅
,𝒏−𝟐
√𝒏
̅ − 𝒕𝟎.𝟎𝟐𝟓,𝒏−𝟐
Lower bound, LB = 𝒅
√𝒏
95% CI (α = 0.05)
𝟐
𝒔𝒅
̅ + 𝒕𝟎.𝟎𝟐𝟓,𝒏−𝟐 𝒔𝒅
Upper bound, UB = 𝒅
𝒏
√
Therefore, the 95% CI is: { LB , UB }
&
Mathematical problem (Paired t-test):
Question-01: Twelve cars were equipped with radial tires and driven over a test course. Then the
same 12 cars (with the same drivers) were equipped with regular belted tires and driven over the
same course. Following data refers to the cars’ gas economy (in km/l) after each run:
Gas economy
1
2
3
4
5
6
7
8
9
10
11
12
Radial tires
4.2
4.7
6.6
7.0
6.7
4.5
5.7
6.0
7.4
4.9
6.1
5.2
Belted tires
4.1
4.9
6.2
6.9
6.8
4.4
5.7
5.8
6.9
4.7
6.0
4.9
Required:
1. Is there evidence that radial tires produce better fuel economy?
2. Find a 95% CI for the mean difference in gas economy between radial tires and belted tires.
Question-02: Suppose we want to know whether or not a certain training program is able to
increase the max vertical jump (in inches) of college basketball players. To test this, we may recruit
a simple random sample of 20 college basketball players and measure each of their max vertical
jumps. Then, we may have each player use the training program for one month and then measure
their max vertical jump again at the end of the month.
Player
Player 1
Player 2
Player 3
Player 4
Player 5
Player 6
Player 7
Player 8
Player 9
Player 10
Player 11
Player 12
Player 13
Player 14
Player 15
Player 16
Player 17
Player 18
Player 19
Player 20
Max vertical jump
before training program
22
20
19
24
25
25
28
22
30
27
24
18
16
19
19
28
24
25
25
23
Max vertical jump
after training program
24
22
19
22
28
26
28
24
30
29
25
20
17
18
18
28
26
27
27
24
Required:
1. Is there evidence that a certain training program is able to increase the max vertical jump
(in inches) of college basketball players?
2. Find a 95% CI for the mean difference in vertical jump between before and after training
program.
Question-03: Artificial Intelligence (AI) training institute evaluating the effectiveness of training
by comparing AI knowledge of the students before and after the execution of live projects. At a
95% confidence level, does this information provide sufficient evidence to indicate that the live
projects will improve the students’ AI knowledge?
Question-04: A dietician is evaluating the individual weights before and after following the diet
plan. At a 95% confidence level, is there any significant evidence of the individuals’ weight
reduction?
4.4 Several mean test (One-way analysis of variance (ANOVA))
Although it is called analysis of variance, it actually tests whether the population mean of more
than two independent population are equal or not. In this test, there is a continuous dependent
variable and a categorical independent variable with more than two categories or groups.
Sometimes, the categorical variable is referred to the treatment and the categories as factors.
Example: Suppose we want to test whether number of years of work experience since graduation
has an effect on beginning salary for accounting majors. The three treatments (or groups) are:
Treatment 1: BBA degree with no work experience
Treatment 2: BBA degree with 1 year of work experience
Treatment 3: BBA degree with 2 years of work experience
In each treatment group, the starting salaries of four graduates were organized to determine
whether there was a significant difference in the average starting salaries of accounting graduates.
To reach this conclusion, we need to perform a one-way ANOVA test.
Using this technique, we can test the hypothesis that the means of more than two groups are equal
against at least one of them is different from the others. ANOVA calculations are conventionally
summarized in an analysis of variance table which is shown below.
Sources of
Sum of
variation
square
Treatment
SSTreatment
k-1
𝑀𝑆𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 =
Error
SSError
N-k
/ Between
Total
𝑀𝑆𝐸𝑟𝑟𝑜𝑟 =
SSTotal
N-1
DF
/ Within
Mean square
Fcal
𝑆𝑆𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡
𝑘−1
𝐹=
𝑀𝑆𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡
𝑀𝑆𝐸𝑟𝑟𝑜𝑟
𝑆𝑆𝐸𝑟𝑟𝑜𝑟
𝑁−𝑘
Calculation of the components of the ANOVA table:
SSTotal = SSTreatment + SSError
or,
𝒌
SSTotal = SSWithin + SSBetween
𝒏𝒊
𝑺𝑺𝑻𝒐𝒕𝒂𝒍 = ∑ ∑ 𝒙𝟐𝒊𝒋 −
𝒊=𝟏 𝒋=𝟏
𝑮𝟐
𝑵
Ftab
Fα, k-1, N-k
𝑛
𝑖
Where, 𝐺 = 𝐺𝑟𝑎𝑛𝑑 𝑡𝑜𝑡𝑎𝑙 = ∑𝑘𝑖=1 ∑𝑗=1
𝑥𝑖𝑗 and 𝑁 = ∑𝑘𝑖=1 𝑛𝑖
𝑻𝟐𝟏 𝑻𝟐𝟐
𝑻𝟐𝒌
𝑮𝟐
𝑺𝑺𝑻𝒓𝒆𝒂𝒕𝒎𝒆𝒏𝒕 = 𝑺𝑺𝑾𝒊𝒕𝒉𝒊𝒏 = ( +
+ ⋯+ ) −
𝒏 𝟏 𝒏𝟐
𝒏𝒌
𝑵
𝑛
𝑖
Where, 𝑇𝑖 = 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝑡𝑜𝑡𝑎𝑙 = ∑𝑗=1
𝑥.𝑗
Therefore,
𝑺𝑺𝑬𝒓𝒓𝒐𝒓 = 𝑺𝑺𝑻𝒐𝒕𝒂𝒍 − 𝑺𝑺𝑻𝒓𝒆𝒂𝒕𝒎𝒆𝒏𝒕
or
𝑺𝑺𝑩𝒆𝒕𝒘𝒆𝒆𝒏 = 𝑺𝑺𝑻𝒐𝒕𝒂𝒍 − 𝑺𝑺𝑾𝒊𝒕𝒉𝒊𝒏
Hypotheses: The null and one-tailed (always) alternative hypotheses are as follows:
H0 : µ1 = µ2 = . . . = µk i.e., there is no significant difference among µ1, µ2, . . ., µk
H1 : At least one of the group mean is different from the others.
Test statistic: 𝐹 =
𝑀𝑆𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡
𝑀𝑆𝐸𝑟𝑟𝑜𝑟
𝑀𝑆
= 𝑀𝑆 𝑊𝑖𝑡ℎ𝑖𝑛
𝐵𝑒𝑡𝑤𝑒𝑒𝑛
Critical region (CR): The one-sided critical region at 5% level of significance (α = 0.05) is:
{ F ≥ Fα, k-1, N-k }
or
| F | > | Fα, k-1, N-k |
or
{ F ≥ F0.05, k-1, N-k }
Decision: If the calculated value of | Fcal | is greater than the tabulated value of | Ftab |, then the
value of test statistic falls in the critical region and thus we reject the null hypothesis at 5% level
of significance, otherwise we do not reject the null hypothesis.
Conclusion: If we reject the null hypothesis, then we conclude that there is significant difference
among µ1, µ2, . . . , µk .
***NOTE***
If the null hypothesis is rejected, i.e., if at least one of the group means is different from the others,
then we should perform a Post-hoc test (pairwise comparison or multiple comparison) to identify
the significantly different pairs of group means. In short, the Post-hoc test is used to compare the
means for two groups similar to a two independent sample means test.
Mathematical problem (One-way ANOVA):
Question-01: Suppose the government of Bangladesh will increase gas tax for a new financial
year to maintain fuel imports from southern Indian countries. One business researcher claimed that
the increase would hit the working class hardest because they commute the furthest to work. Let's
assume that the researcher randomly surveyed 24 people and asked them about their daily oneway commuting mileage and organized the mileage data for statistical analysis to test their claim.
Working class
17.8
26.7
49.4
9.4
65.4
47.1
19.5
51.2
Professional (middle income)
16.5
17.4
22
7.4
9.4
2.1
6.4
13.9
Professional (Wealthy)
8.5
6.3
4.6
12.6
11
28.6
15.4
9.3
Required:
1. Construct a one-way analysis of variance (ANOVA) table using the above data.
2. Test whether there is significant difference in average one-way commuting mileage among
three class of people.
Question-02: A stock analyst thinks four stock mutual funds generate about the same return. She
collected the accompanying rate-of-return data on four different mutual funds during the last 5
years.
Years
Stock mutual fund type
A
B
C
D
1988
12
11
13
15
1989
12
17
19
11
1990
13
18
15
12
1991
18
20
25
11
1992
12
19
19
10
Required:
1. Construct a one-way analysis of variance (ANOVA) table using the above data.
2. Test whether there is significant difference in the average rate-of-return among four stock
mutual funds.
Question-03: A college professor taught an interdisciplinary course in the last 3 years to students
of different majors. He believes that the students’ majors do not have any impact on their
performance in the class. He picked 24 students who represented 3 different majors and recorded
their test scores. Do the data necessitate rejection of the professor’s hypothesis?
Business
Humanities
Science
85
84
83
57
65
93
92
87
83
83
73
94
84
83
79
83
73
74
75
85
95
73
72
98
Question-04: A consumer organization wants to compare the prices charged for a particular
dishwasher at three types of stores in a suburban county: discount stores, department stores, and
appliance stores. Random samples were selected from 4 discount stores, 6 department stores, and
5 appliance stores. The results were as shown.
Discount stores
Department stores
Appliance stores
12
15
18
14
18
15
16
18
16
15
14
16
18
19
15
Required:
1. Construct a one-way analysis of variance (ANOVA) table using the above data.
2. Test whether there is significant difference in average prices between three types of stores.