Forecasting electricity demand
distributions using a
semiparametric additive model
Rob J Hyndman
Joint work with Shu Fan
Forecasting electricity demand distributions
1
Outline
1
The problem
2
The model
3
Long-term forecasts
4
Short term forecasts
5
References and R implementation
Forecasting electricity demand distributions
The problem
2
The problem
We want to forecast the peak electricity
demand in a half-hour period in twenty years
time.
We have fifteen years of half-hourly electricity
data, temperature data and some economic
and demographic data.
The location is South Australia: home to the
most volatile electricity demand in the world.
Sounds impossible?
Forecasting electricity demand distributions
The problem
3
The problem
We want to forecast the peak electricity
demand in a half-hour period in twenty years
time.
We have fifteen years of half-hourly electricity
data, temperature data and some economic
and demographic data.
The location is South Australia: home to the
most volatile electricity demand in the world.
Sounds impossible?
Forecasting electricity demand distributions
The problem
3
The problem
We want to forecast the peak electricity
demand in a half-hour period in twenty years
time.
We have fifteen years of half-hourly electricity
data, temperature data and some economic
and demographic data.
The location is South Australia: home to the
most volatile electricity demand in the world.
Sounds impossible?
Forecasting electricity demand distributions
The problem
3
The problem
We want to forecast the peak electricity
demand in a half-hour period in twenty years
time.
We have fifteen years of half-hourly electricity
data, temperature data and some economic
and demographic data.
The location is South Australia: home to the
most volatile electricity demand in the world.
Sounds impossible?
Forecasting electricity demand distributions
The problem
3
The problem
We want to forecast the peak electricity
demand in a half-hour period in twenty years
time.
We have fifteen years of half-hourly electricity
data, temperature data and some economic
and demographic data.
The location is South Australia: home to the
most volatile electricity demand in the world.
Sounds impossible?
Forecasting electricity demand distributions
The problem
3
South Australian demand data
Forecasting electricity demand distributions
The problem
4
South Australian demand data
Black Saturday →
Forecasting electricity demand distributions
The problem
4
2.0
2.5
3.0
South Australia state wide demand (summer 10/11)
1.5
South Australia state wide demand (GW)
3.5
South Australian demand data
Oct 10
Nov 10
Dec 10
Forecasting electricity demand distributions
Jan 11
Feb 11
The problem
Mar 11
4
South Australian demand data
3.0
2.5
2.0
1.5
South Australian demand (GW)
3.5
South Australia state wide demand (January 2011)
1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31
Date in January
Forecasting electricity demand distributions
The problem
4
Demand boxplots (Sth Aust)
2.5
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Fri
Sat
Sun
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Tue
Wed
Thu
1.5
2.0
●
1.0
Demand (GW)
3.0
3.5
Time: 12 midnight
●
●
Mon
Day of week
Forecasting electricity demand distributions
The problem
5
Temperature data (Sth Aust)
3.5
Time: 12 midnight
2.5
2.0
1.5
1.0
Demand (GW)
3.0
Workday
Non−workday
10
20
30
40
Temperature (deg C)
Forecasting electricity demand distributions
The problem
6
Demand densities (Sth Aust)
2
1
0
Density
3
4
Density of demand: 12 midnight
1.0
1.5
2.0
2.5
3.0
3.5
South Australian half−hourly demand (GW)
Forecasting electricity demand distributions
The problem
7
Industrial offset demand
Summer
Forecasting electricity demand distributions
The problem
8
Outline
1
The problem
2
The model
3
Long-term forecasts
4
Short term forecasts
5
References and R implementation
Forecasting electricity demand distributions
The model
9
Predictors
calendar effects
prevailing and recent weather conditions
climate changes
economic and demographic changes
changing technology
Modelling framework
Semi-parametric additive models with
correlated errors.
Each half-hour period modelled separately for
each season.
Variables selected to provide best
out-of-sample predictions using cross-validation
on each summer.
Forecasting electricity demand distributions
The model
10
Predictors
calendar effects
prevailing and recent weather conditions
climate changes
economic and demographic changes
changing technology
Modelling framework
Semi-parametric additive models with
correlated errors.
Each half-hour period modelled separately for
each season.
Variables selected to provide best
out-of-sample predictions using cross-validation
on each summer.
Forecasting electricity demand distributions
The model
10
Predictors
calendar effects
prevailing and recent weather conditions
climate changes
economic and demographic changes
changing technology
Modelling framework
Semi-parametric additive models with
correlated errors.
Each half-hour period modelled separately for
each season.
Variables selected to provide best
out-of-sample predictions using cross-validation
on each summer.
Forecasting electricity demand distributions
The model
10
Predictors
calendar effects
prevailing and recent weather conditions
climate changes
economic and demographic changes
changing technology
Modelling framework
Semi-parametric additive models with
correlated errors.
Each half-hour period modelled separately for
each season.
Variables selected to provide best
out-of-sample predictions using cross-validation
on each summer.
Forecasting electricity demand distributions
The model
10
Predictors
calendar effects
prevailing and recent weather conditions
climate changes
economic and demographic changes
changing technology
Modelling framework
Semi-parametric additive models with
correlated errors.
Each half-hour period modelled separately for
each season.
Variables selected to provide best
out-of-sample predictions using cross-validation
on each summer.
Forecasting electricity demand distributions
The model
10
Predictors
calendar effects
prevailing and recent weather conditions
climate changes
economic and demographic changes
changing technology
Modelling framework
Semi-parametric additive models with
correlated errors.
Each half-hour period modelled separately for
each season.
Variables selected to provide best
out-of-sample predictions using cross-validation
on each summer.
Forecasting electricity demand distributions
The model
10
Predictors
calendar effects
prevailing and recent weather conditions
climate changes
economic and demographic changes
changing technology
Modelling framework
Semi-parametric additive models with
correlated errors.
Each half-hour period modelled separately for
each season.
Variables selected to provide best
out-of-sample predictions using cross-validation
on each summer.
Forecasting electricity demand distributions
The model
10
Predictors
calendar effects
prevailing and recent weather conditions
climate changes
economic and demographic changes
changing technology
Modelling framework
Semi-parametric additive models with
correlated errors.
Each half-hour period modelled separately for
each season.
Variables selected to provide best
out-of-sample predictions using cross-validation
on each summer.
Forecasting electricity demand distributions
The model
10
Predictors
calendar effects
prevailing and recent weather conditions
climate changes
economic and demographic changes
changing technology
Modelling framework
Semi-parametric additive models with
correlated errors.
Each half-hour period modelled separately for
each season.
Variables selected to provide best
out-of-sample predictions using cross-validation
on each summer.
Forecasting electricity demand distributions
The model
10
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
yt denotes per capita demand (minus offset) at time t
(measured in half-hourly intervals) and p denotes the
time of day p = 1, . . . , 48;
hp (t ) models all calendar effects;
fp (w1,t , w2,t ) models all temperature effects where w1,t is
a vector of recent temperatures at location 1 and w2,t is
a vector of recent temperatures at location 2;
zj,t is a demographic or economic variable at time t
nt denotes the model error at time t.
Forecasting electricity demand distributions
The model
11
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
yt denotes per capita demand (minus offset) at time t
(measured in half-hourly intervals) and p denotes the
time of day p = 1, . . . , 48;
hp (t ) models all calendar effects;
fp (w1,t , w2,t ) models all temperature effects where w1,t is
a vector of recent temperatures at location 1 and w2,t is
a vector of recent temperatures at location 2;
zj,t is a demographic or economic variable at time t
nt denotes the model error at time t.
Forecasting electricity demand distributions
The model
11
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
yt denotes per capita demand (minus offset) at time t
(measured in half-hourly intervals) and p denotes the
time of day p = 1, . . . , 48;
hp (t ) models all calendar effects;
fp (w1,t , w2,t ) models all temperature effects where w1,t is
a vector of recent temperatures at location 1 and w2,t is
a vector of recent temperatures at location 2;
zj,t is a demographic or economic variable at time t
nt denotes the model error at time t.
Forecasting electricity demand distributions
The model
11
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
yt denotes per capita demand (minus offset) at time t
(measured in half-hourly intervals) and p denotes the
time of day p = 1, . . . , 48;
hp (t ) models all calendar effects;
fp (w1,t , w2,t ) models all temperature effects where w1,t is
a vector of recent temperatures at location 1 and w2,t is
a vector of recent temperatures at location 2;
zj,t is a demographic or economic variable at time t
nt denotes the model error at time t.
Forecasting electricity demand distributions
The model
11
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
yt denotes per capita demand (minus offset) at time t
(measured in half-hourly intervals) and p denotes the
time of day p = 1, . . . , 48;
hp (t ) models all calendar effects;
fp (w1,t , w2,t ) models all temperature effects where w1,t is
a vector of recent temperatures at location 1 and w2,t is
a vector of recent temperatures at location 2;
zj,t is a demographic or economic variable at time t
nt denotes the model error at time t.
Forecasting electricity demand distributions
The model
11
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
hp (t ) includes handle annual, weekly and daily seasonal
patterns as well as public holidays:
hp (t ) = `p (t ) + αt,p + βt,p + γt,p + δt,p
`p (t) is “time of summer” effect (a regression spline);
αt,p is day of week effect;
βt,p is “holiday” effect;
γt,p New Year’s Eve effect;
δt,p is millennium effect;
Forecasting electricity demand distributions
The model
12
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
hp (t ) includes handle annual, weekly and daily seasonal
patterns as well as public holidays:
hp (t ) = `p (t ) + αt,p + βt,p + γt,p + δt,p
`p (t) is “time of summer” effect (a regression spline);
αt,p is day of week effect;
βt,p is “holiday” effect;
γt,p New Year’s Eve effect;
δt,p is millennium effect;
Forecasting electricity demand distributions
The model
12
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
hp (t ) includes handle annual, weekly and daily seasonal
patterns as well as public holidays:
hp (t ) = `p (t ) + αt,p + βt,p + γt,p + δt,p
`p (t) is “time of summer” effect (a regression spline);
αt,p is day of week effect;
βt,p is “holiday” effect;
γt,p New Year’s Eve effect;
δt,p is millennium effect;
Forecasting electricity demand distributions
The model
12
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
hp (t ) includes handle annual, weekly and daily seasonal
patterns as well as public holidays:
hp (t ) = `p (t ) + αt,p + βt,p + γt,p + δt,p
`p (t) is “time of summer” effect (a regression spline);
αt,p is day of week effect;
βt,p is “holiday” effect;
γt,p New Year’s Eve effect;
δt,p is millennium effect;
Forecasting electricity demand distributions
The model
12
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
hp (t ) includes handle annual, weekly and daily seasonal
patterns as well as public holidays:
hp (t ) = `p (t ) + αt,p + βt,p + γt,p + δt,p
`p (t) is “time of summer” effect (a regression spline);
αt,p is day of week effect;
βt,p is “holiday” effect;
γt,p New Year’s Eve effect;
δt,p is millennium effect;
Forecasting electricity demand distributions
The model
12
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
hp (t ) includes handle annual, weekly and daily seasonal
patterns as well as public holidays:
hp (t ) = `p (t ) + αt,p + βt,p + γt,p + δt,p
`p (t) is “time of summer” effect (a regression spline);
αt,p is day of week effect;
βt,p is “holiday” effect;
γt,p New Year’s Eve effect;
δt,p is millennium effect;
Forecasting electricity demand distributions
The model
12
Fitted results (Summer 3pm)
0
50
100
150
0.4
0.0
Mon
Tue
Wed
Thu
Fri
Sat
Sun
0.0
0.4
Day of week
−0.4
Effect on demand
Day of summer
−0.4
Effect on demand
0.4
0.0
−0.4
Effect on demand
Time: 3:00 pm
Normal
Day before
Holiday
Day after
Holiday
Forecasting electricity demand distributions
The model
13
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
6
fp (w1,t , w2,t ) =
X
fk,p (xt−k ) + gk,p (dt−k ) + qp (xt+ ) + rp (xt− ) + sp (x̄t )
k =0
+
6
X
Fj,p (xt−48j ) + Gj,p (dt−48j )
j=1
xt is ave temp across two sites (Kent Town and Adelaide
Airport) at time t;
dt is the temp difference between two sites at time t;
xt+ is max of xt values in past 24 hours;
xt− is min of xt values in past 24 hours;
x̄t is ave temp in past seven days.
Each function is smooth & estimated using regression splines.
Forecasting electricity demand distributions
The model
14
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
6
fp (w1,t , w2,t ) =
X
fk,p (xt−k ) + gk,p (dt−k ) + qp (xt+ ) + rp (xt− ) + sp (x̄t )
k =0
+
6
X
Fj,p (xt−48j ) + Gj,p (dt−48j )
j=1
xt is ave temp across two sites (Kent Town and Adelaide
Airport) at time t;
dt is the temp difference between two sites at time t;
xt+ is max of xt values in past 24 hours;
xt− is min of xt values in past 24 hours;
x̄t is ave temp in past seven days.
Each function is smooth & estimated using regression splines.
Forecasting electricity demand distributions
The model
14
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
6
fp (w1,t , w2,t ) =
X
fk,p (xt−k ) + gk,p (dt−k ) + qp (xt+ ) + rp (xt− ) + sp (x̄t )
k =0
+
6
X
Fj,p (xt−48j ) + Gj,p (dt−48j )
j=1
xt is ave temp across two sites (Kent Town and Adelaide
Airport) at time t;
dt is the temp difference between two sites at time t;
xt+ is max of xt values in past 24 hours;
xt− is min of xt values in past 24 hours;
x̄t is ave temp in past seven days.
Each function is smooth & estimated using regression splines.
Forecasting electricity demand distributions
The model
14
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
6
fp (w1,t , w2,t ) =
X
fk,p (xt−k ) + gk,p (dt−k ) + qp (xt+ ) + rp (xt− ) + sp (x̄t )
k =0
+
6
X
Fj,p (xt−48j ) + Gj,p (dt−48j )
j=1
xt is ave temp across two sites (Kent Town and Adelaide
Airport) at time t;
dt is the temp difference between two sites at time t;
xt+ is max of xt values in past 24 hours;
xt− is min of xt values in past 24 hours;
x̄t is ave temp in past seven days.
Each function is smooth & estimated using regression splines.
Forecasting electricity demand distributions
The model
14
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
6
fp (w1,t , w2,t ) =
X
fk,p (xt−k ) + gk,p (dt−k ) + qp (xt+ ) + rp (xt− ) + sp (x̄t )
k =0
+
6
X
Fj,p (xt−48j ) + Gj,p (dt−48j )
j=1
xt is ave temp across two sites (Kent Town and Adelaide
Airport) at time t;
dt is the temp difference between two sites at time t;
xt+ is max of xt values in past 24 hours;
xt− is min of xt values in past 24 hours;
x̄t is ave temp in past seven days.
Each function is smooth & estimated using regression splines.
Forecasting electricity demand distributions
The model
14
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
6
fp (w1,t , w2,t ) =
X
fk,p (xt−k ) + gk,p (dt−k ) + qp (xt+ ) + rp (xt− ) + sp (x̄t )
k =0
+
6
X
Fj,p (xt−48j ) + Gj,p (dt−48j )
j=1
xt is ave temp across two sites (Kent Town and Adelaide
Airport) at time t;
dt is the temp difference between two sites at time t;
xt+ is max of xt values in past 24 hours;
xt− is min of xt values in past 24 hours;
x̄t is ave temp in past seven days.
Each function is smooth & estimated using regression splines.
Forecasting electricity demand distributions
The model
14
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
6
fp (w1,t , w2,t ) =
X
fk,p (xt−k ) + gk,p (dt−k ) + qp (xt+ ) + rp (xt− ) + sp (x̄t )
k =0
+
6
X
Fj,p (xt−48j ) + Gj,p (dt−48j )
j=1
xt is ave temp across two sites (Kent Town and Adelaide
Airport) at time t;
dt is the temp difference between two sites at time t;
xt+ is max of xt values in past 24 hours;
xt− is min of xt values in past 24 hours;
x̄t is ave temp in past seven days.
Each function is smooth & estimated using regression splines.
Forecasting electricity demand distributions
The model
14
Fitted results (Summer 3pm)
40
30
40
Lag 1 day temperature
0.2
0.4
30
40
10
20
30
40
0.0
−0.2
Effect on demand
0.2
0.4
Lag 3 temperature
0.4
Effect on demand
Forecasting electricity demand distributions
0.0
Effect on demand
20
Lag 2 temperature
−0.4
10 15 20 25 30
Last week average temp
−0.2
−0.4
10
0.4
0.2
0.0
Effect on demand
−0.4
20
0.2
0.4
30
−0.2
0.4
0.2
0.0
−0.2
10
0.0
Effect on demand
20
Lag 1 temperature
−0.4
Effect on demand
−0.4
10
−0.4
40
0.2
30
0.0
20
Temperature
−0.2
10
−0.2
0.4
0.2
0.0
Effect on demand
−0.4
−0.2
0.2
0.0
−0.2
−0.4
Effect on demand
0.4
Time: 3:00 pm
15
25
35
Previous max temp
The model
10
15
20
25
Previous min temp
15
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
Other variables described by linear
relationships with coefficients c1 , . . . , cJ .
Estimation based on annual data.
Forecasting electricity demand distributions
The model
16
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
Other variables described by linear
relationships with coefficients c1 , . . . , cJ .
Estimation based on annual data.
Forecasting electricity demand distributions
The model
16
Monash Electricity Forecasting Model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
log(yt ) = log(yt∗ ) + log(ȳi )
log(yt∗ ) = hp (t ) + fp (w1,t , w2,t ) + et
log(ȳi ) =
J
X
cj zj,i + εi
j =1
ȳi is the average demand for year i where t is in
year i.
yt∗ is the standardized demand for time t.
Forecasting electricity demand distributions
The model
17
Monash Electricity Forecasting Model
Forecasting electricity demand distributions
The model
18
Monash Electricity Forecasting Model
Forecasting electricity demand distributions
The model
18
Annual model
log(ȳi ) =
X
log(ȳi ) − log(ȳi−1 ) =
X
cj zj,i + εi
j
cj (zj,i − zj,i−1 ) + ε∗i
j
First differences modelled to avoid
non-stationary variables.
Predictors: Per-capita GSP, Price, Summer CDD,
Winter HDD.
Forecasting electricity demand distributions
The model
19
Annual model
log(ȳi ) =
X
log(ȳi ) − log(ȳi−1 ) =
X
cj zj,i + εi
j
cj (zj,i − zj,i−1 ) + ε∗i
j
First differences modelled to avoid
non-stationary variables.
Predictors: Per-capita GSP, Price, Summer CDD,
Winter HDD.
Forecasting electricity demand distributions
The model
19
Annual model
log(ȳi ) =
X
log(ȳi ) − log(ȳi−1 ) =
X
cj zj,i + εi
j
cj (zj,i − zj,i−1 ) + ε∗i
j
First differences modelled to avoid
non-stationary variables.
Predictors: Per-capita GSP, Price, Summer CDD,
Winter HDD.
zCDD =
X
max(0, T̄ − 18.5)
summer
T̄ = daily mean
Forecasting electricity demand distributions
The model
19
Cooling and Heating degree days
Annual model
400
950 1050
850
whdd
200
scdd
600
Cooling and Heating Degree Days
1990
1995
Forecasting electricity demand distributions
2000
2005
The model
2010
20
Annual model
Variable Coefficient
∆gsp.pc
2.02×10−6
∆price
−1.67×10−8
∆scdd
1.11×10−10
∆whdd
2.07×10−11
Std. Error t value P value
5.05×10−6
0.38
0.711
−9
6.76×10
−2.46
0.026
2.48×10−11
4.49
0.000
−11
3.28×10
0.63
0.537
GSP needed to stay in the model to allow
scenario forecasting.
All other variables led to improved AICC .
Forecasting electricity demand distributions
The model
21
Annual model
Variable Coefficient
∆gsp.pc
2.02×10−6
∆price
−1.67×10−8
∆scdd
1.11×10−10
∆whdd
2.07×10−11
Std. Error t value P value
5.05×10−6
0.38
0.711
−9
6.76×10
−2.46
0.026
2.48×10−11
4.49
0.000
−11
3.28×10
0.63
0.537
GSP needed to stay in the model to allow
scenario forecasting.
All other variables led to improved AICC .
Forecasting electricity demand distributions
The model
21
Annual model
Variable Coefficient
∆gsp.pc
2.02×10−6
∆price
−1.67×10−8
∆scdd
1.11×10−10
∆whdd
2.07×10−11
Std. Error t value P value
5.05×10−6
0.38
0.711
−9
6.76×10
−2.46
0.026
2.48×10−11
4.49
0.000
−11
3.28×10
0.63
0.537
GSP needed to stay in the model to allow
scenario forecasting.
All other variables led to improved AICC .
Forecasting electricity demand distributions
The model
21
Actual
Fitted
1.4
1.3
1.2
1.1
1.0
Annual demand
1.5
1.6
1.7
Annual model
89/90
91/92
93/94
95/96
97/98
99/00
01/02
03/04
05/06
07/08
09/10
Year
Forecasting electricity demand distributions
The model
22
Half-hourly models
log(yt ) = log(yt∗ ) + log(ȳi )
log(yt∗ ) = hp (t ) + fp (w1,t , w2,t ) + et
Separate model for each half-hour.
Same predictors used for all models.
Predictors chosen by cross-validation on
summer of 2007/2008 and 2009/2010.
Each model is fitted to the data twice, first
excluding the summer of 2009/2010 and then
excluding the summer of 2010/2011. The
average out-of-sample MSE is calculated from
the omitted data for the time periods
12noon–8.30pm.
Forecasting electricity demand distributions
The model
23
Half-hourly models
log(yt ) = log(yt∗ ) + log(ȳi )
log(yt∗ ) = hp (t ) + fp (w1,t , w2,t ) + et
Separate model for each half-hour.
Same predictors used for all models.
Predictors chosen by cross-validation on
summer of 2007/2008 and 2009/2010.
Each model is fitted to the data twice, first
excluding the summer of 2009/2010 and then
excluding the summer of 2010/2011. The
average out-of-sample MSE is calculated from
the omitted data for the time periods
12noon–8.30pm.
Forecasting electricity demand distributions
The model
23
Half-hourly models
log(yt ) = log(yt∗ ) + log(ȳi )
log(yt∗ ) = hp (t ) + fp (w1,t , w2,t ) + et
Separate model for each half-hour.
Same predictors used for all models.
Predictors chosen by cross-validation on
summer of 2007/2008 and 2009/2010.
Each model is fitted to the data twice, first
excluding the summer of 2009/2010 and then
excluding the summer of 2010/2011. The
average out-of-sample MSE is calculated from
the omitted data for the time periods
12noon–8.30pm.
Forecasting electricity demand distributions
The model
23
Half-hourly models
log(yt ) = log(yt∗ ) + log(ȳi )
log(yt∗ ) = hp (t ) + fp (w1,t , w2,t ) + et
Separate model for each half-hour.
Same predictors used for all models.
Predictors chosen by cross-validation on
summer of 2007/2008 and 2009/2010.
Each model is fitted to the data twice, first
excluding the summer of 2009/2010 and then
excluding the summer of 2010/2011. The
average out-of-sample MSE is calculated from
the omitted data for the time periods
12noon–8.30pm.
Forecasting electricity demand distributions
The model
23
Half-hourly models
log(yt ) = log(yt∗ ) + log(ȳi )
log(yt∗ ) = hp (t ) + fp (w1,t , w2,t ) + et
Separate model for each half-hour.
Same predictors used for all models.
Predictors chosen by cross-validation on
summer of 2007/2008 and 2009/2010.
Each model is fitted to the data twice, first
excluding the summer of 2009/2010 and then
excluding the summer of 2010/2011. The
average out-of-sample MSE is calculated from
the omitted data for the time periods
12noon–8.30pm.
Forecasting electricity demand distributions
The model
23
Half-hourly models
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
x x1 x2 x3 x4 x5 x6 x48 x96 x144 x192 x240 x288 d d1 d2 d3 d4 d5 d6 d48 d96 d144 d192 d240 d288 x+ x− x̄ dow hol dos MSE
• • • • • • • • •
•
•
• • • • • • • • • •
•
•
•
• • • • • • • 1.037
• • • • • • • • •
•
•
• • • • • • • • • •
•
•
•
• • • • • • 1.034
• • • • • • • • •
•
•
• • • • • • • • • •
•
•
• • • • • • 1.031
• • • • • • • • •
•
•
• • • • • • • • • •
•
• • • • • • 1.027
• • • • • • • • •
•
•
• • • • • • • • • •
• • • • • • 1.025
• • • • • • • • •
•
•
• • • • • • • • •
• • • • • • 1.020
• • • • • • • • •
•
•
• • • • • • • •
• • • • • • 1.025
• • • • • • • • •
•
•
• • • • • • •
•
• • • • • • 1.026
• • • • • • • • •
•
•
• • • • • •
•
• • • • • • 1.035
• • • • • • • • •
•
•
• • • • •
•
• • • • • • 1.044
• • • • • • • • •
•
•
• • • •
•
• • • • • • 1.057
• • • • • • • • •
•
•
• • •
•
• • • • • • 1.076
• • • • • • • • •
•
•
• •
•
• • • • • • 1.102
• • • • • • • • •
•
•
• • • • • • • •
• • • • • • 1.018
• • • • • • • • •
•
• • • • • • • •
• • • • • • 1.021
• • • • • • • • •
• • • • • • • •
• • • • • • 1.037
• • • • • • • •
• • • • • • • •
• • • • • • 1.074
• • • • • • •
• • • • • • • •
• • • • • • 1.152
• • • • • •
• • • • • • • •
• • • • • • 1.180
• • • • •
• • •
•
•
• • • • • • • •
• • • • • • 1.021
• • • •
• • •
•
•
• • • • • • • •
• • • • • • 1.027
• • •
• • •
•
•
• • • • • • • •
• • • • • • 1.038
• •
• • •
•
•
• • • • • • • •
• • • • • • 1.056
•
• • •
•
•
• • • • • • • •
• • • • • • 1.086
• • •
•
•
• • • • • • • •
• • • • • • 1.135
• • • • • • • • •
•
•
• • • • • • • •
• • • • • 1.009
• • • • • • • • •
•
•
• • • • • • • •
•
• • • • 1.063
• • • • • • • • •
•
•
• • • • • • • •
• •
• • • 1.028
• • • • • • • • •
•
•
• • • • • • • •
• • •
• • 3.523
• • • • • • • • •
•
•
• • • • • • • •
• • • •
• 2.143
• • • • • • • • •
•
•
• • • • • • • •
• • • • •
1.523
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Forecasting electricity demand distributions
The model
24
Half-hourly models
80
70
60
R−squared (%)
90
R−squared
12 midnight 3:00 am
6:00 am
9:00 am
12 noon
3:00 pm
6:00 pm
9:00 pm 12 midnight
Time of day
Forecasting electricity demand distributions
The model
25
Half-hourly models
1.5
2.0
2.5
3.0
3.5
Actual
Fitted
1.0
South Australian demand (GW)
4.0
South Australian demand (January 2011)
1
3
5
7
9
11
Forecasting electricity demand distributions
13
15
17
19
Date in January
21
23
The model
25
27
29
31
25
Half-hourly models
Forecasting electricity demand distributions
The model
25
Half-hourly models
Forecasting electricity demand distributions
The model
25
Adjusted model
Original model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
Model allowing saturated usage
qt = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
log(yt ) =
if qt ≤ τ ;
τ + k(qt − τ ) if qt > τ .
qt
Forecasting electricity demand distributions
The model
26
Adjusted model
Original model
log(yt ) = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j =1
Model allowing saturated usage
qt = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
log(yt ) =
if qt ≤ τ ;
τ + k(qt − τ ) if qt > τ .
qt
Forecasting electricity demand distributions
The model
26
Outline
1
The problem
2
The model
3
Long-term forecasts
4
Short term forecasts
5
References and R implementation
Forecasting electricity demand distributions
Long-term forecasts
27
Peak demand forecasting
qt,p = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
Multiple alternative futures created:
hp (t ) known;
simulate future temperatures using double
seasonal block bootstrap with variable blocks
(with adjustment for climate change);
use assumed values for GSP, population and
price;
resample residuals using double seasonal block
bootstrap with variable blocks.
Forecasting electricity demand distributions
Long-term forecasts
28
Peak demand forecasting
qt,p = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
Multiple alternative futures created:
hp (t ) known;
simulate future temperatures using double
seasonal block bootstrap with variable blocks
(with adjustment for climate change);
use assumed values for GSP, population and
price;
resample residuals using double seasonal block
bootstrap with variable blocks.
Forecasting electricity demand distributions
Long-term forecasts
28
Peak demand forecasting
qt,p = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
Multiple alternative futures created:
hp (t ) known;
simulate future temperatures using double
seasonal block bootstrap with variable blocks
(with adjustment for climate change);
use assumed values for GSP, population and
price;
resample residuals using double seasonal block
bootstrap with variable blocks.
Forecasting electricity demand distributions
Long-term forecasts
28
Peak demand forecasting
qt,p = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
Multiple alternative futures created:
hp (t ) known;
simulate future temperatures using double
seasonal block bootstrap with variable blocks
(with adjustment for climate change);
use assumed values for GSP, population and
price;
resample residuals using double seasonal block
bootstrap with variable blocks.
Forecasting electricity demand distributions
Long-term forecasts
28
Seasonal block bootstrapping
Conventional seasonal block bootstrap
Same as block bootstrap but with whole years as the
blocks to preserve seasonality.
But we only have about 10–15 years of data, so there is a
limited number of possible bootstrap samples.
Double seasonal block bootstrap
Suitable when there are two seasonal periods (here we
have years of 151 days and days of 48 half-hours).
Divide each year into blocks of length 48m.
Block 1 consists of the first m days of the year, block 2
consists of the next m days, and so on.
Bootstrap sample consists of a sample of blocks where
each block may come from a different randomly selected
year but must be at the correct time of year.
Forecasting electricity demand distributions
Long-term forecasts
29
Seasonal block bootstrapping
Conventional seasonal block bootstrap
Same as block bootstrap but with whole years as the
blocks to preserve seasonality.
But we only have about 10–15 years of data, so there is a
limited number of possible bootstrap samples.
Double seasonal block bootstrap
Suitable when there are two seasonal periods (here we
have years of 151 days and days of 48 half-hours).
Divide each year into blocks of length 48m.
Block 1 consists of the first m days of the year, block 2
consists of the next m days, and so on.
Bootstrap sample consists of a sample of blocks where
each block may come from a different randomly selected
year but must be at the correct time of year.
Forecasting electricity demand distributions
Long-term forecasts
29
Seasonal block bootstrapping
Conventional seasonal block bootstrap
Same as block bootstrap but with whole years as the
blocks to preserve seasonality.
But we only have about 10–15 years of data, so there is a
limited number of possible bootstrap samples.
Double seasonal block bootstrap
Suitable when there are two seasonal periods (here we
have years of 151 days and days of 48 half-hours).
Divide each year into blocks of length 48m.
Block 1 consists of the first m days of the year, block 2
consists of the next m days, and so on.
Bootstrap sample consists of a sample of blocks where
each block may come from a different randomly selected
year but must be at the correct time of year.
Forecasting electricity demand distributions
Long-term forecasts
29
Seasonal block bootstrapping
Conventional seasonal block bootstrap
Same as block bootstrap but with whole years as the
blocks to preserve seasonality.
But we only have about 10–15 years of data, so there is a
limited number of possible bootstrap samples.
Double seasonal block bootstrap
Suitable when there are two seasonal periods (here we
have years of 151 days and days of 48 half-hours).
Divide each year into blocks of length 48m.
Block 1 consists of the first m days of the year, block 2
consists of the next m days, and so on.
Bootstrap sample consists of a sample of blocks where
each block may come from a different randomly selected
year but must be at the correct time of year.
Forecasting electricity demand distributions
Long-term forecasts
29
Seasonal block bootstrapping
Conventional seasonal block bootstrap
Same as block bootstrap but with whole years as the
blocks to preserve seasonality.
But we only have about 10–15 years of data, so there is a
limited number of possible bootstrap samples.
Double seasonal block bootstrap
Suitable when there are two seasonal periods (here we
have years of 151 days and days of 48 half-hours).
Divide each year into blocks of length 48m.
Block 1 consists of the first m days of the year, block 2
consists of the next m days, and so on.
Bootstrap sample consists of a sample of blocks where
each block may come from a different randomly selected
year but must be at the correct time of year.
Forecasting electricity demand distributions
Long-term forecasts
29
Seasonal block bootstrapping
Conventional seasonal block bootstrap
Same as block bootstrap but with whole years as the
blocks to preserve seasonality.
But we only have about 10–15 years of data, so there is a
limited number of possible bootstrap samples.
Double seasonal block bootstrap
Suitable when there are two seasonal periods (here we
have years of 151 days and days of 48 half-hours).
Divide each year into blocks of length 48m.
Block 1 consists of the first m days of the year, block 2
consists of the next m days, and so on.
Bootstrap sample consists of a sample of blocks where
each block may come from a different randomly selected
year but must be at the correct time of year.
Forecasting electricity demand distributions
Long-term forecasts
29
Seasonal block bootstrapping
Conventional seasonal block bootstrap
Same as block bootstrap but with whole years as the
blocks to preserve seasonality.
But we only have about 10–15 years of data, so there is a
limited number of possible bootstrap samples.
Double seasonal block bootstrap
Suitable when there are two seasonal periods (here we
have years of 151 days and days of 48 half-hours).
Divide each year into blocks of length 48m.
Block 1 consists of the first m days of the year, block 2
consists of the next m days, and so on.
Bootstrap sample consists of a sample of blocks where
each block may come from a different randomly selected
year but must be at the correct time of year.
Forecasting electricity demand distributions
Long-term forecasts
29
Seasonal block bootstrapping
25
20
10
15
degrees C
30
35
40
Actual temperatures
0
10
20
30
40
50
60
50
60
Days
25
20
10
15
degrees C
30
35
40
Bootstrap temperatures (fixed blocks)
0
10
20
30
40
Days
Bootstrap temperatures (variable blocks)
40
Forecasting electricity demand distributions
Long-term forecasts
30
Seasonal block bootstrapping
Problems with the double seasonal bootstrap
Boundaries between blocks can introduce large
jumps. However, only at midnight.
Number of values that any given time in year is
still limited to the number of years in the data
set.
Forecasting electricity demand distributions
Long-term forecasts
31
Seasonal block bootstrapping
Problems with the double seasonal bootstrap
Boundaries between blocks can introduce large
jumps. However, only at midnight.
Number of values that any given time in year is
still limited to the number of years in the data
set.
Forecasting electricity demand distributions
Long-term forecasts
31
Seasonal block bootstrapping
Variable length double seasonal block
bootstrap
Blocks allowed to vary in length between m − ∆
and m + ∆ days where 0 ≤ ∆ < m.
Blocks allowed to move up to ∆ days from their
original position.
Has little effect on the overall time series
patterns provided ∆ is relatively small.
Use uniform distribution on (m − ∆, m + ∆) to
select block length, and independent uniform
distribution on (−∆, ∆) to select variation on
starting position for each block.
Forecasting electricity demand distributions
Long-term forecasts
32
Seasonal block bootstrapping
Variable length double seasonal block
bootstrap
Blocks allowed to vary in length between m − ∆
and m + ∆ days where 0 ≤ ∆ < m.
Blocks allowed to move up to ∆ days from their
original position.
Has little effect on the overall time series
patterns provided ∆ is relatively small.
Use uniform distribution on (m − ∆, m + ∆) to
select block length, and independent uniform
distribution on (−∆, ∆) to select variation on
starting position for each block.
Forecasting electricity demand distributions
Long-term forecasts
32
Seasonal block bootstrapping
Variable length double seasonal block
bootstrap
Blocks allowed to vary in length between m − ∆
and m + ∆ days where 0 ≤ ∆ < m.
Blocks allowed to move up to ∆ days from their
original position.
Has little effect on the overall time series
patterns provided ∆ is relatively small.
Use uniform distribution on (m − ∆, m + ∆) to
select block length, and independent uniform
distribution on (−∆, ∆) to select variation on
starting position for each block.
Forecasting electricity demand distributions
Long-term forecasts
32
Seasonal block bootstrapping
Variable length double seasonal block
bootstrap
Blocks allowed to vary in length between m − ∆
and m + ∆ days where 0 ≤ ∆ < m.
Blocks allowed to move up to ∆ days from their
original position.
Has little effect on the overall time series
patterns provided ∆ is relatively small.
Use uniform distribution on (m − ∆, m + ∆) to
select block length, and independent uniform
distribution on (−∆, ∆) to select variation on
starting position for each block.
Forecasting electricity demand distributions
Long-term forecasts
32
Seasonal block bootstrapping
25
20
10
15
degrees C
30
35
40
Actual temperatures
0
10
20
30
40
50
60
50
60
50
60
Days
25
20
10
15
degrees C
30
35
40
Bootstrap temperatures (fixed blocks)
0
10
20
30
40
Days
25
20
10
15
degrees C
30
35
40
Bootstrap temperatures (variable blocks)
0
10
20
30
40
Days
Forecasting electricity demand distributions
Long-term forecasts
33
Seasonal block bootstrapping
Forecasting electricity demand distributions
Long-term forecasts
33
Peak demand forecasting
Climate change adjustments
CSIRO estimates for 2030:
0.3◦ C for 10th percentile
0.9◦ C for 50th percentile
1.5◦ C for 90th percentile
We implement these shifts linearly from 2010.
No change in the variation in temperature.
Thousands of “futures” generated using a
seasonal bootstrap.
Forecasting electricity demand distributions
Long-term forecasts
34
Peak demand forecasting
Climate change adjustments
CSIRO estimates for 2030:
0.3◦ C for 10th percentile
0.9◦ C for 50th percentile
1.5◦ C for 90th percentile
We implement these shifts linearly from 2010.
No change in the variation in temperature.
Thousands of “futures” generated using a
seasonal bootstrap.
Forecasting electricity demand distributions
Long-term forecasts
34
Peak demand forecasting
Climate change adjustments
CSIRO estimates for 2030:
0.3◦ C for 10th percentile
0.9◦ C for 50th percentile
1.5◦ C for 90th percentile
We implement these shifts linearly from 2010.
No change in the variation in temperature.
Thousands of “futures” generated using a
seasonal bootstrap.
Forecasting electricity demand distributions
Long-term forecasts
34
Peak demand forecasting
Climate change adjustments
CSIRO estimates for 2030:
0.3◦ C for 10th percentile
0.9◦ C for 50th percentile
1.5◦ C for 90th percentile
We implement these shifts linearly from 2010.
No change in the variation in temperature.
Thousands of “futures” generated using a
seasonal bootstrap.
Forecasting electricity demand distributions
Long-term forecasts
34
Peak demand forecasting
Climate change adjustments
CSIRO estimates for 2030:
0.3◦ C for 10th percentile
0.9◦ C for 50th percentile
1.5◦ C for 90th percentile
We implement these shifts linearly from 2010.
No change in the variation in temperature.
Thousands of “futures” generated using a
seasonal bootstrap.
Forecasting electricity demand distributions
Long-term forecasts
34
Peak demand forecasting
Climate change adjustments
CSIRO estimates for 2030:
0.3◦ C for 10th percentile
0.9◦ C for 50th percentile
1.5◦ C for 90th percentile
We implement these shifts linearly from 2010.
No change in the variation in temperature.
Thousands of “futures” generated using a
seasonal bootstrap.
Forecasting electricity demand distributions
Long-term forecasts
34
Peak demand forecasting
Climate change adjustments
CSIRO estimates for 2030:
0.3◦ C for 10th percentile
0.9◦ C for 50th percentile
1.5◦ C for 90th percentile
We implement these shifts linearly from 2010.
No change in the variation in temperature.
Thousands of “futures” generated using a
seasonal bootstrap.
Forecasting electricity demand distributions
Long-term forecasts
34
Peak demand forecasting
qt,p = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
Multiple alternative futures created:
hp (t ) known;
simulate future temperatures using double
seasonal block bootstrap with variable
blocks (with adjustment for climate change);
use assumed values for GSP, population and
price;
resample residuals using double seasonal block
bootstrap with variable blocks.
Forecasting electricity demand distributions
Long-term forecasts
35
Peak demand backcasting
qt,p = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
Multiple alternative pasts created:
hp (t ) known;
simulate past temperatures using double
seasonal block bootstrap with variable
blocks;
use actual values for GSP, population and
price;
resample residuals using double seasonal block
bootstrap with variable blocks.
Forecasting electricity demand distributions
Long-term forecasts
35
Peak demand backcasting
4.0
PoE (annual interpretation)
3.5
10 %
50 %
90 %
●
●
3.0
●
●
●
●
●
●
2.5
●
●
●
●
●
2.0
PoE Demand
●
98/99
00/01
02/03
04/05
06/07
08/09
10/11
Year
Forecasting electricity demand distributions
Long-term forecasts
36
Peak demand forecasting
qt,p = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
Multiple alternative futures created:
hp (t ) known;
simulate future temperatures using double
seasonal block bootstrap with variable
blocks (with adjustment for climate change);
use assumed values for GSP, population and
price;
resample residuals using double seasonal block
bootstrap with variable blocks.
Forecasting electricity demand distributions
Long-term forecasts
37
120
South Australia GSP
60
80
100
High
Base
Low
40
billion dollars (08/09 dollars)
Peak demand forecasting
1990
1995
2000
2005
2010
2015
2020
2015
2020
2015
2020
Year
2.0
South Australia population
1.4
1.6
million
1.8
High
Base
Low
1990
1995
2000
2005
2010
Year
Average electricity prices
18
16
12
14
c/kWh
20
22
High
Base
Low
1990
1995
2000
2005
2010
Year
0
Forecasting electricity demand distributions
Major industrial offset demand Long-term forecasts
38
Peak demand distribution
1.0
0.5
0.0
Density
1.5
2.0
Forecast density of annual maximum demand: 2009/2010
2.5
3.0
3.5
4.0
4.5
5.0
Demand (GW)
Forecasting electricity demand distributions
Long-term forecasts
39
Peak demand distribution
5
6
Annual POE levels
4
●
3
●
●
●
●
●
●
●
●
●
●
●
●
2
PoE Demand
●
1 % POE
5 % POE
10 % POE
50 % POE
90 % POE
Actual annual maximum
98/99 00/01 02/03 04/05 06/07 08/09 10/11 12/13 14/15 16/17 18/19 20/21
Year
Forecasting electricity demand distributions
Long-term forecasts
40
Peak demand forecasting
Base
1.0
Density
0.5
1.0
0.0
0.0
0.5
Density
1.5
1.5
Low
2.5
3.0
3.5
4.0
4.5
5.0
5.5
Demand (GW)
2.5
3.0
3.5
4.0
4.5
5.0
5.5
Demand (GW)
1.0
2011/2012
2012/2013
2013/2014
2014/2015
2015/2016
2016/2017
2017/2018
2018/2019
2019/2020
2020/2021
0.0
0.5
Density
1.5
High
2.5
3.0
3.5
4.0
4.5
5.0
5.5
Demand (GW)
Forecasting electricity demand distributions
Long-term forecasts
41
Peak demand forecasting
Percentage
60
80
100
Base
0
20
40
60
40
0
20
Percentage
80
100
Low
2.5
3.0
3.5
4.0
4.5
5.0
Quantile
2.5
3.0
3.5
4.0
4.5
5.0
Quantile
60
40
2011/2012
2012/2013
2013/2014
2014/2015
2015/2016
2016/2017
2017/2018
2018/2019
2019/2020
2020/2021
0
20
Percentage
80
100
High
2.5
3.0
3.5
4.0
4.5
5.0
Quantile
Forecasting electricity demand distributions
Long-term forecasts
41
Outline
1
The problem
2
The model
3
Long-term forecasts
4
Short term forecasts
5
References and R implementation
Forecasting electricity demand distributions
Short term forecasts
42
Short term forecasts
qt,p = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
Bootstrapping temperatures and residuals is ok
for long-term forecasts because short-term
dynamics wash out after a few weeks.
But short-term forecasts need to take account
of recent temperatures and recent residuals
due to serial correlation.
Short-term temperature forecasts are available.
Building a separate model for nt is possible, but
there is a simpler approach.
Forecasting electricity demand distributions
Short term forecasts
43
Short term forecasts
qt,p = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
Bootstrapping temperatures and residuals is ok
for long-term forecasts because short-term
dynamics wash out after a few weeks.
But short-term forecasts need to take account
of recent temperatures and recent residuals
due to serial correlation.
Short-term temperature forecasts are available.
Building a separate model for nt is possible, but
there is a simpler approach.
Forecasting electricity demand distributions
Short term forecasts
43
Short term forecasts
qt,p = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
Bootstrapping temperatures and residuals is ok
for long-term forecasts because short-term
dynamics wash out after a few weeks.
But short-term forecasts need to take account
of recent temperatures and recent residuals
due to serial correlation.
Short-term temperature forecasts are available.
Building a separate model for nt is possible, but
there is a simpler approach.
Forecasting electricity demand distributions
Short term forecasts
43
Short term forecasts
qt,p = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
Bootstrapping temperatures and residuals is ok
for long-term forecasts because short-term
dynamics wash out after a few weeks.
But short-term forecasts need to take account
of recent temperatures and recent residuals
due to serial correlation.
Short-term temperature forecasts are available.
Building a separate model for nt is possible, but
there is a simpler approach.
Forecasting electricity demand distributions
Short term forecasts
43
Short term forecasts
qt,p = hp (t ) + fp (w1,t , w2,t ) +
J
X
cj zj,t + nt
j=1
Bootstrapping temperatures and residuals is ok
for long-term forecasts because short-term
dynamics wash out after a few weeks.
But short-term forecasts need to take account
of recent temperatures and recent residuals
due to serial correlation.
Short-term temperature forecasts are available.
Building a separate model for nt is possible, but
there is a simpler approach.
Forecasting electricity demand distributions
Short term forecasts
43
Short-term forecasting model
log(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) +
J
X
cj zj,t + nt
j=1
yt,p denotes per capita demand (minus offset) at time t
(measured in half-hourly intervals) during period p,
p = 1, . . . , 48;
hp (t ) models all calendar effects;
fp (w1,t , w2,t ) models all temperature effects where w1,t is
a vector of recent temperatures at location 1 and w2,t is
a vector of recent temperatures at location 2;
zj,t is a demographic or economic variable at time t
nt denotes the model error at time t
y t = [yt , yt−1 , yt−2 , . . . ]
ap (y t−1 ) models effects of recent demands.
Forecasting electricity demand distributions
Short term forecasts
44
Short-term forecasting model
log(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) +
J
X
cj zj,t + nt
j=1
yt,p denotes per capita demand (minus offset) at time t
(measured in half-hourly intervals) during period p,
p = 1, . . . , 48;
hp (t ) models all calendar effects;
fp (w1,t , w2,t ) models all temperature effects where w1,t is
a vector of recent temperatures at location 1 and w2,t is
a vector of recent temperatures at location 2;
zj,t is a demographic or economic variable at time t
nt denotes the model error at time t
y t = [yt , yt−1 , yt−2 , . . . ]
ap (y t−1 ) models effects of recent demands.
Forecasting electricity demand distributions
Short term forecasts
44
Short-term forecasting model
log(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) +
J
X
cj zj,t + nt
j=1
yt,p denotes per capita demand (minus offset) at time t
(measured in half-hourly intervals) during period p,
p = 1, . . . , 48;
hp (t ) models all calendar effects;
fp (w1,t , w2,t ) models all temperature effects where w1,t is
a vector of recent temperatures at location 1 and w2,t is
a vector of recent temperatures at location 2;
zj,t is a demographic or economic variable at time t
nt denotes the model error at time t
y t = [yt , yt−1 , yt−2 , . . . ]
ap (y t−1 ) models effects of recent demands.
Forecasting electricity demand distributions
Short term forecasts
44
Short-term forecasting model
log(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) +
J
X
cj zj,t + nt
j=1
yt,p denotes per capita demand (minus offset) at time t
(measured in half-hourly intervals) during period p,
p = 1, . . . , 48;
hp (t ) models all calendar effects;
fp (w1,t , w2,t ) models all temperature effects where w1,t is
a vector of recent temperatures at location 1 and w2,t is
a vector of recent temperatures at location 2;
zj,t is a demographic or economic variable at time t
nt denotes the model error at time t
y t = [yt , yt−1 , yt−2 , . . . ]
ap (y t−1 ) models effects of recent demands.
Forecasting electricity demand distributions
Short term forecasts
44
Short-term forecasting model
log(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) +
J
X
cj zj,t + nt
j=1
yt,p denotes per capita demand (minus offset) at time t
(measured in half-hourly intervals) during period p,
p = 1, . . . , 48;
hp (t ) models all calendar effects;
fp (w1,t , w2,t ) models all temperature effects where w1,t is
a vector of recent temperatures at location 1 and w2,t is
a vector of recent temperatures at location 2;
zj,t is a demographic or economic variable at time t
nt denotes the model error at time t
y t = [yt , yt−1 , yt−2 , . . . ]
ap (y t−1 ) models effects of recent demands.
Forecasting electricity demand distributions
Short term forecasts
44
Short-term forecasting model
log(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) +
J
X
cj zj,t + nt
j=1
yt,p denotes per capita demand (minus offset) at time t
(measured in half-hourly intervals) during period p,
p = 1, . . . , 48;
hp (t ) models all calendar effects;
fp (w1,t , w2,t ) models all temperature effects where w1,t is
a vector of recent temperatures at location 1 and w2,t is
a vector of recent temperatures at location 2;
zj,t is a demographic or economic variable at time t
nt denotes the model error at time t
y t = [yt , yt−1 , yt−2 , . . . ]
ap (y t−1 ) models effects of recent demands.
Forecasting electricity demand distributions
Short term forecasts
44
Short-term forecasting model
log(yt,p ) = hp (t ) + fp (w1,t , w2,t ) + ap (y t−1 ) +
J
X
cj zj,t + nt
j=1
yt,p denotes per capita demand (minus offset) at time t
(measured in half-hourly intervals) during period p,
p = 1, . . . , 48;
hp (t ) models all calendar effects;
fp (w1,t , w2,t ) models all temperature effects where w1,t is
a vector of recent temperatures at location 1 and w2,t is
a vector of recent temperatures at location 2;
zj,t is a demographic or economic variable at time t
nt denotes the model error at time t
y t = [yt , yt−1 , yt−2 , . . . ]
ap (y t−1 ) models effects of recent demands.
Forecasting electricity demand distributions
Short term forecasts
44
Short-term forecasting model
ap (y t−1 ) =
n
X
bk,p (yt−k ) +
k =1
m
X
Bj,p (yt−48j )
j =1
+ Qp (yt ) + Rp (yt− ) + Sp (ȳt )
+
where
yt+ is maximum of yt values in past 24 hours;
yt− is minimum of yt values in past 24 hours;
ȳt is average demand in past 7 days
bk,p , Bj,p , Qp , Rp and Sp are estimated using
cubic splines.
Forecasting electricity demand distributions
Short term forecasts
45
Short-term forecasting model
ap (y t−1 ) =
n
X
bk,p (yt−k ) +
k =1
m
X
Bj,p (yt−48j )
j =1
+ Qp (yt ) + Rp (yt− ) + Sp (ȳt )
+
where
yt+ is maximum of yt values in past 24 hours;
yt− is minimum of yt values in past 24 hours;
ȳt is average demand in past 7 days
bk,p , Bj,p , Qp , Rp and Sp are estimated using
cubic splines.
Forecasting electricity demand distributions
Short term forecasts
45
Short-term forecasting model
ap (y t−1 ) =
n
X
bk,p (yt−k ) +
k =1
m
X
Bj,p (yt−48j )
j =1
+ Qp (yt ) + Rp (yt− ) + Sp (ȳt )
+
where
yt+ is maximum of yt values in past 24 hours;
yt− is minimum of yt values in past 24 hours;
ȳt is average demand in past 7 days
bk,p , Bj,p , Qp , Rp and Sp are estimated using
cubic splines.
Forecasting electricity demand distributions
Short term forecasts
45
Short-term forecasting model
ap (y t−1 ) =
n
X
bk,p (yt−k ) +
k =1
m
X
Bj,p (yt−48j )
j =1
+ Qp (yt ) + Rp (yt− ) + Sp (ȳt )
+
where
yt+ is maximum of yt values in past 24 hours;
yt− is minimum of yt values in past 24 hours;
ȳt is average demand in past 7 days
bk,p , Bj,p , Qp , Rp and Sp are estimated using
cubic splines.
Forecasting electricity demand distributions
Short term forecasts
45
Weakest assumptions
Temperature effects are independent of day of
week effects.
Historical demand response to temperature will
continue into the future.
Climate change will have only a small additive
increase in temperature levels.
Locally generated electricity (e.g., PV
generation) is not captured in demand data.
Forecasting electricity demand distributions
Short term forecasts
46
Weakest assumptions
Temperature effects are independent of day of
week effects.
Historical demand response to temperature will
continue into the future.
Climate change will have only a small additive
increase in temperature levels.
Locally generated electricity (e.g., PV
generation) is not captured in demand data.
Forecasting electricity demand distributions
Short term forecasts
46
Weakest assumptions
Temperature effects are independent of day of
week effects.
Historical demand response to temperature will
continue into the future.
Climate change will have only a small additive
increase in temperature levels.
Locally generated electricity (e.g., PV
generation) is not captured in demand data.
Forecasting electricity demand distributions
Short term forecasts
46
Weakest assumptions
Temperature effects are independent of day of
week effects.
Historical demand response to temperature will
continue into the future.
Climate change will have only a small additive
increase in temperature levels.
Locally generated electricity (e.g., PV
generation) is not captured in demand data.
Forecasting electricity demand distributions
Short term forecasts
46
Kaggle competition
Forecasting electricity demand distributions
Short term forecasts
47
Outline
1
The problem
2
The model
3
Long-term forecasts
4
Short term forecasts
5
References and R implementation
Forecasting electricity demand distributions
References and R implementation
48
References and R implementation
Main papers
å Hyndman, R.J. and Fan, S. (2010) “Density forecasting for
long-term peak electricity demand”, IEEE Transactions
on Power Systems, 25(2), 1142–1153.
å Fan, S. and Hyndman, R.J. (2012) “Short-term load
forecasting based on a semi-parametric additive model”.
IEEE Transactions on Power Systems, 27(1), 134–141.
R package
We have an R package that implements all
methods, but it is not yet publicly available.
Forecasting electricity demand distributions
References and R implementation
49
References and R implementation
Main papers
å Hyndman, R.J. and Fan, S. (2010) “Density forecasting for
long-term peak electricity demand”, IEEE Transactions
on Power Systems, 25(2), 1142–1153.
å Fan, S. and Hyndman, R.J. (2012) “Short-term load
forecasting based on a semi-parametric additive model”.
IEEE Transactions on Power Systems, 27(1), 134–141.
R package
We have an R package that implements all
methods, but it is not yet publicly available.
Forecasting electricity demand distributions
References and R implementation
49