Challenges in forecasting
peak electricity demand
Part 1
Rob J Hyndman
Challenges in forecasting peak electricity demand
1
Outline
1 The problem
2 The model
3 Forecasts
4 Challenges and extensions
5 References
Challenges in forecasting peak electricity demand
2
The problem
We want to forecast the peak electricity
demand in a half-hour period in twenty years
time.
We have fifteen years of half-hourly electricity
data, temperature data and some economic
and demographic data.
The location is South Australia: home to the
most volatile electricity demand in the world.
Sounds impossible?
Challenges in forecasting peak electricity demand
The problem
3
South Australian demand data
Black Saturday →
Challenges in forecasting peak electricity demand
The problem
4
South Australian demand data
3.0
2.5
2.0
1.5
South Australia state wide demand (GW)
3.5
South Australia state wide demand (summer 10/11)
Oct 10
Nov 10
Dec 10
Jan 11
Challenges in forecasting peak electricity demand
Feb 11
Mar 11
The problem
5
South Australian demand data
3.0
2.5
2.0
1.5
South Australian demand (GW)
3.5
South Australia state wide demand (January 2011)
1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31
Date in January
Challenges in forecasting peak electricity demand
The problem
6
Predictors
calendar effects
prevailing and recent weather conditions
climate changes
economic and demographic changes
changing technology
Modelling framework
Semi-parametric additive models with
correlated errors.
Each half-hour period modelled separately for
each season.
Variables selected to provide best
out-of-sample predictions using cross-validation
on each summer.
Challenges in forecasting peak electricity demand
The model
7
Monash Electricity Forecasting Model
yt = ȳi × yt∗
yt denotes per capita demand (minus offset) at
time t (measured in half-hourly intervals);
ȳi is the average demand for year i where t is in
year i.
yt∗ is the standardized demand for time t.
log(yt ) = log(ȳi ) + log(yt∗ )
log(ȳi ) = f (GSP, price, HDD, CDD) + εi
log(yt∗ ) = f (calendar effects, temperatures) + et
Challenges in forecasting peak electricity demand
The model
8
Monash Electricity Forecasting Model
Challenges in forecasting peak electricity demand
The model
9
Annual model
log(yt ) = log(ȳi ) + log(yt∗ )
log(ȳi ) = f (GSP, price, HDD, CDD) + εi
log(yt∗ ) = f (calendar effects, temperatures) + et
log(ȳi ) = log(ȳi−1 ) +
X
j
cj (zj,i − zj,i−1 ) + εi
First differences modelled to avoid
non-stationary variables.
Predictors: Per-capita GSP, Price, Summer CDD,
Winter HDD.
zCDD =
X
max(0, T̄ − 18.5)
T̄ = daily mean
summer
Challenges in forecasting peak electricity demand
The model
10
Annual model
Variable
∆gsp.pc
∆price
∆scdd
∆whdd
Coefficient
2.02
−1.67
1.11
2.07
Std. Error
5.05
0.68
0.25
0.33
t value
0.38
−2.46
4.49
0.63
P value
0.711
0.026
0.000
0.537
GSP needed to stay in the model to allow
scenario forecasting.
All other variables led to improved AICC .
Challenges in forecasting peak electricity demand
The model
11
Actual
Fitted
1.4
1.3
1.2
1.1
1.0
Annual demand
1.5
1.6
1.7
Annual model
89/90
91/92
93/94
95/96
97/98
99/00
01/02
03/04
05/06
07/08
09/10
Year
Challenges in forecasting peak electricity demand
The model
12
Monash Electricity Forecasting Model
log(yt ) = log(ȳi ) + log(yt∗ )
log(ȳi ) = f (GSP, price, HDD, CDD) + εi
log(yt∗ ) = f (calendar effects, temperatures) + et
Calendar effects
“Time of summer” effect (a regression spline)
Day of week factor (7 levels)
Public holiday factor (4 levels)
New Year’s Eve factor (2 levels)
Challenges in forecasting peak electricity demand
The model
13
Fitted results (Summer 3pm)
50
100
150
0.4
Mon
Tue
Wed
Thu
Fri
Sat
Sun
0.0
0.4
Day of week
−0.4
Effect on demand
Day of summer
0.0
Effect on demand
0
−0.4
0.4
0.0
−0.4
Effect on demand
Time: 3:00 pm
Normal
Day before
Holiday
Day after
Holiday
Challenges in forecasting peak electricity demand
The model
14
Monash Electricity Forecasting Model
log(yt ) = log(ȳi ) + log(yt∗ )
log(ȳi ) = f (GSP, price, HDD, CDD) + εi
log(yt∗ ) = f (calendar effects, temperatures) + et
Temperature effects
Ave temp across two sites, plus lags for
previous 3 hours and previous 3 days.
Temp difference between two sites, plus lags
for previous 3 hours and previous 3 days.
Max ave temp in past 24 hours.
Min ave temp in past 24 hours.
Ave temp in past seven days.
Each function is smooth & estimated using regression splines.
Challenges in forecasting peak electricity demand
The model
15
Fitted results (Summer 3pm)
40
30
40
Lag 1 day temperature
0.4
0.2
0.0
Effect on demand
20
30
40
10
30
40
0.4
0.2
0.0
Effect on demand
−0.4
Effect on demand
20
Lag 3 temperature
0.4
Lag 2 temperature
−0.4
10 15 20 25 30
Last week average temp
−0.2
−0.4
10
0.4
0.2
0.0
Effect on demand
−0.4
20
0.2
0.4
30
−0.2
0.4
0.2
−0.2
−0.4
10
0.0
Effect on demand
20
Lag 1 temperature
0.0
Effect on demand
−0.4
10
−0.2
40
0.2
30
0.0
20
Temperature
−0.2
10
−0.2
0.4
0.2
0.0
Effect on demand
−0.2
−0.4
−0.4
−0.2
0.0
Effect on demand
0.2
0.4
Time: 3:00 pm
15
25
35
Previous max temp
Challenges in forecasting peak electricity demand
10
15
20
25
Previous min temp
The model
16
Half-hourly models
log(yt∗ ) = f (calendar effects, temperatures) + et
Separate model for each half-hour.
Same predictors used for all models.
Predictors chosen by cross-validation on
summer of 2007/2008 and 2009/2010.
Each model is fitted to the data twice, first
excluding the summer of 2009/2010 and then
excluding the summer of 2010/2011. The
average out-of-sample MSE is calculated from
the omitted data for the time periods
12noon–8.30pm.
Gradient boosting used to reduce variance.
Challenges in forecasting peak electricity demand
The model
17
Half-hourly models
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
x x1 x2 x3 x4 x5 x6 x48 x96 x144 x192 x240 x288 d d1 d2 d3 d4 d5 d6 d48 d96 d144 d192 d240 d288 x+ x− x̄ dow hol dos MSE
• • • • • • • • •
•
•
• • • • • • • • • •
•
•
•
• • • • • • • 1.037
• • • • • • • • •
•
•
• • • • • • • • • •
•
•
•
• • • • • • 1.034
• • • • • • • • •
•
•
• • • • • • • • • •
•
•
• • • • • • 1.031
• • • • • • • • •
•
•
• • • • • • • • • •
•
• • • • • • 1.027
• • • • • • • • •
•
•
• • • • • • • • • •
• • • • • • 1.025
• • • • • • • • •
•
•
• • • • • • • • •
• • • • • • 1.020
• • • • • • • • •
•
•
• • • • • • • •
• • • • • • 1.025
• • • • • • • • •
•
•
• • • • • • •
•
• • • • • • 1.026
• • • • • • • • •
•
•
• • • • • •
•
• • • • • • 1.035
• • • • • • • • •
•
•
• • • • •
•
• • • • • • 1.044
• • • • • • • • •
•
•
• • • •
•
• • • • • • 1.057
• • • • • • • • •
•
•
• • •
•
• • • • • • 1.076
• • • • • • • • •
•
•
• •
•
• • • • • • 1.102
• • • • • • • • •
•
•
• • • • • • • •
• • • • • • 1.018
• • • • • • • • •
•
• • • • • • • •
• • • • • • 1.021
• • • • • • • • •
• • • • • • • •
• • • • • • 1.037
• • • • • • • •
• • • • • • • •
• • • • • • 1.074
• • • • • • •
• • • • • • • •
• • • • • • 1.152
• • • • • •
• • • • • • • •
• • • • • • 1.180
• • • • •
• • •
•
•
• • • • • • • •
• • • • • • 1.021
• • • •
• • •
•
•
• • • • • • • •
• • • • • • 1.027
• • •
• • •
•
•
• • • • • • • •
• • • • • • 1.038
• •
• • •
•
•
• • • • • • • •
• • • • • • 1.056
•
• • •
•
•
• • • • • • • •
• • • • • • 1.086
• • •
•
•
• • • • • • • •
• • • • • • 1.135
• • • • • • • • •
•
•
• • • • • • • •
• • • • • 1.009
• • • • • • • • •
•
•
• • • • • • • •
•
• • • • 1.063
• • • • • • • • •
•
•
• • • • • • • •
• •
• • • 1.028
• • • • • • • • •
•
•
• • • • • • • •
• • •
• • 3.523
• • • • • • • • •
•
•
• • • • • • • •
• • • •
• 2.143
• • • • • • • • •
•
•
• • • • • • • •
• • • • •
1.523
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Challenges in forecasting peak electricity demand
The model
18
Half-hourly models
80
60
70
R−squared (%)
90
R−squared
12 midnight 3:00 am
6:00 am
9:00 am
12 noon
3:00 pm
6:00 pm
9:00 pm 12 midnight
Time of day
Challenges in forecasting peak electricity demand
The model
19
Half-hourly models
1.5
2.0
2.5
3.0
3.5
Actual
Fitted
1.0
South Australian demand (GW)
4.0
South Australian demand (January 2011)
1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31
Date in January
Temperatures (January The
2011)
model
45
Challenges in forecasting peak electricity demand
20
Peak demand forecasting
35
log(yt ) = log(ȳi ) + log(yt∗ )
30
log(ȳi ) = f (GSP, price, HDD, CDD) + εi
log(yt∗ ) = f (calendar effects, temperatures) + et
25
Temperature (deg C)
40
Kent Town
Airport
10
15
20
Multiple alternative futures created:
Calendar effects known;
Future temperatures simulated
(taking account of climate change);
1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31
Assumed values for GSP, population and price;
Date in January
Residuals simulated
Challenges in forecasting peak electricity demand
Forecasts
21
Peak demand backcasting
log(yt ) = log(ȳi ) + log(yt∗ )
log(ȳi ) = f (GSP, price, HDD, CDD) + εi
log(yt∗ ) = f (calendar effects, temperatures) + et
Multiple alternative pasts created:
Calendar effects known;
Past temperatures simulated;
Actual values for GSP, population and price;
Residuals simulated
Challenges in forecasting peak electricity demand
Forecasts
22
Peak demand backcasting
4.0
PoE (annual interpretation)
3.5
10 %
50 %
90 %
●
●
●
3.0
PoE Demand
●
●
●
●
●
●
2.5
●
●
●
●
2.0
●
98/99
00/01
02/03
04/05
06/07
08/09
10/11
Year
Challenges in forecasting peak electricity demand
Forecasts
23
120
South Australia GSP
60
80
100
High
Base
Low
40
billion dollars (08/09 dollars)
Peak demand forecasting
1990
1995
2000
2005
2010
2015
2020
2015
2020
2015
2020
Year
2.0
South Australia population
1.4
1.6
million
1.8
High
Base
Low
1990
1995
2000
2005
2010
Year
Average electricity prices
18
16
12
14
c/kWh
20
22
High
Base
Low
1990
1995
2000
2005
Year
2010
200
MW
300
400
Challenges in forecasting peak electricity demand
Major industrial offset demand
High
Base
Low
Forecasts
24
Peak demand distribution
5
6
Annual POE levels
4
●
●
3
PoE Demand
●
1 % POE
5 % POE
10 % POE
50 % POE
90 % POE
Actual annual maximum
●
●
●
●
●
●
●
●
●
2
●
●
98/99 00/01 02/03 04/05 06/07 08/09 10/11 12/13 14/15 16/17 18/19 20/21
Year
Challenges in forecasting peak electricity demand
Forecasts
25
Challenges
Weakest assumptions
Temperature effects independent of day of week.
Historical demand response to temperature will
continue into the future.
Climate change will have only a small additive
increase in temperature levels.
Further improvements
We have a separate model for PV generation
based on solar radiation and temperatures.
Our annual model is now quarterly.
Our quarterly model is adjusted for autocorrelation.
Challenges in forecasting peak electricity demand
Challenges and extensions
26
Implementation
Our model is used for long-term forecasting in:
Victoria’s Vision 2030 energy plan;
all regions of the National Energy Market;
South Western Interconnected System
(WA);
some local distributors.
It is also used for short-term
forecasting comparisons in:
all regions of the
National Energy Market.
Challenges in forecasting peak electricity demand
Challenges and extensions
27
References
Main papers
å Hyndman, R.J. and Fan, S. (2010) “Density forecasting for
long-term peak electricity demand”, IEEE Transactions
on Power Systems, 25(2), 1142–1153.
å Fan, S. and Hyndman, R.J. (2012) “Short-term load
forecasting based on a semi-parametric additive model”.
IEEE Transactions on Power Systems, 27(1), 134–141.
å Ben Taieb, S. and Hyndman, R.J. (2014) “A gradient
boosting approach to the Kaggle load forecasting
competition”, International Journal of Forecasting, 30(2),
382–394.
Challenges in forecasting peak electricity demand
References
28