Stochastic Optimization in Energy Systems
DIMACS Workshop on Algorithmic Decision Theory
Rutgers University
October 27, 2010
Warren Powell
CASTLE Laboratory
Princeton University
http://www.castlelab.princeton.edu
© 2010 Warren B. Powell, Princeton University
Slide 1
Lecture outline





The problem of uncertainty
Modeling stochastic optimization problems
Energy storage portfolios
The unit commitment problem for PJM
Long-term energy resource planning
© 2010 Warren B. Powell
Slide 2
Challenges in models and algorithms

Strategic questions in energy policy and economics
» How do we design market mechanisms to control green
house gases?
» How do we design policies to achieve energy goals (e.g.
30% renewables by 2030) with a given probability?
» How does the imposition of a carbon tax or investment tax
credit change the likelihood of meeting this goal?
» We need models of competitive equilibria and cooperative
games in the presence of uncertainties about technology,
policy and climate change.
» How do we quantify the risks associated with the pairing
of energy from wind with dispatchable energy from
hydroelectric facilities in the presence of climate change?
Challenges in models and algorithms

Design and control of energy systems
» How do we balance storage of energy from the grid in
electric vehicles in the presence of variability of energy
from wind, volatile prices, and uncertainty in usage
patterns for vehicles?
» How do we control the storage of energy across a
portfolio of energy storage devices?
» How should a utility work with urban building managers
to balance energy consumption from the grid, storage
devices and backup generators?
» How can grid operators modify their unit commitment
models to capture significant contributions of energy from
wind and solar?
» How can companies design efficient verification policies
for forests being used as carbon offsets?
Challenges in models and algorithms

All of these problems can be formulated as some
type of stochastic optimization problem.

in
st ra
a
c
e
For
For
dy
.3
e
ga m
dule
Sche
Cancel
game
$2300
-$200
6
y.
nn
we
ath
er
lou
-$1400
-$200
su
Use
st c
me
le ga
u
d
e
Sch
Canc
el g am
e
.1
ast
rep
ort
eca
rec
Fo
Simple problems can
be solved as decision
trees…
 Most problems are
hard.
se rt
t u po
no er re
Do eath
w
e
gam
dule
e
h
c
S
Cancel
game
$3500
-$200
e
gam
dule
Sche
Cancel
game
$2400
-$200
Decision making under uncertainty

Mixing optimization and uncertainty
Energy
sources
Time
?
?
?
?
?
“large optimization model (e.g. NEMS, MARKAL, …)”
Decision making under uncertainty

Mixing optimization and uncertainty
Energy
sources
Time
12.334 Solar
No
6.2
142
Scenario 1
Decision making under uncertainty

Mixing optimization and uncertainty
Energy
sources
Time
18.917 Wind
No
3.6
89.1
Scenario 2
Decision making under uncertainty

Mixing optimization and uncertainty
Energy
sources
Time
22.314 Solar
Yes
5.9
117
Scenario 3
Decision making under uncertainty
22.314 Solar
Yes
18.917 Wind
5.9
No
12.334 Solar
89.1
3.6 No
142
6.2
117
Scenario 3
Scenario 2
Scenario 1
Now we have to combine the results of these three optimizations to make decisions.
Decision making under uncertainty
Don’t gamble; take all your
savings and buy some good stock
and hold it till it goes up, then
sell it. If it don’t go up, don’t
buy it.
Will Rogers
It is not enough to mix “optimization” (intelligent decision
making) and uncertainty. You have to be sure that each decision
has access to the information available at the time.
Decision making under uncertainty

Socrates – used by Pacific, Gas and Electric
Wetter
Known
15% exceedence
Unknown
50% exceedence
0.3
85% exceedence
0.4 probability (weight)
15% exceedence
0.3
50% exceedence
Now
Jan
Average over 30
Flow forecast scenarios
Feb
Mar
Drier
Apr
85% exceedence
May…
Sep…
Stochastic programming does not “cheat”, but it does not scale. It is best
designed for coarse-grained sources of uncertainty, but will not handle finegrained temporal resolution.
Jan
Lecture outline





The problem of uncertainty
Modeling stochastic optimization problems
Energy storage portfolios
The unit commitment problem for PJM
Long-term energy resource planning
© 2010 Warren B. Powell
Slide 13
Modeling stochastic optimization problems

Attribute vectors:
a
Commodity type  Type 

 

Location

  Location 
 Age 
 Location 
 Dam type 


 Water level 


Generators


Slide 14
Modeling stochastic optimization problems

Modeling resources:
» The attributes of a single resource:
a  The attributes of a single resource
a A The attribute space
» The resource state vector:
Rta  The number of resources with attribute a
 
Rt  Rta
aA
The resource state vector
» The information process:
Rˆta  The change in the number of resources with
attribute a.
Slide 15
Modeling stochastic optimization problems

The system state:









St   Rt , Dt , t   System state, where:
Rt  Resource state (how much capacity, reserves)
Dt  Market demands
t  "system parameters"
State of the technology (costs, performance)
Climate, weather (temperature, rainfall, wind)
Government policies (tax rebates on solar panels)
Market prices (oil, coal)
Slide 16
Modeling stochastic optimization problems

Making decisions:









xt  A decision to buy, sell, move, repair, price,
or control
X  ( St )  Decision function (or "policy"); maps states to
decisions (also known as actions or controls).
We generally have a family of policies/functions
so we write   
Slide 17
Modeling stochastic optimization problems

Exogenous information:










Wt  New information = Rˆt , Dˆ t , ˆt

Rˆt  Exogenous changes in capacity, reserves
Dˆ t  New demands for energy from each source
ˆt  Exogenous changes in parameters.
Slide 18
Modeling stochastic optimization problems

The transition function









St 1  S (St , xt ,Wt 1 )
M
Known as the:
“Transition function”
“Transfer function”
“State transition model”
“System model”
“Plant model”
“Model”
Slide 19
Modeling stochastic optimization problems
Resources
Demands
Slide 20
Modeling stochastic optimization problems
t
t+1
t+2
Slide 21
Modeling stochastic optimization problems
Optimizing over time
t
t+1
t+2
Optimizing at a point in time
Slide 22
Modeling stochastic optimization problems

The objective function


t

max E  C  St , X ( St )  
 t


Expectation over
allState variable
Contribution
function Decision function (policy)
Finding the best policy
random outcomes
Modeling stochastic optimization problems

Policies
» 1) Myopic policies
• Take the action that maximizes contribution (or minimizes
cost) for just the current time period:
X M (St )  argmax xt C(St , xt )
» 2) Lookahead policies
• Plan over the next T periods, but implement only the action it
tells you to do now.
X ( St )  arg max xt , xt 1 ,..., xt T
M
T
C(S , x )
t ' t
t'
t'
Modeling stochastic optimization problems

Policies (cont’d)
» 3) Policies based on value function approximations
Let Vt ( St ) be an approximation of the value of being in state St
X M ( St )  arg max xt  C ( St , xt )   EVt 1 ( St 1 ) 
» 4) Policy function approximations
Let X ( St ) be a function that directly tells you an action
given that you are in a state St .
Modeling stochastic optimization problems

Ways of approximating functions (policies or
value functions)
» 1) Lookup tables
• When in a (discrete) state, returns an action
• When in a (discrete) state, returns the value of being in that
state.
• There is one value (parameter) to determine for each state.
» 2) Parametric models
• A closed form, analytic function determined by one or more
parameters.
 1 (sell) If St =pt  
M
X ( St |  )  
0 (hold) Otherwise
Vt ( St )   0  1St   2 ( St ) 2
Modeling stochastic optimization problems

Ways of approximating functions (policies or
value functions)
» 3) Nonparametric models
n
V n ( s) 
i
i
ˆ
v
k
(
s
,
S
 h )
i 1
n
 k ( s, S )
i
i 1
h
i 2

(
s

S
) 
kh ( s, S i )  exp  

h


Nonparametric methods approximate a function by using a
weighted sum of observations, where the weight declines with
the distance between the observed state and the state we are
trying to estimate.
Lecture outline





The problem of uncertainty
Modeling stochastic optimization problems
Energy storage portfolios
The unit commitment problem for PJM
Long-term energy resource planning
© 2010 Warren B. Powell
Slide 28
Energy storage portfolios

Premise
» The optimal mix of energy resources depends on the
entire portfolio.
» The performance of any single source of energy
(including storage) depends on the marginal difference
between demand and all other forms of energy in the
portfolio. This is what determines the residual demand.
» It is not just what is happening (the physical process)
but also how well you could predict what was going to
happen (the information process).
© 2010 Warren B. Powell
Slide 29
Energy storage portfolios

Wind
» Varies with multiple
freqeuencies (seconds,
hours, days, seasonal).
» Spatially uneven, generally
not aligned with population
centers.
 Solar
» Shines primarily during the
day (when it is needed), but
not entirely reliably.
» Strongest in
south/southwest.
© 2010 Warren B. Powell
Slide 30
Energy storage portfolios
30 days
1 year
© 2010 Warren B. Powell
Slide 31
Energy storage portfolios
Hydroelectric
Flywheels
Ultracapacitors
Batteries
© 2010 Warren B. Powell
Slide 32
Energy storage portfolios

Storage technologies
Name
Charge rate Discharge rate
Power rating
Capacity
Loss
Efficiency
Batteries
sodium-sulfur 5 hours
5 hours
10 MW
20 Wh/kg < lithium
80-90%
lead-acid
10 hours
10 mins.-hour
100 kW
20 Wh/kg 10%/month
75%
lithium
1 hour
1 hour
10-100 kW
150 Wh/kg10%/month
90-95%
Ultracapacitor
seconds
few seconds
10-100 kW
5 Wh/kg
Flow battery
Pumped hydro storage
(PHS)
5 hours
5 hours
hours
Flywheel
SMES (supercond. mag. )
5%/day
95-100%
200 kW-1MW
minimal
70-78%?
10 hours
1 GW
evaporation 75%-80%
minutes
minutes
10kW-10MW
10 Wh/kg frictional
95%
seconds
seconds
10-100 MW
.2 Wh/kg
large
95%
above ground hours
2 hours
15 MW
minimal
48%
below ground hours
10 hours
100-300MW
minimal
48%
Comp. air energy storage (CAES)
» Storage technologies different in terms of:
•
•
•
•
•
Charge rates
Discharge rates
Power rating
Capacity
Loss
© 2010 Warren B. Powell
Slide 33
Optimal control of wind and storage

Controlling the storage process
» Imagine that we would like to use storage to reduce
demand when electricity prices are high.
» We use a simple policy controlled by two parameters.
Price
 Withdraw
 Store
Energy storage portfolios

We now have to optimize over policies
» This means finding the best values for   withdraw ,  store 
» The policy  refers to the structure of the decision
function, and the parameters   withdraw ,  store 
» This means we have to solve
T
max E  tC ( St , X  ( St ))

t 0
» We can write
T
F (  withdraw ,  store |  )   tC ( St ( ), X  ( St ()))
t 0
© 2010 Warren B. Powell
Slide 35
Optimizing storage policy
 Store
 Withdraw
36
Optimizing storage policy

Initially we think the concentration is the same everywhere:
Estimated contribution
Knowledge gradient
» We want to measure the value where the knowledge gradient is the
highest. This is the measurement that teaches us the most.
Optimizing storage policy

After four measurements:
Estimated contribution
Measurement
Knowledge gradient
Value of another measurement
New optimum
at same location.
» Whenever we measure at a point, the value of another
measurement at the same point goes down. The knowledge
gradient guides us to measuring areas of high uncertainty.
Optimizing storage policy

After five measurements:
Estimated contribution
Knowledge gradient
Optimizing storage policy

After six samples
Estimated contribution
Knowledge gradient
Optimizing storage policy

After seven samples
Estimated contribution
Knowledge gradient
Optimizing storage policy

After eight samples
Estimated contribution
Knowledge gradient
Optimizing storage policy

After nine samples
Estimated contribution
Knowledge gradient
Optimizing storage policy

After ten samples
Estimated contribution
Knowledge gradient
Optimizing storage policy

After 10 measurements, our estimate of the surface:
Estimated contribution
True concentration
Optimizing storage policy

After 10 measurements, our estimate of the surface:
Estimated contribution
Knowledge gradient
Knowledge Gradient
Estimation of Annual Profit
1.26
0
-20
1.24
-40
-60
log(KG)
Dollars x 107
1.22
1.2
1.18
-80
-100
-120
-140
1.16
-160
1.14
-180
60
60
60
60
62
64
65
55
55
66
68
70
70
72
50
theta0
50
74
75
76
80
45
78
theta1
theta0
80
45
theta1
Lecture outline





The problem of uncertainty
Modeling stochastic optimization problems
Energy storage portfolios
The unit commitment problem for PJM
Long-term energy resource planning
© 2010 Warren B. Powell
Slide 47
The unit commitment problem

Designing energy portfolios….
» … is like building a stone wall, where energy comes
from sources which vary in terms of capital cost,
operating cost and flexibility.
© 2010 Warren B. Powell
Slide 48
The unit commitment problem

The day-ahead problem
» Determines which coal/nuclear/natural gas/…plants to
turn on/off and when.
» Will use wind/solar when available and if needed.
» Requires point forecast of wind/solar/demand.
24
min
x1 ,..., x24
C( x )
t
t 1
….subject to numerous constraints, including integrality.
» The problem is solved using two adjustment parameters
•  Fraction of generator capacity assumed (e.g. 93 percent)
• q Quantile of wind forecast assumed for advance
commitments.
The unit commitment problem

The hour-ahead problem
» Each hour, we can make modest adjustments. Plants
that are “on” can be adjusted up and down.
» Cannot turn coal plants on and off.
» Actual may differ from forecast:
• Wind may be higher or lower than forecast. If higher, may not
be able to use it because of inability to scale back other
sources.
• Demand may exceed forecast. Wind/solar may fall below
forecast. In this case, we find the least cost unit that can be
scaled up quickly enough.
The unit commitment problem

Modeling
» A deterministic model
24
min
x1 ,..., x24
C( x )
t 1
t
» Stochastic formulation – I
S
min
s
s
x1 ,..., x24
24
s
s
p
C
(
x

t )
s  "scenario"
s 1 t 1
• Ambiguous whether decisions are deterministic or stochastic.
• Day ahead decisions are deterministic, but hour ahead
decisions are stochastic.
The unit commitment problem

Modeling
» Stochastic formulation – II
24
min E  C ( xt ,t ' , yt ',t ' )
x t ,t '
 xt ,t ' t '1,...,24
y t ',t '
t '1
( yt ',t ' )t ' 1,...,24
• xt ,t ' is determined at time t, to be implemented at time t’
• y t ',t ' is determined at time t’, to be implemented at time t’+1
» Important to recognize information content
• At time t, xt ,t ' is deterministic.
• At time t, y t ',t ' is stochastic.
The unit commitment problem

The unit commitment problem
» Rolling forward with perfect forecast of actual wind, demand, …
hour 0-24
hour 25-48
hour 49-72
The unit commitment problem

The unit commitment problem
» Rolling forward with perfect forecast of actual wind, demand, …
hour 0-24
hour 25-48
hour 49-72
The unit commitment problem

The unit commitment problem
» Using forecast of future wind.
hour 0-24
hour 25-48
hour 49-72
The unit commitment problem

The unit commitment problem
» Using forecast of future wind.
hour 0-24
hour 25-48
hour 49-72
The unit commitment problem

The unit commitment problem
» Stepping through the next day.
hour 0-24



 xt ,t '  t ' 


The unit commitment problem

The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
yt ',t '
The unit commitment problem

The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
The unit commitment problem

The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
The unit commitment problem

The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
The unit commitment problem

The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
The unit commitment problem

The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
The unit commitment problem

The unit commitment problem
» Stepping forward observing actual wind, making small adjustments
hour 0-24
The unit commitment problem

Papers can be divided into four categories with respect to
the handling of information:
» The model is wrong, and the paper is implementing an incorrect
algorithm.
• “Incorrect” means cheating by using information from the future.
» The model is wrong (or imprecise), but the experimental work is
correct.
• Many people know how to do a proper simulation, but don’t know
how to model it.
» The model is correct, but the experimental work is wrong.
• The programmer cheated by using information from the future; the
modeler used proper mathematics that did not match the code.
» The model is correct, and the experimental work is correct.
The unit commitment problem

Safest way to write out the objective function


t

max E  C  St , X ( St )  
 t


» Our policy combines:
• Lookahead policy to solve unit commitment problem
• Myopic policy to solve hourly adjustment problem. Myopic
policy exploits tunable parameters (p,q) in the lookahead
policy.
• We can write the dependence on (p,q) using
F   S 0 | ( p , q )     t C  S t , X   S t | ( p, q )  
t
The unit commitment problem

The value of optimizing (p,q)
The unit commitment problem

Energy profile with 2 percent from wind
Feb 15
Feb 16
Feb 17
Feb 18
Feb 19
Day-ahead generation
5000
4000
3000
2000
1000
Real-time adjusted generation
0
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
5000
4000
3000
2000
1000
0
© 2010 Warren B. Powell
Slide 68
The unit commitment problem

Forecast from unit commitment model
Feb 18 Day-ahead
Feb 18 Real-time
5000
Real-time adjusted generation
Day-ahead scheduled generation
5000
4000
3000
2000
1000
1
1
6
12
Hour
18
24
4000
3000
2000
1000
1
1
© 2010 Warren B. Powell
6
12
Hour
18
24
Slide 69
The unit commitment problem

Energy profile with 20 percent from wind
Jan 8
Jan 9
Jan 10
Jan 11
Jan 12
Day-ahead generation
5000
4000
3000
2000
1000
Real-time adjusted generation
0
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
1
6
12 18
Hour
24
5000
4000
3000
2000
1000
0
© 2010 Warren B. Powell
Slide 70
The unit commitment problem

The effect of modeling uncertainty in wind
The unit commitment problem
With better wind forecasts
2009 Wind Data - Forecast using
2009 past
Windcorresponding
Data
hour of past 7 days
Wind Generation MW
3000
2009 Wind Data
0.3 quantile of corresponding hour from past 7 days
2500
2000
1500
1000
500
0
-500
1000
2000
3000
4000
5000
Hour
6000
7000
8000
2009 Wind Data - Forecast using past 7 hours
3000
Wind Generation MW

2009 Wind Data
0.3 quantile of previous 7 hours
2500
2000
1500
1000
500
0
-500
1000
2000
3000
4000
5000
Hour
© 2010 Warren B. Powell
6000
7000
8000
Slide 72
The unit commitment problem

With better wind forecasts:
($ millions)
Total Day-ahead
system cost
Total Real-time
system cost
Total Generator
dispatch cost
Total Generator
turndown savings
20% Wind – forecast with past 7 days at corresponding hour
Winter
69.240
79.584
11.109
0.766
Summer
83.886
90.222
7.606
1.269
20% Wind – forecast with past 7 hour
Winter
61.389
68.764
7.730
0.354
Summer
78.299
85.000
7.608
0.907
» Using a stochastic model that properly captures the
flow of information, we can quantify the value of better
forecasts.
© 2010 Warren B. Powell
Slide 73
Lecture outline





The problem of uncertainty
Modeling stochastic optimization problems
Energy storage portfolios
The unit commitment problem for PJM
Long-term energy resource planning
© 2010 Warren B. Powell
Slide 74
Goals for an energy policy model

Potential questions
» Policy questions
• How do we design policies to achieve energy goals (e.g. 20%
renewables by 2015) with a given probability?
• How does the imposition of a carbon tax change the likelihood
of meeting this goal?
• What might happen if ethanol subsidies are reduced or
eliminated?
• What is the impact of a breakthrough in batteries?
» Energy economics
• What is the best mix of energy generation technologies?
• How is the economic value of wind affected by the presence of
storage?
• What is the best mix of storage technologies?
• How would climate change impact our ability to use
hydroelectric reservoirs as a regulating source of power?
© 2010 Warren B. Powell
Slide 75
Intermittent energy sources
Wind speed
Solar energy
© 2010 Warren B. Powell
Slide 76
Long term uncertainties….
Tax policy
2010
2015
Solar panels
Batteries
Price of oil
2020
2025
Carbon capture and
sequestration
© 2010 Warren B. Powell
2030
Climate change
Slide 77
SMART-Stochastic, multiscale model

SMART: A Stochastic, Multiscale Allocation model for
energy Resources, Technology and policy
» Stochastic – able to handle different types of uncertainty:
• Fine-grained – Daily fluctuations in wind, solar, demand, prices, …
• Coarse-grained – Major climate variations, new government policies,
technology breakthroughs
» Multiscale – able to handle different levels of detail:
• Time scales – Hourly to yearly
• Spatial scales – Aggregate to fine-grained disaggregate
• Activities – Different types of demand patterns
» Decisions
• Hourly dispatch decisions
• Yearly investment decisions
• Takes as input parameters characterizing government policies,
performance of technologies, assumptions about climate
© 2010 Warren B. Powell
Slide 78
The annual investment problem
2008
New information 2009
New information
oil
oil oil ˆ oil ˆ oil
oil oil
ˆ
ˆ
ˆ
ˆ
R x R Dt t Rt 1 xt 1 Rt 1Dt 1 t 1
oil
t
oil oil
t
t
windˆ wind wind wind windˆ windˆ wind wind
ˆ
Rtwindxtwind
Rt Dt ˆt Rt 1 xt 1Rt 1 Dt 1 ˆt 1
R x Rˆ Dˆ ˆ
R x Rˆ Dˆ ˆ
nat gasnat gas
nat gasnat gasnat gas nat gasnat gas
nat gasnat gasnat gas
t
t
t
t 1 t 1 t
t
t
t 1 t 1
R x Rˆ Dˆ ˆ
coal coal coal coal coal
t
t
t
t
t
R x Rˆ Dˆ ˆ
coal coal coal coal coal
t 1 t 1 t 1 t 1 t 1
© 2010 Warren B. Powell
Slide 79
The hourly dispatch problem
Hourly electricity “dispatch” problem
© 2010 Warren B. Powell
Slide 80
The hourly dispatch problem

Hourly model
» Decisions at time t impact t+1 through the amount of water held in
the reservoir.
Hour t
Hour t+1
© 2010 Warren B. Powell
Slide 81
The hourly dispatch problem

Hourly model
» Decisions at time t impact t+1 through the amount of water held in
the reservoir.
Hour t
Value of holding water in the reservoir
for future time periods.
© 2010 Warren B. Powell
Slide 82
The hourly dispatch problem
© 2010 Warren B. Powell
Slide 83
The hourly dispatch problem
Hour
2008
1
2
3
4
© 2010 Warren B. Powell
8760
2009
1
2
Slide 84
The hourly dispatch problem
Hour
2008
1
2
3
4
© 2010 Warren B. Powell
8760
2009
1
2
Slide 85
SMART-Stochastic, multiscale model
2008
2009
© 2010 Warren B. Powell
Slide 86
SMART-Stochastic, multiscale model
2008
2009
oil
 2009
wind
 2009
nat gas
 2009
coal
 2009
© 2010 Warren B. Powell
Slide 87
SMART-Stochastic, multiscale model
2008
2009
2010
© 2010 Warren B. Powell
2011
2038
Slide 88
SMART-Stochastic, multiscale model
2008
2009
2010
© 2010 Warren B. Powell
2011
2038
Slide 89
SMART-Stochastic, multiscale model
2008
2010
2009
~5 seconds
~5 seconds
~5 seconds
© 2010 Warren B. Powell
2011
~5 seconds
2038
~5 seconds
Slide 90
Approximate dynamic programming
Step 1: Start with a pre-decision state Stn
Step 2: Solve the deterministic optimization using
Deterministic
an approximate value function:
optimization
n
n
n 1
M ,x
n
vˆt  max x  Ct ( St , xt )  Vt
(S
( St , xt )) 
to obtain xtn.
Step 3: Update the value function approximation
Vt n1 (Stx,1n )  (1  n1 )Vt n11 (Stx,1n )  n1vˆtn
Recursive
statistics
Step 4: Obtain Monte Carlo sample of Wt (n ) and
Simulation
compute the next pre-decision state:
Stn1  S M (Stn , xtn ,Wt 1 ( n ))
Step 5: Return to step 1.
© 2010 Warren B. Powell
Slide 91
SMART-Stochastic, multiscale model

Use statistical methods to learn the
value of resources in the future.
Resources may be:
Vt ( Rt )
» Stored energy
• Hydro
• Flywheel energy
• …
» Storage capacity
Value

• Batteries
• Flywheels
• Compressed air
» Energy transmission capacity
• Transmission lines
• Gas lines
• Shipping capacity
» Energy production sources
Amount of resource
• Wind mills
• Solar panels
• Nuclear power plants
© 2010 Warren B. Powell
Slide 92
Benchmarking on hourly dispatch

ADP objective function relative to optimal LP
2.50
Percentage error from optimal
2.50%
2.00%
2.00
1.50%
1.50
1.00%
1.00
0.50%
0.50
0.06% over optimal
0.00%
0.00
0
50
100
150
200
250
Iterations
300
© 2010 Warren B. Powell
350
400
450
Slide 93
500
Benchmarking on hourly dispatch

Optimal from linear program
Optimal from linear program
Reservoir level
Rainfall
Demand
© 2010 Warren B. Powell
Slide 94
Benchmarking on hourly dispatch

Approximate dynamic programming
ADP solution
Reservoir level
Rainfall
Demand
© 2010 Warren B. Powell
Slide 95
Benchmarking on hourly dispatch

Optimal from linear program
Optimal from linear program
Reservoir level
Rainfall
Demand
© 2010 Warren B. Powell
Slide 96
Benchmarking on hourly dispatch

Approximate dynamic programming
ADP solution
Reservoir level
Rainfall
Demand
© 2010 Warren B. Powell
Slide 97
Multidecade energy model

Optimal vs. ADP – daily model over 20 years
40.00%
35.00%
Percent over optimal
30.00%
25.00%
20.00%
15.00%
10.00%
0.24% over optimal
5.00%
0.00%
0
100
200
300
400
500
600
Iterations
© 2010 Warren B. Powell
Slide 98
Energy policy modeling

Traditional optimization models tend to produce
all-or-nothing solutions
Investment in IGCC
Traditional
optimization
IGCC is cheaper
Pulverized coal is cheaper
Approximate dynamic
programming
Cost differential: IGCC - Pulverized coal
© 2010 Warren B. Powell
Slide 99
Stochastic rainfall
700
600
Precipitation
Sample paths
500
400
300
200
100
0
0
100
200
300
400
500
600
700
800
Time period
© 2010 Warren B. Powell
Slide 100
Stochastic rainfall
9000
8000
ADP
Reservoir level
7000
Optimal for individual
scenarios
6000
5000
4000
3000
2000
1000
0
0
100
200
300
400
500
600
700
800
Time period
© 2010 Warren B. Powell
Slide 101