Optimal Electricity Supply
Bidding by Markov Decision
Process
Authors: Haili Song, Chen-Ching Liu, Jacques Lawarree, & Robert Dahlgren
Presentation Review By:
Feng Gao, Esteban Gil, & Kory Hedman
IE 513 Analysis of Stochastic Systems
Professor Sarah Ryan
February 14, 2005
Outline
 Introduction
 Purpose
 Problem Formulation
 Model Overview
 Summary
Introduction
 Electric Industry has Transitioned from
Regulated to Deregulated
 Regulated: Vertically Integrated, Monopolistic
Market
 Deregulated: Ideally, Perfect Competition
Market
 Decision Analysis: Based on Profit &
Competition
Introduction Cont’d
 Traditional Power System
 Generation, Transmission, Distribution,
Consumption
Introduction Cont’d
 Structure of
Power Market
 Optimize
Resources
With
Competition
Introduction Cont’d






Day Ahead Market Considered
Inelastic Demand
Generation Companies (GenCos) are Risk-Neutral
GenCos Bid a Price (P) & Quantity (Q)
Bids are Chosen from Cheapest to Most Expensive
Market Clearing Price: P from the Highest Chosen
Q
 (Similar to the New Zealand & Great Britain
Electricity Markets)
Purpose
 Overall Objective: Maximize Expected Profit over
a Planning Horizon of 7 Days
 For all States, Determine Optimal Bidding Strategy.
Depends on:




Competitors’ Bidding
Load Forecasting
Remaining Time Horizon (given day i, 7 – i)
Production Limit (Max Available Supply Remaining
over Planning Horizon)
 Accumulated Data over Time (Past Load & Price
Data)
Problem Formulation
 States are Defined by 7 Variables:
 Peak Load & Peak Price (2)
 Off-Peak Load & Off-Peak Price (2)
 Current Production Limit for the Remaining
Planning Horizon (1)
 Load Forecast for the Following Day (2)
 Aggregation Limits Number of States
 P & D Broken into High, Medium, & Low
 Illogical States Ignored: (High P & Low Q, etc.)
Problem Formulation Cont’d
 Transition Probability
Depends on:
 Current State i, Subsequent
State j, Decision a
 Pr (i, j, a)
 Decision Maker Receives
Reward
 R (i, j, a)
 State of the Market Defines
the Competitors’ Bids &
Decision Maker’s Bidding
Options
 Bid Prices are Determined
Using a Staircase Supply Fn
for Varying MW
Problem Formulation Cont’d
 MDP Algorithm
Considers:
 Rewards Based on Load
Forecast
 Decisions of a State
 Competitors’ Bidding
Characteristics
 Decision Options Affect
Transition Probabilities &
Rewards
 Competitors’ Bids are
Independent
 Scenarios (s) are exclusive
Model Overview
 Probability of Scenario s:
 Pr (i, n, k): Probability that
Supplier n (n ~= m) Chooses
Option k in State i
 m is decision maker
 Competitors’ Bids are Independent
 Remaining Production Limit:
 q (i, s, t) is the Q used in period t for
scenario s.
 Spot Price for Scenario s: SP(i, s, t)
Model Overview Cont’d
 Probability to Move
from State i to j:
 Pr (i, LF (j, t) =
Probability that Load
Forecast for Day After
Tomorrow is LF (j, t)
Given Present State i
 Reward for Decision
Maker, r (i, s):
Model Overview Cont’d
 Reward for Transition
from i to j & Decision A
is Sum of Rewards
Weighted by
Conditional
Probabilities:
 V (i, T+1): Total
Expected Reward
in T+1 Remaining
Stages from State i
 Solved by Value
Iteration
Summary
 Introduction
 Electric Market is now Competitive
 GenCos Bid on Demand
 Purpose
 MDP Used to Determine Optimal Bidding Strategy
 Problem Formulation
 Transition Probability Determined by Current State, Subsequent State,
& Decision Made
 7 Variables to Define a State
 Aggregation Used to Limit Dimensionality Problems
 Model Overview
 7 Day Planning Horizon
 Objective is to Maximize Summation of Expected Reward
 Value Iteration
Questions???
References
•
•
Song, H.; Liu, C.-C.; Lawarree,
J.; Dahlgren, R.W, “Optimal
Electricity Supply Bidding by
Markov Decision Process,”
IEEE Transactions. Power
Systems, Vol. 15, no. 2, pp 618624, May 2000.
http://www.acclaimimages.com/
_gallery/_pages/0037-04090607-4216.html