Optimal Electricity Supply
Bidding by Markov Decision
Process
Authors: Haili Song, Chen-Ching Liu, Jacques Lawarree, & Robert Dahlgren
Presentation Review By:
Feng Gao, Esteban Gil, & Kory Hedman
IE 513 Analysis of Stochastic Systems
Professor Sarah Ryan
March 28, 2005
Outline
 Summary of the previous presentation
 Model Validation
 Implementation and case study
 Description of Examples
 Summary
Summary of previous presentation
 Introduction
 Electric Market is now Competitive
 GenCos Bid on Demand
 Purpose
 MDP Used to Determine Optimal Bidding Strategy
 Problem Formulation
 Transition Probability Determined by Current State, Subsequent State,
& Decision Made
 7 Variables to Define a State
 Aggregation Used to Limit Dimensionality Problems
 Model Overview
 7 Day Planning Horizon
 Objective is to Maximize Summation of Expected Reward
 Value Iteration
Value Iteration Discussion
 V (i, T+1): Total




Expected Reward in
T+1 Remaining Stages
from State I
At the last stage T = 0
Value Iteration
(Backward Induction)
Ignore discount factor
The immediate reward
is dependent on the
initial state, following
state and decision a
Model Overview Clarification
 Sum of all Scenarios S
that result in a given spot
price, cleared quantity,
and production limit.
 Prob to Move from State i
to j given decision a =
[Prob (that the spot price,
production level are
correct and load forecast
= demand)*prob(of having
the proper load forecast)]
 Resulting Spot Price can
be dependent on Decision
a if the bidder has market
power
Model Validation
 For model validation:
 Accumulate actual data and observations from the
market over a period of time (e.g. 1 year)
 Market data set provides the actual scenarios
 Relationship between estimated by the BIDS
representation r(i,j,a) and actual rewards w(i,j,a)
can be analyzed by linear regression.
Case Study
 3 suppliers: GenCoA, GenCoB, and GenCoC, all





bidding in the spot market
GenCoA is the decision maker using the Markov
Decision Process technique
GenCoA: 1 generating unit
GenCoB: 2 generating units
GenCoC: 2 generating units
Planning Horizon: 7 days (bid decision for next day
considers the entire week ahead
Case Study
 GenCoA makes a decision from a set of pre-specified
decision options
 GenCoA does not know exactly how GenCoB and
GenCoC are going to bid
 But their individual bidding behavior is modeled by
bid prices, quantities and the associated probabilities
based on GenCoA’s knowledge and information
 Transition probabilities and rewards are calculated
using algorithm described in previous presentation
Two Basic Market Situations
 EXAMPLE 1:
 Decision-maker has a production limit over the
planning horizon
 Decision-maker does not have market power
(perfect competition)
 Optimal strategy is time dependent due to the
production limit
 In some states the optimal decision is not to sell,
but to save the resources for more profitable days
Two Basic Market Situations
 EXAMPLE 2:
 Decision-maker has market power: it can manipulate
the bid to influence the spot price
 Decision-maker has no production limit
 Decision-maker makes the bidding decision to
maximize the expected reward over the planning
horizon
 Daily maximum strategy is time independent:
decision-maker makes the same decision as long as the
system is in the same state
 BIDS value iteration is time dependent: it takes into
account how current biddings affect future spot prices
Comparison of Two Cases
 Without market power, bidder is concerned with
saving resources for more expensive periods
 With market power, bidder is concerned with
properly influencing the future spot price to
maximize profit
 Knowing whether the bidder has market power or
not is crucial since the relationship between spot
prices and decisions would depend on each other
Summary
 Model Overview
 7 Day Planning Horizon
 Objective is to Maximize Summation of Expected Reward
 Value Iteration
 Model Validation
 Comparison of Predicted and Actual Results (by linear
regression)
 Implementation and case study
 Three GenCos, GenCo A is the Decision Maker
 5 Generators among the 3 GenCos
 Description of 2 Examples:
 Production Limit without Market Power
 Market Power without Production Limit
 Next Time: Presentation and Discussion of Results and Conclusions
Questions???