Richard O’Neill
richard.oneill@ferc.gov
Chief Economic Advisor
Federal Energy Regulation Commission
NAS Resource Allocation: Economics and Equity
The Aspen Institute
Queenstown, MD
March 20, 2002
This does not necessarily reflect the view of the Commission.
Richard P. O’Neill
Federal Energy Regulatory Commission
Benjamin F. Hobbs
The Johns Hopkins University and Energieonderzoek Centrum
Nederland
Paul M. Sotkiewicz
University of Florida
William Stewart
College of William and Mary
Michael Rothkopf
Rutgers University
Udi Helman
Federal Energy Regulatory Commission and The Johns Hopkins
University
menu
gin
scotch
hot dog
burger
$4
$6
$3
$5
You must order
from the menu
value
gin
$5
scotch
$9
hot dog
$3
burger
$5
$5
Adam Smith on
network regulation
“The tolls for the maintenance of a high
road, cannot with any safety be made
the property of private persons. ... It is
proper, therefore, that the tolls for the
maintenance of such work should be
put under the management of
commissioners or trustees.” [Wealth of
Nations , Book V, Chap 1, p. 684]
If Adam Smith is not enough, why intervene?
To provide for reliability
To ensure revenue adequacy (no subsidies)
To facilitate entry
generation, transmission, consumption
To mitigate market power
To provide competitive price signals
To protect property rights
How to intervene?
 nondiscriminatory
 lower transactions costs
 more not less options
 control market power
 highly efficient
 “just” prices
will prices be too low?
Are low prices bad?
 are market prices
unconstitutional?
All power corrupts, but we need the electricity.
Electric markets are
incomplete and complex
incomplete if not pricing all desired products
asymmetric markets: vertical demand curve
bidding nonconvexities
Supply: start-up and no-load
demand: for y continuous hours at < $x
intertemporal dependencies
reactive (imaginary/orthogonal) power
socialize transient stability/ voltage
socialize generation and load characteristics
can decentralized markets handle this?
Basics of market design
Contracts (not compacts)
Marginal/incremental cost bidding
Start-up and min run
Trading rules
Financially sound
Market clearing prices
Incentives
For doing “good” things
For not doing “bad” things(socal gas, no withholding)
With collars for political reasons
Information
Auction Design Principles
"Everything should be made as simple as
possible ... but not simpler." Einstein
Don'ts  =22/7
Create gaming opportunities in the name of simplicity
Foreclose marginal (incremental) cost bids
Assume away non-convexities
increase risk
Allow bids that are not firm offers
favor large players
Do's  = 4[1 -1/3 + 1/5 – 1/7 + …]
Allow marginal cost bidding
market clearing price (with scarcity rents)
make the process internally consistent
Create property rights and simple alternatives
Allow self scheduling
differences in auction vs.
COS based regulation
Issue
estimating short run marginal costs
estimating capacity
hold-up problem
estimating return on equity
estimating depreciation
estimating units of service
cost allocation
estimating proper discounts
measuring withholding
free-rider problem
Auction
yes
yes
yes
no
no
no
no
no
yes
yes
COS
yes
yes
yes
yes
yes
yes
yes
yes
yes
no
market design objectives
max bid efficiency within constraints
All RTO (centralized) markets are optional
self-scheduling and voluntary market bids
low transactions costs; no new risks
allow marginal costs bidding(multipart bids)
minimize incentives for market power abuse
don’t favor large participants/portfolio
max arbitrage/min averaging
simultaneous market clearing
lots of information
Coexistence of RTO and off-RTO markets
eliminate bias between off-RTO and RTO markets
RTO markets should not be severely constrained
to promote off-RTO markets
Allow clearing of mutually beneficial trades?
Why are there mutually beneficial trades?
Should off-RTO markets be subsidized? No
Should the RTO be in the insurance biz? No
bilateral physical markets: who pays for the
cleanup after the party?
Self supply
options
Self designed zonal configurations
balanced scheduling
self supply of ancillary services
self designed transmission rights
optional bidding in RTO/ISO markets
no bill from RTO/ISO
pay or get paid for imbalances
Pre-day-ahead markets
for transmission rights: CRT/TCC/TRCs/FGRs
for generation capacity/resevres (ICAP)
market power mitigation via options contracts
day-ahead market for reliability(valium substitute)
simultaneous nodal market-clearing auctions for energy,
ancillary services and congestion
allow multi-part bidding
higher of market or bid cost recovery
allow self scheduling
allow price limit bids on ancillary and congestion
Real-time balancing myopic market
markets are nodal-based LMP with fish protection
Pre-day-ahead markets
 Annual, seasonal, monthly, weekly
 Simultaneous clearing of all products
 Demand side bidding or capacity options (physical?)
 Capacity markets (two year ahead for entry)
 Transmission contracts
 energy forward contracts: FCCs, FTCs
 energy options contracts: CRTs
 flow gate options: FGRs
 FTR options for reserves
 Unbalanced contracts!
 market power mitigation contracts
 bid marginal costs including start up and no load
 pay higher of market clearing price or costs
Day-ahead reliability
management
Optimize system topology over large areas
reserves, real and reactive power balanced
clear most congestion
reserves in place including tx capacity
set imports
clear mutually beneficial trades
interperiod interdependencies: start-up, ramp
rate and minimum load
Design principles for
day-ahead RTO market
Objective: max efficiency within reliability
allow marginal cost bids(start-up/min load)
self-scheduling with optional bidding
simultaneous clearing of all services at LMP
pay higher of bid costs or market clearing
price
financially binding/physically if needed
low risk and transactions costs
Can the real-time market
handle reliability by itself?
 Is a real-time market enough? In theory yes
 can too much real-time scheduling threaten
system stability?
 Neighborhood reliability of the AC load flow
What if it was a DC load flow? Simple
 should there be an incentive not to be more
than x% out of balance?
 first, eliminate all bias to be in the RTM
Design principles for
real-time RTO market
No other markets need be in real-time balance
max efficiency within system balance
deviations from day-ahead priced at market
those operating as scheduled in day-ahead pay
nothing
physically binding
pay market clearing price
low risk and transactions costs
fast market info: prices and quantities
non-simultaneous auctions without
LMP and marginal cost bidding
 Socialize not privatize cost
 higher cost passthroughs: uplift
 create incentives for market power to lower the
risks of the market design
 higher transactions cost of bid preparation
 market power mitigation is more difficult
 constant redesign to correct flaws
 problems in England, Columbia, California
 Australia moving from zonal to nodal
Zonal markets (Cal, PJM, NE, UK)
Sequential markets for energy and anc services
One settlement systems
Infeasible markets (Cal PX and UK)
Ignore nonconvexities (start-up and no-load)
Ignore market power
As-bid pricing
all ended in administrative intervention
No property rights to market power or
poor market design
monopoly and scarcity rents
1
< withheld capacity >
demand
0.8
monopoly
price
$/unit
0.6
0.4
monopoly rents
lost
surplus
competitive
price
scarcity rents
0.2
variable costs
0
0
5
10
quantity
15
20
25
30
good market design allows
proper mitigation
Pre-day-ahead markets
Tx rights
Pre commitment of generators
Feasibility of capacity markets
Day-ahead market
Demand bidding
Marginal costs bid includes startup, noload and running
costs
Real-time balancing market
Bid running costs if you did not bid in DAM
Reliability by adjusting generators via bids
No fault market power mitigation
C
100 MW
2/3
A
150 MW
150 MW
100 MW
1/3
1/3
100 MW
B
Max bt + B(y)
βt
+ K(y) <= f
(μ)
Sequence of auctions; forward and real-time
40,000+ nodes
400, 000+ contingency constraints
Could be highly redundant constraints
K(y) can be non-convex (electromagnetic eqns)
Bids are non-convex mixed integer
Solvable? Bixby says not to worry: 6 orders
of magnitude in 10 years
A Four Node Network with Nomogram Flowgate
NW
NE
Nomogram Flowgate
SW
SE
Bidder
B1
B2
B3
B4
B5
B6
S1
Bid Type
FB option,
NE to NW.
Buy up to
100 MW
FB option,
NW to SW.
Buy up to
100 MW
PtP
forward,
NW to SW.
Buy up to
100 MW
PtP option,
NW to SW.
Buy up to
100 MW
PtP
forward,
NE to NW.
Buy up to
100 MW.
PAR
capacity.
Buy up to
100 MW.
Forward
generation
at SW.
Sell up to
200 MW.
Bid
($/MW)
10
30
20
25
25
25
-10
Quantity
awarded
100
100
0
100
40
20
200
Reduced
cost
10
5
-5
0
0
25
20
PTDFs:
Shadow Price:
NW to SW
0.0
1.0
0.8
0.8
0.6
0.0
0.0
25
NE to NW
1.0
0.0
-0.2
0.0
0.6
0.0
0.0
0
NE to SE
0.0
0.0
0.2
0.2
0.4
0.0
0.0
0
SE to SW
0.0
0.0
0.2
0.2
0.4
0.0
0.0
25
PAR
0.0
0.0
0.0
0.0
0.0
1.0
0.0
5
SF
nomogram
0.0
0.0
0.0
0.0
0.0
0.0
-1.0
30
Application: Electric Power Generator
Unit Commitment
fixed
su su
sd sd

(
C
g

S
z

S
z

S
Maximize:
  i it i it i it i zit )
i
subject to:
t
  git   dt
i
git  zit G
t,
0
i, t,
0
i, t,
zit  zi ,t 1  zitsu  0
i, t,
 zit  zi ,t 1  z  0
i, t,
min
i
git  zit G
max
i
sd
it
zitsu, zitsd  {0,1}; all other variables  0; zit < 1
Problem 1: Single Period, 3 Plants
Plant
1
2
3
MIN Q
MAX Q
MC/unit
Fixed$/hr
StartUp$
ShutDown$
50
150
4
125
25
0
100
200
2
50
150
0
20
50
6
0
0
0
0  Load  400
Commodity Price
7
6
Price $/Unit
5
4
3
2
1
0
0
100
200
Quantity
300
400
Average Cost vs. Price
8
Average Cost
Price
7
6
$/Unit
5
4
3
2
1
0
0
100
200
Quantity
300
400
Startup Payments
200
Start P1
Start P2
Start P3
Dollars
150
100
50
0
0
100
200
300
400
Unit Commitment Extensions
1. Multi-Period Considerations: e.g., ramp rate limits
- Problem 2 (2 hours): Assume RR limit = 105 MW/hr,
demands = 180 MW and 395 MW
- Both plants start up in period 1 because of ramp rate
- Plant 1 gets paid $150 to start up in period 1 (commodity
price alone supports operation only in period 2); profit = $175
- Degeneracy/multiple duals a problem
2. Ancillary Services
3. Transmission Congestion Payments
4. Demand Bidding
Smokestack versus High Tech
(from Scarf, 1994)
Production
Characteristics
Capacity
Construction Cost
Marginal Cost
Average Cost at
Capacity
Total Cost at
Capacity
Smokestack
(Type 1 Unit)
16
53
3
6.3125
High Tech
(Type 2 Unit)
7
30
2
6.2857
101
44
Formulate and Solve MIP
(Simulates Bid Evaluation by
Auctioneer)
Let:
z1, z2 = construction decisions of types 1 & 2,
respectively
q1, q2 = output for types 1 & 2
MIP:
Max -i (53z1i + 3q1i) - Σi (30z2i + 2q2i)
i (q1i + q2i) = Q
-16z1i + q1i  0; z1i  {0, 1}, q1i  0, i =1,2,…
-7z2i + q2i  0; z2i  {0, 1}, q2i  0, i =1,2,…
Efficient Outcome
• To optimally satisfy a demand of, say, 61 units:
#Type 1 # Type 2 Type 1
Demand
Units
Units
Output Output
Type 2
Total
Cost
61
3
2
47
14
388
Note that one Type 1 does not operate at capacity
• The output price according to the LP solution is $3
– But both types make negative profits
 not an equilibrium outcome
6.43
Average cost as a function of demand
6.42
for Scarf's problem
6.41
6.4
6.39
average cost
6.38
6.37
6.36
6.35
6.34
6.33
6.32
6.31
6.3
6.29
6.28
55
56
57
58
59
60
61
62 63
demand
64
65
66
67
68
69
70
optimal value as a function of demand
for Scarf's problem
450
440
430
optimal value
420
410
400
390
380
370
360
350
340
55
56
57
58
59
60
61
62 63
demand
64
65
66
67
68
69
70
450
440
optimal value as a function of demand
430
for Scarf's problem
420
optimal value
410
400
390
380
370
360
350
340
55
56
57
58
59
60
61
62
63
de mand
64
65
66
67
68
69
70
LP That Solves the MIP
Max -i (53z1i + 3q1i) -Σi (30z2i + 2q2i)
Duals
i (q1i + q2i) = Q
(y0**)
-16z1i + q1i  0 , i =1,2,..
(y1i)
z1i = z1i*
q1i  0
-7z2i + q2i  0 , i =1,2,..
z2i = z2i*
q2i  0,
(w1i**)
(y2i)
(w2i **)
** Used in payment scheme
Prices at an Output of 61
Unit type 1
(Smokestack)
(y)
(w1i)
(y1i)
Demand Commodity Price Start-up Capacity
61
3**
-53**
0
Unit type 2
(High Tech)
(w2i) (y2i)
Start-up Capacity
-23**
1
**Prices paid by unit to auctioneer
Thus, each unit is paid a start-up cost, ensuring nonnegative profit
(here, 0)
Dual Prices for Scarf's Problem
Unit 1 (Smokestack)
Dual Price Commodity
Set
Price
Unit 2 (High Tech)
Start-up
Price
Capacity
Price
Start-up
Price
Capacity
Price
3
53
0
23
-1
Set II
6.3125
0
-3.3125
-.1875
-4.3125
Set III
6.2857
.429
-3.2857
0
-4.2857
Set I
Non-Convexities in Markets
• While market models often assume away non-convexities (e.g.,
integral decisions and economies of scale),
… they exist!
• Electric utility industry:
– Still economies of scale in generation and especially transmission
– Unit commitment: start-up, shut-down costs; minimum run levels
• Why disregard non-convexities in market models?
… with convex profit maximization problems (concave
objective, convex feasible region), we can usually:
– define linear (“one-part”) market clearing prices
– establish existence, uniqueness properties for market equilibria
– create tests for entering activities
The Problem with Non-Convexities
• Linear prices can no longer clear the market …
an equilibrium cannot be guaranteed to exist
• E.g., electric power operations:
– At P < P*, inadequate supply
– At P > P*, a lump of additional supply enters that breaks
even (covers fixed costs), but supply exceeds demand. If
force any generator to back off, its profit < 0
– “Administrative” solution to reach the optimum:
• Adjust outputs to restore feasibility
• Side payments to ensure no one loses money
• As Scarf (1990, 1994) then points out, there is no
price test for Pareto improving entry of new
production processes
Why Address Non-Convexities in
Auctions Now?
1) New markets for electric power have non-convexities
2) A debate surrounding these power markets: the use of
prices to induce efficient, decentralized decisions a la
Walrasian auctions
3) Auction mechanisms in the NYISO and PJM attempt to
account for integer decisions
4) California said no to such an auction because of
complaints:
- these prices are not “equilibrium supporting” and
- administrative adjustments appear arbitrary
(e.g., Johnson, Oren, Svoboda 1998)
5) Our result: If integral decisions can be priced, a market
equilibrium can be supported
Some Related Literature
• Scarf (1990, 1994)
– Emphasized the divergence of math programming and economics
– Searched (unsuccessfully) for a way to find prices in the presence of
integral choices, and for pricing tests for improvements
• Gomory and Baumol (1960)
– The use of cutting plans that are combinations of existing constraints
to arrive at an integer solution
– Interpreted duals of those planes; stopped short of pricing individual
integer activities
• Wolsey (1981)
– Pure IP with integer constraint coefficients & RHSs
– Approaches to constructing price functions yielding dual problems
that satisfy weak and strong duality. Functions generally nonlinear
• Williams (1996)
– Examines possible duality for integer programs, but concludes no
“satisfactory” duals (Lagrange multipliers) exist
Pricing Integral Activities
• One can think of the traditional pricing approach as a
misspecification of the commodity space
– The commodity space could include integer decisions as an
“intermediate good”
• The pricing system derived here is similar to multi-part
pricing for utilities
– For example, an demand charge (fixed costs) and an energy
charge
• Buyers’ clubs with multipart contract
Our Approach
to Addressing Non-Convexities
1. Formulate the non-convex problem as a maximization MIP.
Bids include all costs and internal constraints.
2. Solve MIP
–
Take advantage of modern MIP technology
3. Take integer solutions z* and define equality constraints z=z*
in a LP -- “convexifying” the problem.
4. Solve LP
5. Duals on z=z* are prices on the integer variables
–
–
If market/auction participant pays those prices, together with the
duals on commodity and other coupling constraints…
…. then those prices support an equilibrium
For (0,1) variables, can use only negative prices for z*=1 and positive
prices for z* = 0
General Formulation: MIP
Let:
k = index for auction participants
xk, zk = activities
ck, dk = marginal benefits of activities (cost, if <0).
(ckxk + dkzk is the total benefit to participant k)
Ak1, Ak2, Bk1, Bk2 = constraint coefficients
b0 = commodities to be auctioned. In double auction, b0 =
0
bk = RHS of internal constraints of participant k
MIP:
max k (ckxk + dkzk)
subject to: k (Ak1xk + Ak2zk)  b0
}
Bk1xk + Bk2zk  bk
all k
An LP That Solves The MIP
LP(z*): Max k (ckxk + dkzk)
s.t.
k (Ak1xk + Ak2zk)  b0
Bk1xk + Bk2zk  bk
zk = zk *
xk  0
}
all k
where z* indicates an optimal value of z in MIP
Definition of Equilibrium
Definition 1. A “market clearing” set of contracts has the following
characteristics:
1. Each bidder is in equilibrium in the following sense. Given
•
•
prices {y0*, wk*} and payment function Pk(xk,zk) defined by the
contract
no restrictions on xk and zk other than k’s internal constraints
(Bk1xk + Bk2zk  bk)
then no bidder k can find feasible xk’, zk’ for which:
(ckxk’ + dkzk’ - Pk(xk’,zk’)) > (ckxk* + dkzk* - Pk(xk*,zk*))
Thus, the prices support the equilibrium {xk*,zk*}.
2. Supply meets demand for the commodities. I.e.,
k (Ak1xk* + Ak2zk*) < b0
A Candidate Set of Market
Clearing Prices and Quantities
Definition 2. Consider the contract Tk with the following terms:
1. Bidder k sells zk=zk*, xk0=xk0* (where xk0 is the subset of xk
with nonzero Ak1)
2. Bidder k pays auctioneer:
Pk(xk,zk) = y0* (Ak1xk+Ak2zk) + wk* zk.
where * indicates an optimal solution to MIP / LP(z*)
Variant. Define:
wk*’ = Max(0, wk*) if zk* = 0
= Min(0, wk*) if zk* = 1
Pk(xk,zk) = y0* (Ak1xk+Ak2zk) + wk*’ zk
Existence of Market Clearing
Contracts for MIP Auction
Theorem. T  { Tk} is a market clearing set of contracts.
• Proof exploits complementary slackness conditions from
auction LP to show that at the candidate equilibrium prices,
{zk*,xk0*} is optimal for each bidder’s own MIP:
Max [ckxk + dkzk] - [y0* (Ak1xk+Ak2zk) + wk* zk]
s.t.
Bk1xk + Bk2zk  bk
xk  0, zk  {0,1}
• There may be alternative optima for the bidder
• Profits nonnegative
• If wk* < 0 and zk* =1, interpretable as NYISO/PJM mechanism
for preventing winning bidders from losing money
• In this framework, there are payments directly for individual
capacity
A Welfare Result
Corollary. If each participant k bids truthfully (submits a bid
reflecting its true valuations (ckxk + dkzk) and true constraints
(Bk1xk + Bk2zk  bk; xk  0; zk  {0,1}),
... Then an auction defined as follows will:
(a) maximize net social benefits (k [ckxk + dkzk]) and
(b) clear the market
The auction includes the following steps:
1.
The auctioneer solves MIP, yielding primal {xk0*,zk*};
2.
The auctioneer solves LP(z*), obtaining prices {y0*, wk*}
3.
The auctioneer imposes contract T upon the bidders
Extensions
• Tests for Pareto improving entry of new activities
• How useful is this really likely to be?
– For small systems, where lumpiness looms large, market power is
important
– For large systems, the duality gap shrinks .. into irrelevancy?
• Strategic bidding over these integral activities.
– Comparisons to iterative single part bid auctions.
– Do more bidding degrees of freedom facilitate strategic behavior?
What are the impacts?
• Consequences for distribution of rents in the market
between consumers and producers, and among groups of
producers
• Tests on larger systems
Wholestic Market Design AGORAPHOBIA
You don’t
always get it
right the
first time.
Now you have
experience
Try
LMP
Are you a
Copernican or a
Ptolemain?
We had to destroy the market to save it