Factored Customer Models for
Agent-based Smart Grid Simulation
Prashant Reddy
Manuela Veloso
Machine Learning Department
Computer Science Department
SSchool
h l off Computer
C
t Science
S i
Carnegie Mellon University
8th Annual CMU Conference on the Electricity Industry
March 14, 2012
Smart Grid Distribution
Source: EPRI
2
Distribution Grid Complexity
Source: IEEE
3
Outline
1. Agent-based Smart Grid simulation
 Power Trading Agent Competition
2. Intermediary agent strategies
 Strategy learning for broker agents
 Interactions of multiple learning broker agents
3. Factored customer models
 Timeseries simulation using Bayesian learning
 Decision-theoretic demand side management
4
Outline
1. Agent-based Smart Grid simulation
 Power Trading Agent Competition
2. Intermediary agent strategies
 Strategy learning for broker agents
 Interactions of multiple learning broker agents
3. Factored customer models
 Timeseries simulation using Bayesian learning
 Decision-theoretic demand side management
5
Agent based Smart Grid Simulation
Agent-based
 Distribution
Di t ib ti
grid
id modeled
d l d as a multi-agent
lti
t system
t
 Focus on emergent economics of self-interested behavior 1
 Do not assume rationality
y nor determinism
 Agents contributed by independent research teams
 Competitive benchmarking to drive innovation
1 Leigh
Tesfatsion, ACE: A Constructive Approach to Economic Theory. Ch. 16,
Handbook of Computational Economics, 2005.
6
Agent based Smart Grid Simulation
Agent-based
 Distribution
Di t ib ti
grid
id modeled
d l d as a multi-agent
lti
t system
t
 Focus on emergent economics of self-interested behavior 1
 Do not assume rationality
y nor determinism
 Agents contributed by independent research teams
 Competitive benchmarking to drive innovation
 Power Trading Agent Competition (Power TAC)
 Annual tournament at major
j AI or MAS conference
 Builds upon experience with other TAC domains
 Simulation platform available for offline research
 Assumes liberalized retail markets
 Customers have choice of “broker agents”
1 Leigh
Tesfatsion, ACE: A Constructive Approach to Economic Theory. Ch. 16,
Handbook of Computational Economics, 2005.
7
Power TAC Scenario
Competition
Participants
Simulation
Infrastructure
http://www.powertac.org
p //
p
g
8
9
Outline
1. Agent-based Smart Grid simulation
 Power Trading Agent Competition
2. Intermediary agent strategies
 Strategy learning for broker agents
 Interactions of multiple learning broker agents
3. Factored customer models
 Timeseries simulation using Bayesian learning
 Decision-theoretic demand side management
10
Strategy Learning for Broker Agents
Reddy & Veloso. Strategy Learning for Autonomous Agents in Smart Grid Markets.
Twenty-Second
Twenty
Second Intl
Intl. Joint Conf
Conf. on Artificial Intelligence (IJCAI),
(IJCAI) Barcelona,
Barcelona 2011
2011.
11
Interactions of Multiple Learning Broker Agents
Reddy & Veloso. Learned Behaviors of Multiple Autonomous Agents in Smart Grid Markets.
Twenty-Fifth
Twenty
Fifth AAAI Conf
Conf. on Artificial Intelligence
Intelligence, San Francisco
Francisco, 2011
2011.
12
Outline
1. Agent-based Smart Grid simulation
 Power Trading Agent Competition
2. Intermediary agent strategies
 Strategy learning for broker agents
 Interactions of multiple learning broker agents
3. Factored customer models
 Timeseries simulation using Bayesian learning
 Decision-theoretic demand side management
13
Factored Customer Models
 Power
P
TAC iincludes
l d fundamental
f d
t l and
d statistical
t ti ti l models
d l
 Trade-off on behavioral accuracy vs. scalability
14
Factored Customer Models
 Power
P
TAC iincludes
l d fundamental
f d
t l and
d statistical
t ti ti l models
d l
 Trade-off on behavioral accuracy vs. scalability
 Goals for statistical models:
1. Representation
a. Represent diverse types of consumers and producers
b. Represent varying levels of granularity
2 Automated learning
2.

Learn parameters from “real-world” data
3. Facilitate agent algorithms

Develop algorithms that can be applied in real-world
15
Factored Customer Model Representation
C = h{B
{ i}Ni= 1 ,{S
{ i}Ni= 1 ,U i, B i = {O
{ ij }Mj= i1
16
Factored Customer Model Representation
C = h{B
{ i}Ni= 1 ,{S
{ i}Ni= 1 ,U i, B i = {O
{ ij }Mj= i1
Factored Customer
Tariff Subscriber
Load Bundle
Load Originator
Load Originator
Utility Optimizer
Load Bundle
Tariff Subscriber
Load Originator
17
Factored Customer Model Representation
C = h{B
{ i}Ni= 1 ,{S
{ i}Ni= 1 ,U i, B i = {O
{ ij }Mj= i1
Tariff Terms
- Fixed Payments
- Variable Rates
- Contract Time
- Exit Penalties
- Reputation
Factored Customer
Tariff Subscriber
Load Bundle
Load Originator
Load Originator
Tariff Evaluation
- Inertia
- Rationality
Load Origination
- Reactivity
- Receptivity
- Rationality
Utility Optimizer
Load Bundle
Tariff Subscriber
Load Originator
Calendar
- Time of day
- Day of week
- Month of year
Weather
- Temperature
- Cloud Cover
- Wind Speed
- Wind Direction
Tariff Terms
- Price Elasticity
- Control Events
18
Outline
1. Agent-based Smart Grid simulation
 Power Trading Agent Competition
2. Intermediary agent strategies
 Strategy learning for broker agents
 Interactions of multiple learning broker agents
3. Factored customer models
 Timeseries simulation using Bayesian learning
 Decision-theoretic demand side management
19
Timeseries Simulation using Bayesian Learning
 Gi
Given smallll samples
l off observed
b
dd
data,
t fit a model
d l th
thatt can
generate a long range time series forecast
 Use “similar” samples
p
to improve
p
the fit
20
Timeseries Simulation using Bayesian Learning
 Gi
Given smallll samples
l off observed
b
dd
data,
t fit a model
d l th
thatt can
generate a long range time series forecast
 Use “similar” samples
p
to improve
p
the fit
 ARIMA forecasting over long range is poor
Consumption
Time (Hours)
Consumption
Time (Hours)
21
Bayesian Timeseries Simulation Method
Use similar
training data
Fit hierarchical
Bayesian model
using
Gibbs
G
bbs sa
sampling
p g
Latent factor
optimization
to boost fit
Forecast from
small sample
22
Boosted Bayesian Timeseries Simulation
Consumption
Time (Hours)
Consumption
Time (Hours)
23
Boosted Bayesian Forecasting Accuracy
24
Boosted Bayesian Forecasting Accuracy
25
Outline
1. Agent-based Smart Grid simulation
 Power Trading Agent Competition
2. Intermediary agent strategies
 Strategy learning for broker agents
 Interactions of multiple learning broker agents
3. Factored customer models
 Timeseries simulation using Bayesian learning
 Decision-theoretic demand side management
26
Decision theoretic DSM
Decision-theoretic
 Multi-scale
M lti
l d
decision-making
ii
ki
iin ttwo dimensions
di
i
1. Temporal: Metering period vs. tariff contract period
2. Contextual: Individual load vs. bundle/customer/co-op
/
/
p
27
Decision theoretic DSM
Decision-theoretic
 Multi-scale
M lti
l d
decision-making
ii
ki
iin ttwo dimensions
di
i
1. Temporal: Metering period vs. tariff contract period
2. Contextual: Individual load vs. bundle/customer/co-op
/
/
p
argm ax U S (pt,yt,U N (yt))
yt
argm ax U S (P tz0 ,Y t0 ,U N (Y t0 ))
z2 Z
Self Utility
t0
Price Profile
Load Profile
Neighborhood Utility
28
Decision theoretic DSM
Decision-theoretic
 Multi-scale
M lti
l d
decision-making
ii
ki
iin ttwo dimensions
di
i
1. Temporal: Metering period vs. tariff contract period
2. Contextual: Individual load vs. bundle/customer/co-op
/
/
p
argm ax U S (pt,yt,U N (yt))
yt
argm ax U S (P tz0 ,Y t0 ,U N (Y t0 ))
z2 Z
t0
 Probabilistic
P b bili ti multi-attribute
lti tt ib t utility
tilit model:
d l
S
eλ U ⌧ (z )
P r(z) = P
S (z )
λU ⌧
Z e
29
Decision theoretic DSM
Decision-theoretic
 Multi-scale
M lti
l d
decision-making
ii
ki
iin ttwo dimensions
di
i
1. Temporal: Metering period vs. tariff contract period
2. Contextual: Individual load vs. bundle/customer/co-op
/
/
p
argm ax U S (pt,yt,U N (yt))
yt
argm ax U S (P tz0 ,Y t0 ,U N (Y t0 ))
z2 Z
t0
 Probabilistic
P b bili ti multi-attribute
lti tt ib t utility
tilit model:
d l
Monte
o e Carlo
Ca o Sampling
Sa p g
S
eλ U ⌧ (z )
P r(z) = P
S (z )
λU ⌧
Z e
30
Peak Shifting (Herding) Behavior
Ramchurn, et al. Agent-Based Control for Decentralised Demand Side Management in the
Smart Grid. Autonomous Agent and Multi-Agent Systems (AAMAS), 2011.
31
Household Demand Shifting
C
Consumption
n
 Based
B
d on d
data
t from
f
Germany’s
G
’ MeRegio
M R i project
j t
Time ((Hours))
32
Household Demand Shifting
 Lower
L
variance
i
and
d costt savings
i
off ~10%
10%
Combined
Balancing
g
Temporal
Original
Consumption
33
DSM Deployment Options
ARM C
Cortex-A8
t A8 and
d Zi
Zigbee
b S
SoC
C
34
Conclusion
 Summary
S
 Versatile customer model representation
 Decision-theoretic algorithms
g
for DSM
 Bayesian learning algorithms for timeseries simulation
 Future
F t
Work
W k
 Customer type-specific factor modeling
 Non-cooperative decision-making
g models
 Participating in Power TAC
 Hosted at AAMAS
AAMAS, Valencia
Valencia, June 2012
 More information at http://www.powertac.org
35
Factored Customer Models for
Agent-based Smart Grid Simulation
Prashant Reddy
Manuela Veloso
ppr@cs.cmu.edu
mmv@cs.cmu.edu
School of Computer
p
Science
Carnegie Mellon University