Graduate Student: Emre Can Kara, CEE, CMU
Using Smart Devices for System-Level Management and Control in the Smart Grid
Research Advisors: ! Mario
Bergés,between
Assistant
Prof.,
CEE,
CMU. and
Bruce
Krogh, Prof.,
The balance
the
power
demand
generation
in ECE, CMU.
Using Smart Devices for System-Level Management and Control
the electricity grid is managed by traditional generation and
Statement
reserve assets (such as gas turbines and spinning reserves)
in the Smart Grid
that act on larger time scales (minutes).
Student: Emre Can Kara, CEE, CMU
Graduate Student: Emre Can Kara, CEE, CMU
! The balance between the power demand and generation in
! Attempts to integrate intermittentAdvisors:
energy Mario
sources
(i.e. CEE,
windCMU. Bruce Krogh, ECE, Research
Bergés,
CMU.
Soummya
Kar,
ECE,
CMU.
Advisors:
Mario
Bergés,
Assistant Prof., CEE, CMU. Bruce Krogh, Prof., ECE, CMU.
the electricity grid is managed by traditional generation and
and solar) to the grid contributes to the imbalance between the
Problem
Statement
reserve
assets
(such
as
gas
turbines
and
spinning
reserves)
t Grid Conceptual
Framework
Problem Statement
power supply and the demand, hence threatening the stability
that act on larger time scales (minutes).
! The balance between the power demand and generation in
Research
Questions
and rapidly
security
of thewith
network.
Energy infrastructure is being
populated
smart devices that can
City Power Solar Farm Wind Farm
the electricity grid is managed by traditional generation and
ttempts
to integrate intermittent energy
Ø What is the reserve
optimum
control
for different
types reserves)
of
assets
(suchstrategy
as gas turbines
and spinning
monitor and control individual end-use loads, opening the door !
forAa
wide
Plantsources (i.e. wind
that act
on larger time scales (minutes).
appliances/ building
types?
solar) to the grid contributes to the imbalance between the
variety of applications and a re-thinking of the traditional power and
distribution
! Attempts communication
to integrate intermittent
energy
sources
(i.e. wind
Source: IEEE Smart Grid Conceptual Framework
system. The typical use
cases for these smart devices relate
to
the
Ø
W
hat
are
desirable
protocols
for
distinct
High Capacity
Power hence threatening the stability
power supply and
the demand,
and solar) to the grid contributes to the imbalance between the
reduction of energy consumption through a number of mechanisms ranging
combinations ofpower
controlsupply
strategy
and
the
appliance
type?
Plants
and the demand, hence threatening the stability
and
security
of
the
network.
from
load
shedding
during
peak
demand
hours
(demand
response),
to
Ø Can contextual
buildings help develop better
and information
security of theon
network.
Unlike
traditional
methods,
appliances
can be used as
to maintain the
occupancy-controlled heating, ventilation and air conditioning (HVAC) and
demand
side
management
techniques?
providers
of
short-term
(seconds)
ancillary
services,
such
as
generation
to
lighting systems, to simply providing real-time energy consumption
Ø Can sensor fusion and machine learning techniques be used to
Background
Research
and more
load energy-efficient
balancing.
research
areas
information
to end-usersfrequency
in order tocontrol
encourage
und
Research
develop
solutions
that leverage
the methods,
existing infrastructure?
Unlike
traditional
appliances can be used as
Researchers
are
seeking
different
techniques
to
maintain
the
behavior. However, more
dynamic Questions
mechanisms are possible through
Research
providers of short-term (seconds) ancillary services, such as
balance between the power demand and generation to
Unlike
canstability
be and
used
as
distributeddifferent
fine-grained
control of small
loads, which
of this traditional methods, appliancesensure
e seeking
techniques
to maintain
theis the focus
frequency control and load balancing.
security in the grid. Main research
areas
Research Timeline
!
W
hat
is
the
optimum
control
strategy
for
different
types
of
research.
include:
providers of short-term (seconds) ancillary
services, such as
Research Questions
en the power demand and generation to
Timeline
The main problem we seek
to address in building
this projecttypes?
is how to leverage endappliances/
! Large-Scale Energy Storage Systems, ex:
! What
is the optimumYearcontrol
strategy for
different types of
Systems)
Year
1
2
Year
3
frequency
control
and
load
balancing.
anduse
security
in
the
grid.
Main
research
areas
appliances/
types?
electrical loads to provide ancillary services (particularly balancing
! Plug-In Hybrid Vehicles (Vehicle to Grid Systems)
Control Algorithm
Development building
of a local
Development of a
Development of a distributed
!
W
hat
are
desirable
communication
protocols
for
distinct
controller
controller
controller
! What
are desirable centralized
communication
protocols
for distinct
Research
Questions
! Thermal Storage Systems
generation to load through frequency regulation) in the power grid. In
Type of Load
Small TCLs*
Passive Loads, Small and
All Loads
combinations
of
control
strategy
and
the
appliance
type?
combinations
of
control
strategy
and
the
appliance
type?
! Large Scale Battery Systems
Large TCLs
particular
we
would
like
to
shed
light
on
the
system-level
properties
that
can
Energy Storage Systems, ex:
! What is the optimum control strategy for
different
types
of
! Can
contextual
buildingsMixed
help develop better
Laboratory,
Residential information
Commercial,on
Residential
! Demand-side management (DSM) techniques, ex: Type of Facility
be influenced through coordinated
(centralized
or
otherwise)
control
of
a
!
C
an
contextual
information
on
buildings
help
develop
better
demand
side management
techniques?
ues,
ex:
appliances/
building
types?
Number of Loads
Implementation:
10
Implementation:
10
Implementation: 10
!
G
rid
Friendly
Appliances
ybrid
Vehicles
(Vehicle
to
Grid
Systems)
large collection of smart loads, where we use the term smart to denote their
Distribution
Transmission
Simulation:
10 fusion and
Simulation:
10
Simulation:
10
!
C
an
sensor
machine
learning
techniques
be used
demand
side
management
techniques?
!
D
ynamic
Pricing
ability Systems
to react to measurements or signals.
Grid protocols for distinct
Grid
! What are desirable communication
to develop solutions that leverage the existing infrastructure?
Storage
! Can sensor fusion and machine
learning
techniques
be
used
combinations of control strategy and the appliance type?
ale Battery Systems
to develop solutions that leverage the existing infrastructure?
Vision and Objectives
Vision
and
Objectives
! Can contextual information on buildings help develop better
e management (DSM) techniques, ex:
Vision:
demand
side
management
techniques?
Envisioned Non-Intrusive
Our
research
method
will
involve
a
combination of theoretical analysis,
dly Appliances
Smart devices in the power grid to provide
Smart Controller:
Grid
simulations
and machine
real-life experiments
to evaluate alternative
methods for
!
C
an
sensor
fusion
and
learning
techniques
be
used
An Embedded Platform
load-balancing
functions
currently
assigned
Pricing
Control Strategy
harnessing the coordinated responses
of
smart
devices
for
system-level
to a infrastructure?
relatively small number of bulk power
to develop solutions
that
leverage
the
existing
Data Collection
purposes. These methods will range from centralized strategies, where signals
Source: IEEE Smart Grid Conceptual Framework
1
2
3
2
4
5
*TCL: Thermostatically Controlled Loads
Regional Signals
2 Modeling Approach
Nbin
The individual TCL model is the same as that used in
[6]. The parameter, a, which governs the thermal characteristics of each TCL, i, is defined as:
1
3
2
(1)
where Ci and Ri are the thermal capacitance and resistance of TCL i, and h is the time step.
The difference equation describing the temperature dynamics of TCL i is:
OFF
to x as a vector of probability mass. The A-matrix can
be thought of as a Markov transition matrix describing the
probability of TCLs moving from one state bin to the next.
Figure 1 shows how the state bins map to the temperature
dead-band.
We next present an analytical derivation of the Amatrix, which we compare to an identified A-matrix. We
will discuss the structure of the B and C matrices and u
and y vectors in subsequent sections.
C'$#1+,4
?&1>'&
2.2.1 Analytical Derivation of the A-matrix
In this section, we will analytically derive the elements
of the A-matrix. The purpose of this exercise is to demonstrate that the*$$&'$,/'G'8-&#3/#1+8@;'&5#-'8HE1+8/&,#+/8H;-.'6%4'B
modeling framework is rooted in basic probabilistic logic. In subsequent sections, we will estimate the
C'81%&-'Ͳ4'5'4G'8-&#3/#1+@;'&5#-'8HE1+8/&,#+/8H;-.'6%4'B
A-matrix with data generated by simulating (2).
We assume that all E1221+E122%+#-,/#1+A,F'&
TCLs have the same resistance
(Ri = R) and rated power (Prate,i = Prate ). This implies
that all TCLs have the same temperature gain (θg,i = θg ).
We also assume that all TCLs experience the same ambient temperature (θa,i = θa ), which is constant over time.
Lastly, we ignore the noise process ω. Therefore, (2) becomes:
?%#46#+$@A1-,4B
*$$&'$,/'G'8-&#3/#1+8@;'&5#-'8HE1+8/&,#+/8H;-.'6%4'B
C'81%&-'Ͳ4'5'4G'8-&#3/#1+@;'&5#-'8HE1+8/&,#+/8H;-.'6%4'B
Example: TCLs
Data
Power
*TCL: Thermostatically Controlled Loads
Model
-Event detection
-Classification
-TCL Parametric
Model
Machine
Learning
Techniques
A1-,4
?&1>'&
Communication Protocol
"#$%&'()*+,&-.#/'-/%&'01&#234'2'+/#+$,4/'&+,/#5'41,6Ͳ7,8'6-1+/&148/&,/'$#'8)
State
Indicator
Nominal
Temperature
"#$%&'9):.'#+/'&+'/'+,74'6;'+81&*+6&'<8'+81&=,-/%,/1&+'/<1&>)
!
Test
Bed
–The
PSII
Lab
at
CMU
as
part
of
the
Campus
Wide
Sensor
Andrew
Network
•  Centralized
C'$#1+
A1-,4
?&1>'&
D417,4;#$+,48
?
where Prate,i is the TCL’s rated power. θg,i is positive
for cooling TCLs and negative for heating TCLs.
2.2 TCL Population Model
Appliance State Models:
to do appliance state
prediction
C'$#1+,4;#$+,48
where θ is the internal temperature of the TCL, θa is the
ambient temperature, m is a dimensionless discrete variable equal to 1 when the TCL is ON and 0 when it is OFF,
and ω is a noise
�process. The ON temperature gain is:
Ri Prate,i for cooling devices
θg,i =
,
(3)
−Ri Prate,i for heating devices
?%#46#+$;#$+,48
Plug Level NILM:
cs:
to determine the appliance
e
type
ect Motivating
the type of the
device
Information
Case and Future Research Directions
θi,t+1 = ai θi,t + (1 − ai )(θa,i − mi,t θg,i ) + ωi,t , (2)
?
N
Nbin Nbin Nbin
-2
-1 bin
-3
2
2
2
2
4
Sensor
Fusion
C'$#1+,4
?&1>'&
E1221+E122%+#-,/#1+A,F'&
D417,4;#$+,48
ai = e−h/Ci Ri ,
Plug Level NILM:
to determine the appliance
type
Nbin Nbin Nbin Nbin
2 +4 2 +3 2 +2 2 +1
Nbin-1 Nbin-2 Nbin-3
•  Centralized
Information
ON
temperature
Figure 1: State bin transition model.
2.1 Individual TCL Model
C'$#1+
C'$#1+,4;#$+,48
Resource-­‐level Description (Services, Constraints, Schedule)
Information
D&#6
D417,4;#$+,48
Aggregate Descriptions (Services, Constraints, Schedule)
Common Communication Layer
?%#46#+$;#$+,48
The paper proceeds as follows. In Section 2, we describe our modeling approach and derive an LTI system
that captures the temperature dynamics of the TCL population. In Section 3, we present different options for information exchange between the central controller and the
TCL population. Section 4 presents the MPC approach,
which is tested in a case study in Section 5. In Section 6,
we present ideas for further research and conclude.
Decision Hierarchy
•  Distributed
C'$#1+,4;#$+,48
Building Signals
ovide
d
objectives
gned
ower
Data Collection
s inBuilding the(Local)
power grid to provide
g functions currently assigned
y small number of bulk power
devices.
are introduced through the distribution system that trigger and coordinate the
responses of individual devices, to highly
distributed strategies, where each
Characteristics:
device is making decisions based entirely
on local information, but these
• Non-Intrusive
Regional • Ability
to detect
the type
ofthe
thedesired
device
decisions
are
implemented
so
that
the
emergent
behavior
is
Broker
Control Strategy
connected
system-level control action.
awareness
Our initial focus will be on the design• Aofppliance
softwarecondition
for plug-level
devices for
Envisioned Non-Intrusive
!"
• Actuation
controlling
thermal
loads
such
as
refrigerators.
Then we will focus on
Smart Controller:
Decision Hierarchy
understanding the aggregate effect of smart devices. Adjusting their power
An Embedded
Platform
•  Distributed
!"
Research Timeline:
D&#6
consumption based on the need forControl
ancillaryStrategy
services will depend upon several
Local
factors, including the duration of the ON-OFF periods, the total power change
Data
Collection
Broker
C'$#1+
that can be effected by each device, the number of devices taking action in the
Hierarchy
system, and the relative phases ofDecision
the control
actions. Finally we will focus on
•  Centralized
•  Distributed
different control strategies
that
leverage on the responsiveness of smart loads
D&#6
at the grid level. It is necessary to incorporate models of the load influenced by
?%#46#+$@A1-,4B
Sensor
smart devices into system-level simulation studies.
Fusion
Global Signals
Envisioned Non-Intrusive
Smart Controller:
An Embedded Platform
Region
sources and devices.
C'$#1+,4
?&1>'&
all TCLs*
i
start
end
i,t g
a
end
i start
i
*$$&'$,/'G'8-&#3/#1+8@;'&5#-'8HE1+8/&,#+/8H;-.'6%4'B
a
i,t g
C'81%&-'Ͳ4'5'4G'8-&#3/#1+@;'&5#-'8HE1+8/&,#+/8H;-.'6%4'B
A1-,4
?&1>'&
E1221+E122%+#-,/#1+A,F'&
k
k
k
k
?
start
Indicator
Year
3
-Event
detection
=>%
*Development
"#*$##################*%&'(## of a
Development of a -Classification
-TCL##distributed
Parametriccontroller
Nominal
centralized controller
Model
Power
Temperature
Passive Loads, Small All Loads
and Large TCLs
Commercial,
Residential
Mixed
ementation: 101
ulation: 103
Implementation: 102
Simulation: 104
Implementation: 102
Simulation:
105
!
!
!
!
!
!
Power
!
!"#$%&'()!!!!!"#$ ! ! !!!!"#!!
!
! Test bed for sensor
fusion opportunities and
HVAC control.
!
!
!
Example: TCLs
!
!"#"$%&'()*+#,)'%
!
!!
!
!!!"#$ ! ! !!!!"#!!
! !"#$%&'()!!
-)'"*).%!13'#.%
-)'"*).%/$0121)'%
!
!
!"#$%&'()!!
!
!
!!!!"#!!
!!!"#$
%
!
!"!!
! ! !!"#
!
!"#$%&'()!!
! ! !!!!!"#!!
=>%
!!!"#$
!
!
##!
#
!!!"#$
!
"# $################
%&'( )$#
%
!
!
! !"!!!!!"#$ ! ! !!"#!!"# !!
C control
nd actuate the
: Thermostatically Controlled Loads
,-2%<)0,-2%
Data
oratory,
idential
!
!!!"!!
!"#$%!!!"#"$!
!!!"!!
%
end
a
start
i
i
a
end
t g
a
start
t g
start
start
t g
Appliance State Models:
to do appliance state
prediction
each temperature interval into two state bins, one for TCLs
that are ON and one for TCLs that are OFF. This results
in Nbin state bins. The state vector x contains the number
of TCLs in each state bin, or, if normalized by the total
number of TCLs, the fraction of TCLs in each state bin,
which is equivalent to!'-#*'+%897'-%
probability mass in the infinite system limit. In the remainder
of the paper, we will refer
#'%:,#9;%.344+$%
!"#"$%&'()*+#,)'%4$"*1$5#.%
Year 2
end
end
i
bin
Model
t
i
set
Timeline
Data
i,t
Power [kW]
k+1
!"!!!!!"#$ ! ! !!"# !!
!!!"!!
!"!!!!!"#$ ! ! !!"# !!
!!!"!!
14
Consider the same group of TCLs.
The probability of
TCLs going from θstart to some θend , which is within a state
bin (θn < θend < θn+1 ), is:
20
P(θn < θend <θn+1 |θstart )
(10)
� a2
15
= P(a1 < a < a2 ) =
p(a) da,
a1
10
Temperature Interval
Population
102
elopment
of a
l controller
i i,t
15
16
Techniques
17
18
19
20
18
19
20
Temperature Interval Evolution
"#$%&'9):.'#+/'&+'/'+,74'6;'+81&*+6&'<8'+81&=,-/%,/1&+'/<1&>)
5
Model
-Event detection
-Classification
-TCL Parametric
Model
0
State
14
Indicator
Temperature
[C]
r1
i,t+1
?%#46#+$;#$+,48
! Fine-grained and programmable HVAC control
! Test bed for sensor
Communication Protocol
Plug
Level
NILM:
"#$%&'()*+,&-.#/'-/%&'01&#234'2'+/#+$,4/'&+,/#5'41,6Ͳ7,8'6-1+/&148/&,/'$#'8)
?%#46#+$@A1-,4B
fusion opportunities and
To simulate the behavior of a population of TCLs, we
Sensor
!
A
bility
to
measure
each
plug
load
and
actuate
the
θ
=
a
θ
+
(1
−
a
)(θ
−
m
θ
).
(6)
could aggregate thousands of single TCL models using
Machine
determine
appliance
Future
Research
(2); however, this to
would be
computationally intensive and the
Centralized
control
strategy
with
1200
TCLs
based
on
Callaway
et.
al.,
2011
Take
a
TCL
going
from
θ
to
θ
in
one
time
step:
ndition
awareness
HVAC control.
appliances
remotely
Fusion
.'+,*%/,*0%
1"-2%/,*0%
!"#$%&'()*%&+,-#%
the aggregate system would not be in a form amenable to
θ = a θ + (1 − a )(θ − m θ ).
(7)
many control techniques. Instead, in this paper we will
.344+$% Learning
type
formulating
work with a discrete LTI system in state space form:
Test
bed for a Markov Decision Processes
! Programmable control for plug-in appliances Currently we are working! on
Appliance State Models: 5-6'*0,7'-%Techniques
Now consider a group of TCLs that are either all ON or all
!'-#*'++)*%
OFF (m = m ). The probability of TCLs going from
x
= Ax + Bu
(4)
Aggregated Power vs. Reference
θ to θ is:
(MDP)
based model tocommunication
provide a more generic foundation to support
y = Cx ,
(5)
to do appliance state
15000
!
V
arious
sensor
nodes
to
provide
feedback
and
help
P(θ |θ ) = P(a ),
(8)
Plant Aggregeted Power
a
which allows us to use a wide range of system analysis
Reference
framework
between
the
various Reinforcement
Learning
techniques.
where we have assumed that
P(a ) is independent of teminvestigate
algorithms
of
state
prediction
10000
tools and advanced controls techniques.
prediction
Communication
Protocol
oller
perature. This probability can be computed by solving for
Assume all TCLs in a population have the same tem"#$%&'()*+,&-.#/'-/%&'01&#234'2'+/#+$,4/'&+,/#5'41,6Ͳ7,8'6-1+/&148/&,/'$#'8)
a:
appliances and other
Example: TCLs
5000
.#,#)%perature setpoint, θ , and temperature dead-band width,
Machine
δ, (or normalize diverse dead-bands). Divide the deadθ −θ −m θ
meline:
θ −θ
a =1−
=
. (9)
5-6'*0,7'-%
State
band into N /2 temperature intervals. A TCL in a cerθ −θ −m θ
θ −θ −m θ
members.
0
tain temperature interval can be either ON or OFF. Divide
Learning
!
15
16
17
Advantages
•  A case with partially observable states can easily be formulated as
POMDPs (Partially Observable Markov Decision Processes)
•  Model-free approaches can yield to implementations with more
realistic assumptions
Temperature Evolution (Selected 200 TCLs)
20.5
Challenges
•  Relatively large state and decision spaces
20
Nominal
Temperature
19.5
14
"#$%&'9):.'#+/'&+'/'+,74'6;'+81&*+6&'<8'+81&=,-/%,/1&+'/<1&>)
15
16
17
18
19
20
Time [hours]
!
Acknowledgments
This research is being funded by HP Labs Innovation Research Program
under topic:26 Resource Management in Urban Infrastructure.