This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TETC.2014.2335541, IEEE Transactions on Emerging Topics in Computing
1
User-Centric Demand Response Management in the
Smart Grid with Multiple Providers
Sabita Maharjan, Member, IEEE, Yan Zhang, Senior Member, IEEE, Stein Gjessing Senior Member, IEEE, and
Danny Hin-Kwok Tsang Fellow, IEEE
Abstract—The smart grid is the next generation power grid
with bidirectional communications between the electricity users
and the providers. Demand Response Management is vital in
the smart grid to reduce power generation costs as well as to
lower the users’ electricity bills. In this paper, we introduce
multiple fossil-fuel and multiple renewable energy sources based
utility companies on the supply side, and propose an enduser oriented utility company selection scheme to minimize user
costs. We formulate the problem as a game, incorporating the
uncertainty associated with the power supply of the renewable
sources, and prove that there exists a Nash equilibrium for the
game. To further reduce users’ costs, we develop a joint scheme
by integrating shiftable load scheduling with utility company
selection. We model the joint scheme also as a game, and prove
the existence of a Nash equilibrium for the game. For both
schemes, we propose distributed algorithms for the users to find
the equilibrium of the game using only local information. We
evaluate our schemes and compare their performances to two
other approaches. The results show that our joint utility company
selection and shiftable load scheduling scheme incurs the least
cost to the users.
Index Terms—Demand response management, discrete time
Markov chain, game theory, Nash equilibrium, renewable energy
sources, smart grid, user cost.
I. I NTRODUCTION
A smart grid is an intelligent electricity network that makes
use of the advanced information, control, and communication
technologies to save energy, reduce cost and enhance reliability
and transparency of the grid [1], [2]. The realization of the
smart grid is geared by the goal of effective management
of power supply and demand loads. Demand response (DR)
is the response system of the end-users to manage their
consumption of electricity in response to supply conditions.
Demand response management (DRM) smooths out the system
power demand profile across time and provides short-term
reliability benefits as it can offer load relief to resolve system
and/or local capacity constraints. Thus, from the operator
aspect, DRM reduces the cost of operating the grid, while
from the user aspect it lowers their bills. In the smart grid,
bidirectional communications between energy providers and
end users, will greatly enhance DR capabilities of the whole
system. E.g., consumers will be able to control and manage
S. Maharjan is with Simula Research Laboratory, Norway, Martin Linges
Vei 15, Fornebu, Norway. Email: sabita@simula.no
Y. Zhang and S. Gjessing are with Simula Research Laboratory and
Department of Informatics, University of Oslo, Gaustadalleen 23, Oslo,
Norway. Email: yanzhang@ieee.org, steing@ifi.uio.no
D.H.K. Tsang is with Department of Electronic and Computer Engineering, Hongkong University of Science and Technology, Hongkong. Email:
eetsang@ece.ust.hk
their electricity usage, and the providers can plan and manage
the generation and supply of power in a more structured
and efficient manner. Thus, power generation, distribution
and consumption are envisaged to be more efficient, more
economical and more reliable in the smart grid.
The smart grid aims to incorporate components such as
renewable energy sources, distributed micro-generators and
energy storage entities like plug-in electric vehicles and batteries. With the internationalization of energy market and the
incorporation of new renewable energy sources (e.g., wind
and solar power) in the smart grid, users will have more
options in terms of choosing providers or the type or energy
they use. With this motivation, we include multiple renewable
energy sources on the supply side, in addition to the traditional
fossil-fuel based sources. While existing literature has focused
mainly on minimizing the power generation cost of a single
provider or utility company (UC) by scheduling the shiftable
load of the end-users to different time-slots, we introduce
a utility company selection (UCS) approach for DRM, to
minimize the costs of the users, considering the uncertainty
about the states of the renewable energy sources. To this end,
in order to reduce the costs of the users further, we develop a
joint scheme by combining UCS and conventional shiftable
load scheduling (SLS) schemes. This joint scheme is very
generic, and is useful for residential, commercial as well as
industrial electricity consumers.
The cost of a user depends not only on its own demand
and scheduling, but also on the demands and scheduling of
other users, who choose the same UC. Moreover, in a multiple
UC scenario which is the focus of this paper, the cost of
each user varies according to the cost of power generation
of many sources, and the selection of the UC by other
users, thus making the interactions between users and sources,
and among users, complicated. This complicated coupling
makes game theory an appealing approach to model users’
costs in this scenario. The users schedule their demands by
optimally choosing the UCs and by optimally scheduling their
shiftable appliances in order to minimize their payments to
the providers. We model these interactions among the users
as games, and we characterize the Nash equilibrium (NE)
solution of the games.
Our contributions in this work are as follows.
1) We study DRM with multiple UCs with a user-centric
approach, incorporating uncertainty regarding the power
supply from the renewable energy sources.
2) We propose a UC selection game among users, and
develop a distributed algorithm to find the NE of the
2168-6750 (c) 2013 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE
permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TETC.2014.2335541, IEEE Transactions on Emerging Topics in Computing
2
game using only local information.
3) We introduce a joint UC selection and load scheduling
game, to further improve the users’ costs, and propose
a distributed algorithm to reach the NE of the game.
The rest of the paper is organized as follows: Related work
is described in Section II. The system model that includes the
generator availability model and the cost model, is described
in Section III. In Section IV, we introduce a utility company
selection game among the users and design a distributed
algorithm for the game to converge to the NE. In Section
V, we develop a joint scheme by integrating shiftable load
scheduling with the utility company selection scheme, and
propose a distributed algorithm to find the NE of the game
using only local information. In Section VI we present and
discuss our results and Section VII concludes the paper.
II. R ELATED W ORK
There are several studies on DRM in the smart grid [3][7]. In [3], the authors formulated the energy consumption
scheduling problem as a game among the consumers for
increasing and strictly convex cost functions. In [4], the
authors considered a distributed system where price is modeled
by its dependence on the overall system load. Based on the
price information, the users adapt their demands to maximize
their own utilities. In [5], a robust optimization problem was
formulated to maximize the utility of a consumer, taking into
account price uncertainties at each hour. In [6], the authors
have exploited the awareness of the end-users and proposed
a method to aggregate and manage end-users’ preferences to
maximize energy efficiency and user satisfaction. In [7], a dynamic pricing scheme was proposed to incentivize customers
to achieve an aggregate load profile suitable for providers, and
the demand response problem was investigated for different
levels of information sharing among the consumers in the
smart grid. We observe that [3]-[7] mainly focus on either
only one source or a number of sources/providers treated as
one entity, and the cost minimization problem was solved from
the provider’s perspective. Differently in our study, we include
multiple renewable and non-renewable energy sources based
UCs, and we focus on minimizing the cost for each user.
We notice that there are few studies exploring DRM in the
context of multiple energy sources, incorporating renewable
energy sources on the supply side [10], [12]. However, these
two studies also treat the sources as parts of a single entity,
and are also based on the cost minimization of the supply side.
In our previous work [8], we introduced multiple UCs who
aim to maximize their revenues, and multiple consumers who
aim to maximize their utilities. In [8], we focused our analysis
on a static one time-slot game. In this paper, we deploy a
pricing scheme that is closely related to the cost of power
generation, and we also introduce the temporal aspect of DRM
in the joint scheme.
Recently there has been a drastic increase in the volume of
work considering batteries as storage devices, or solar panels at
buildings to reduce the peak-to-average ratio of power demand,
and thus the cost for the users [14], [15]. The problem we
address in this paper is different. When each user has a battery
or a solar panel, these alternatives are local for each user. In
our study, the renewable energy sources are on the supply
side and the same renewable sources can supply power to
multiple users, which makes the problem more complicated.
Moreover, there has been an increasing interest in integrating
renewable energy sources into the grid with storage units [16],
[17]. Our focus in this paper is different than the analyses in
[16], [17]. While[16], [17] focus on the integration issues of
renewable sources into the grid, we concentrate our analysis
on the market level and provide user-centric solutions to get
economic benefits by making optimal choices.
III. S YSTEM M ODEL
Fig. 1 shows the system model. There are N energy users,
N := {1, 2, ....., N}, and K different UCs, K := {1, 2, ....., K}.
Some of the UCs provide electricity from fossil-fuel based
sources, and some UCs are based on renewable energy sources.
Then, we have, K = K f + Kr , where K f and Kr are the number
of fossil-fuel sources based UCs and renewable energy sources
based UCs, respectively. The fossil-fuel based sources are
available all the time but the price charged to the users by the
corresponding UCs will include the cost due to their pollution
emission. On the other hand, the renewable energy sources
are pollution free but they cannot provide a stable supply of
power.
The daily energy consumption schedule of each user is
divided into different time-slots. Each user intends to select the
UC that minimizes the cost it pays to the UCs. In each timeslot, each user can select only one UC, but in case of deficit
power from a renewable energy based UC, the demand will
be served by using another generator. There is bidirectional
communication between the providers and the consumers.
The consumers form a Neighborhood Area Network (NAN).
The data from the users’ side (total demand from each UC),
and from the supply side (price) is exchanged through the
Neighborhood Area Network Gateway (NAN-GW) as shown
in Figure 1. Users can communicate through a Local Area
Network (LAN), as described in [3].
In the current power system and the electricity market, the interactions among the generators, energy service
providers/UCs, transmission companies (transcos), distribution
network operator (DNO), distribution system operator (DSO),
etc. are quite complex. Although our model might be simplified in this respect, since the transmission and distribution
costs are normally fixed, the market price and the consumerdemands are mainly affected by the generation costs from the
supply side and consumers’ parameters from the demand side.
Thus our model leads to simplified decision problems, which
allows detailed analysis and yields some useful insights about
the interactions, without losing generality of the grid structure.
Moreover, considerable volume of work has used this kind
of scenario with electricity consumers and providers/UCs
interacting with each other without including transcos, DNOs,
DSOs, etc. [3], [18], [19], [20], in the context of DRM in the
smart grid.
There are two components associated with the price charged
by a UC to the consumers: the cost of power generation and
the availability of the renewable energy sources.
2168-6750 (c) 2013 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE
permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TETC.2014.2335541, IEEE Transactions on Emerging Topics in Computing
3
!"#-%(&'
!"#$%$&'
!"#$%(&'
!"#-%$&'
)*+*,''
-'
)*+*,''
$'
!"#-%-&'
)*+*,''
('
....'
!"#$%-&'
!"#(%(&'
!"#(%$&'
!"#(%-&'
Fig. 2. DTMC for renewable energy based utility company k
Fig. 1. Communication between the utility companies and the users
A. Cost of Power Generation
We choose the cost function of each UC such that the
following properties are satisfied for both fossil-fuel based and
renewable energy sources based UCs.
Property 1. The cost functions and hence the prices charged
to the users are increasing with higher demand for any timeslot, i.e., if d k,tot , d˜k,tot ≥ 0, are the total demands from UC
k, (k ∈ K ) such that d k,tot < d˜k,tot , and if CUC,k (d k,tot ) and
CUC,k (d˜k,tot ) are the corresponding costs of power generation,
then, CUC,k (d k,tot ) < CUC,k (d˜k,tot ), ∀ d k,tot < d˜k,tot , ∀k ∈ K .
Property 2. The cost functions and hence the prices
charged to the users are strictly convex in any timeslot, i.e., CUC,k (θ d k,tot + (1 − θ )d˜k,tot ) < θCUC,k (d k,tot ) + (1 −
θ )CUC,k (d˜k,tot ), ∀k ∈ K , where d k,tot , d˜k,tot ≥ 0, and 0 < θ < 1
is any real number.
Examples of cost functions that satisfy these properties
can be the two-step conservation rate model, quadratic cost
function, etc. These types of cost functions are of significant
importance and are realistic too, since they have been deployed
by many big power plants such as BC Hydro [9], and they can
be closely related to the cost of power generation. Hence, we
employ a quadratic cost function for the UCs.
Let the coefficients of UC k for the cost of power generation be βk,2,p , βk,1,p and βk,0,p , respectively. Let the pollution coefficients be αk,2 , αk,1 and αk,0 , respectively. Then
βk,2 := βk,2,p + αk2 , βk,1 := βk,1,p + αk,1 and βk,0 := βk,0,p + αk,0
are the coefficients of the cost function of UC k. Note that for a
renewable energy sources based UC, k, αk,2 = αk,1 = αk,0 = 0.
Let dnh denote the demand of user n at time-slot h, h ∈ H ,
where H := {1, 2, . . . , H}. Let gn := {ghn,k , k ∈ K , h ∈ H },
denote the UCS strategy of user n, where ghn,k is the decision
of user n for UC k in time-slot h, i.e.,
1 if user n selects UC k in time-slot h,
h
gn,k =
(1)
0 otherwise.
We use vectors ghn to represent the strategy of user n for time-
slot h, i..e., ghn := {ghn,k , k ∈ K }.
Suppose user n prefers UC k in time-slot h. Let Nkh :=
{1, 2, . . . , Nkh }, denote the set of users who select UC k, where
Nkh := |Nkh |, is the total number of users who prefer power
from UC k in time-slot h. Then, the total demand from UC k in
time-slot h is ∑m∈N h dmh . The cost of generating this amount
k
of power is
2

h
= βk,2 
CUC,k
∑
m∈Nkh

dmh  + βk,1 

∑
dmh  + βk,0 .
(2)
m∈Nkh
B. Availability of Renewable Energy based Utility Companies
When renewable energy sources such as wind power and
solar power are incorporated into the system, the inherent
uncertainty is added to the supply side. We employ a discrete
time Markov chain (DTMC) to model the power available
with the renewable energy sources based UCs. Although the
choice of the DTMC model is motivated for mathematical
tractability as it facilitates the derivation of the structure of
the optimal strategies for the games, it is able to satisfactorily
represent the transition between different states as has been
shown by [21], [22] for wind power, and by [23], [24], [25] for
solar radiation. Moreover, DTMC has been extensively used to
model the availability of different kinds of renewable energy
sources [10], [24], [25], [26].
The users do not know the exact state of the renewable
energy sources, but they have knowledge about the statistical
behavior of the renewable energy sources based UCs, i.e., the
only information the users have access to, is the steady state
probabilities of the DTMC. The availability of the renewable
energy based UCs is represented as an M state DTMC as
shown in Fig. 2. Let the state of the renewable energy based
UC k (k ∈ Kr , Kr := {1, 2, . . . , Kr }; Kr := |Kr |) be shk at timeslot h. Each state of a UC represents its power supply level.
The power supply state space can be divided into M discrete
levels i.e., shk = mk , (mk ∈ M , M := {1, 2, . . . , M}; M := |M |).
Let the 1 × Kr vector sh := (sh1 , sh2 , ....., shKr ) represent the power
supply state of all renewable energy based UCs, i.e., the system
state in time-slot h. Then, the state space of this vector is
Ωs = {(ω1 , ω2 , ....., ωKr )|ωk = mk , k ∈ Kr }. If the UCs are
independent, the dynamics of sh follows DTMC with transition
0
probability from state vector ω to ω , given by
2168-6750 (c) 2013 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE
permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TETC.2014.2335541, IEEE Transactions on Emerging Topics in Computing
4
If a user chooses a renewable energy based UC but the source
is not available or if the power supply level of the UC is
0
0
0
0
P(ω, ω ) = Pr(sh+1 = ω |sh = ω) = ∏ Pk (ωk , ωk ), ∀ω, ω ∈ Ωs , not adequate to meet the demand, the total available power is
k=1
supplied proportionally to the users’ demands. The amount of
(3)
power that can not be supplied by the source (deficit power)
0
th
where ωk and ωk are the k element of the state vector ω and
will have to be generated by and bought from either a source
0
0
ω , respectively, and Pk (ωk , ωk ) represents the state transition that can immediately generate the deficit power, or from a
probability matrix of UC k.
reserve. In either case, the power from this additional source
For a time homogeneous DTMC model, the steady state (backup source) is normally costlier than the power from the
probabilities of renewable energy based UC k, (k ∈ Kr ), being committed units. We consider a quadratic cost function for this
in state m at time t can be obtained as follows. Let us define additional source too, and denote the coefficients of its cost
function as {βres,2 , βres,1 , βres,0 }.
0
0
πkt (ω ) = Pr(stk = ω ), ∀k ∈ Kr .
(4)
Let phk,mk denote the steady state probability of the renewable energy based UC k, k ∈ Kr , being in state mk , mk ∈
Then,
are obtained from πkh =
t+1
t
πk = πk P,
(5) Mh. The steady state probabilities
h
{pk,m , mk ∈ M , h ∈ H }. Let Savl, UC,k,mk be the available
where πk is a row vector. The stationary distribution is
power of UC k in time-slot h if it is in state mk . Then, the
amount of power supplied by UC k is


πk = lim πkt , if the limit exists.
(6)
t→∞
h
h
 h
Ssupp,
.
UC,k,mk = min Savl, UC,k,mk , ∑ dl
If πk exists, it can be obtained by solving
h
Kr
l∈Nk
0
πk = πk P,
∑0 πk (ω ) = 1.
(7)
ω
The average supply of renewable energy based UC k is
h
Ssupp,
UC,k =
C. Interactions between Utility Companies and Users
∑
mk ∈M
h
phk,mk Ssupp,
UC,k,mk .
(9)
Each residential user has a smart meter that, with the aid of
the bidirectional communication, can perform load scheduling
and UCS optimally in order to minimize the cost for the user.
The data communication between the supply and demand sides
is carried out through a NAN-GW as shown in Fig. 1.
If a user chooses renewable energy based UC k, k ∈ Kr , in
time-slot h i.e. if ghn,k = 1, then, the amount of power that it
gets from k is
!
dnh
h
h
h
Savl, UC,k,mk .
Savl, user,n,mk = min dn ,
∑l∈N h dlh
IV. U TILITY C OMPANY S ELECTION : A G AME
T HEORETICAL A PPROACH
The amount of power that user n gets, averaged over all the
states of UC k is
k
Different users choose the UCs independently. However, the
aim of each user is to minimize the cost that it pays to the
UCs. In this section, we discuss the pricing scheme, i.e., the
cost model for the users, and formulate the UCS game among
the users, for which, we show that a Nash equilibrium (NE)
solution exists. Then, we present a distributed algorithm to
reach a NE and prove that the algorithm converges to a NE.
h
Savl,
user,n =
∑
mk ∈M
h
phk,mk Savl,
user,n,mk .
(10)
If the demand from the renewable energy based UC in timeslot h is higher than the available power in state mk , the deficit
power for user n will be
h
h
h
Sdef,
user,n,mk = dn − Savl, user,n,mk .
(11)
The average deficit for user n will be
h
Sdef,
user,n =
A. Pricing Mechanism: Cost Model for Users
In [3], the authors showed that charging each user in
proportion to its demand leads to minimizing the cost of the
single UC when each user schedules its demands to minimize
its own charges. Thus, in order to motivate the users to help in
the demand side management, the providers employ a pricing
scheme such that the consumers are charged according to the
cost of generating the required power, as in [3].
0
If a user selects a fossil-fuel based UC k ∈ K f , in time-slot
h, i.e., if gh 0 = 1, the price charged to the user for time-slot
n,k
h will be


βk0 ,0


h
(8)
Cuser,n
= βk0 ,2  ∑ dmh  dnh + βk0 ,1 dnh + h .
N0
m∈N h
k
0
k
∑
mk ∈M
h
phk,mk Sdef,
user,n,mk .
(12)
The total amount of deficit power for renewable energy based
UC k in time-slot h is


h

Sdef,
UC,k,mk = max
h

dlh − Savl,
UC,k,mk , 0
∑
l∈Nkh
The deficit power for all renewable sources k ∈ Kr , which are
in states m1 , m2 , . . . mKr , respectively, is supplied by an extra
source such as a gas turbine, which can generate the shortage
power immediately. The total amount of power to be supplied
by the extra source is
h
Sdef,
tot,{m1 ,...mK } =
r
∑
h
Sdef,
UC,k,mk .
(13)
k∈Kr
2168-6750 (c) 2013 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE
permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TETC.2014.2335541, IEEE Transactions on Emerging Topics in Computing
5
Using (13) and the steady-state probabilities in each time-slot,
we can compute the total deficit power from all UCs averaged
over all the states of all the renewable energy based UCs as
h
Sdef,
tot =
∑
m1 ∈M
ph1,m1
∑
m2 ∈M
ph2,m2 · · ·
(14)
r
Therefore, if user n chooses renewable energy based UC k in
time-slot h, the price charged to the user will be
h
h
h
h
Cuser,n
:= βk,2 Ssupp,
UC,k Savl, user,n + βk,1 Savl, user,n +
h
h
h
+ βres,2 Sdef,
tot Sdef, user,n + βres,1 Sdef, user,n +
βres,0
,
h
Nres
(15)
B. Utility Company Selection Game Among Users
The centralized cost minimization problem for users can be
written as
min
∑
Cuser,n =
s.t.
min
∑ ∑
{gn ,n∈N } n∈N h∈H
h
Cuser,n
,
ghn,k dnh ≤ Pk,max , ∀k ∈ K f ,
∑
(16)
(17)
n∈Nkh
where Pk,max is the capacity of fossil-fuel based UC k, k ∈ K f .
Since, each user is selfish, the objective of each user is to
minimize its own cost. The optimization problem for user n
given the strategies of other users, (OPuser,n ) is
min
gn
∑
h
Cuser,n
, s.t. (17).
(18)
h∈H
For the UCS game, the power consumptions and thus the
costs are decoupled in time. Hence, minimizing the total price
over a time span of H time-slots is equivalent to minimizing
the price in each time-slot, i.e., (OPuser,n ) takes the form
min
{ghn,k ,k∈K }
h
, ∀h ∈ H , s.t. (17).
Cuser,n
For a finite N-user game, a NE is a solution at which a
player cannot improve its payoff by deviating alone from the
equilibrium, given the strategies of all other players. For the
UCS game, the NE is any strategy set (g∗n , g∗−n ) at which
uuser,n (g∗n , g∗−n ) ≥ uuser,n (gn , g∗−n ), ∀n ∈ N ,
βk,0
Nkh
where Nres is the total number of users whose deficit powers
are served by the additional generator. Note that the above
formulation can easily accommodate the scenario with more
than one kind of backup sources. In this scenario, a possible
approach is to choose the cheapest type of backup generator
if it can make up the deficit, otherwise multiple generators
supply the deficit power.
{gn ,n∈N } n∈N
Payoffs: uuser,n (gn , g−n ), for each user n ∈ N , where
uuser,n (gn , g−n ) = −Cuser,n (gn , g−n ), and g−n is the strategy of all users other than n.
C. Existence of Nash equilibrium
h
phKr ,mKr Sdef,
tot,{m1 ,...mK } .
∑
mKr ∈M
•
(19)
The choice of UCs by consumers depends not only on their
own demands but also on the demands and UCS strategies of
the other consumers. This implies that the strategy of each user
is coupled with the strategies of the other users. As each user
is concerned about maximizing its own payoff, game theory
[11] is a natural fit to model the behavior of the users in this
case.
The components of the game are as follows:
• Players: Users in the set N .
• Strategies: The strategy set of user n is the selection of
UCs for all time-slots in a day, gn . The strategy space is
defined as g := {g1 × g2 × . . . gN }.
where g∗n is the optimal UCS strategy of user n, for given optimal strategies of other users g∗−n , and uuser,n (.) = −Cuser,n (.).
Theorem 1. The UCS game between the consumers is a
concave N-user game and a Nash equilibrium exists for the
game.
Proof: For the UCS game for one whole day, since there
is no coupling in time, it is sufficient to prove that an NE
exists for the game for each time-slot.
For time-slot h, the demand of user n from UC k can
h = gh d h = {0, d h }. If k is a renewable
be written as dn,k
n
n,k n
h
from k, and
energy based UC, user n will get Savl,user,n
h
Sdef,user,n from the corresponding additional source. The demand of users other than n who also prefer UC k, can
be represented as dh−n,k = ghm,k dmh , ∀m ∈ Nkh , m 6= n. Let us
write the payoff of user n, who gets power from UC k, as
h , . . . , d h , . . . , d h ). Since the cost
uuser,n (dhall,k ) = uuser,n (d1,k
n,k
Nkh ,k
functions are increasing and strictly convex for each UC, the
h , dh
h
payoff function uuser,n (dn,k
−n,k ) is strictly concave w.r.t. dn,k
h
h
h , . . . , dh
for given (d1,k
n−1,k , dn+1,k , . . . , d h 0 ) and is continuous
Nk ,k
in dhall,k . This holds for all users n ∈ N , who prefer a different
0
0
UC k ∈ K , k 6= k, too. Thus, the UCS game is a concave Nuser game. An NE exists for a concave N-user game [13].
The game will be a concave game even if a part of the
demand of user n from k ∈ Kr , is supplied by the additional
generator, since the sum of the convex functions is also convex.
Consequently, an NE exists for this game.
D. Distributed Algorithm to Converge to Nash Equilibrium
Equation (16)-(17) can be solved in a centralized manner
by using convex programming techniques such as interior
point method (IPM). Nonetheless, a solution that can be
implemented and updated autonomously to accommodate the
changes within the demand side, is of arguably higher value
compared to a centralized solution. We are therefore interested
in presenting a distributed solution for (16)-(17). Notably, (18)
or (19) can be solved by each user locally provided the users
can select their strategies sequentially. Exploiting this fact,
next, we propose a distributed algorithm to implement the UCS
scheme.
Let, gn,t = {ghn,t , h ∈ H } denote the UCS strategy of user
n. Each user starts by randomly selecting a UC for each timeslot. In the next iteration, one of the consumers solves the
local optimization problem (19), ∀h ∈ H , and updates its
strategy for the whole day. Then, another user gets its turn to
2168-6750 (c) 2013 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE
permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TETC.2014.2335541, IEEE Transactions on Emerging Topics in Computing
6
solve its local optimization problem and to update its choice.
If in one complete round of updating, the selection of UCs
from all users do not change, the point is an equilibrium.
We call this user-centric algorithm, Algorithm 1. The strategy
update iterations should be performed until the strategies of
all users do not change compared to their strategies in the
previous iterations. Note that strategy update should be done
asynchronously by each user. The NAN-GW can be used to
send the control signal (e.g., a 1-bit signal) for asynchronously
giving turn to each user to update their strategies. Users’
strategies can be updated randomly, thus making the optimal
strategies independent of the order of their turns.
Next, we display Algorithm 1 in more detail. Note that
the term iteration ’t’ and the term time-slot ’h’ are used to
represent different time scales. An optimal UCS game for
one time-slot needs many iterations for the users’ strategy to
converge (see results in Section VI).
Algorithm 1 : Distributed Algorithm for NE
1: t=1; Arbitrarily choose gn,t = {ghn,t , h ∈ H }, ∀n ∈ N ,
and announce the starting {gn,t , n ∈ N }.
2: t = t + 1
3: Randomly choose user n, n ∈ N .
4: User n: Solve optimization problem (19), ∀h ∈ H
to find optimal gn,t .
5: If gn,t changes compared to gn,t−1 ,
6:
Send a control message to announce the new gn,t
to other smart meters.
7: end
8: If any user has not updated its strategy for iteration t,
9:
Send a control message to user l who has not
updated its strategy for iteration t, l ∈ N , l 6= n.
10: Update n = l. User n, go to Step 4.
11: end
12: If gn,t changes for at least one user,
13: Go to Step 2.
14: end
Theorem 2. Algorithm 1 converges to an NE as long as the
individual strategies are updated asynchronously.
Proof: Line 4 in Algorithm 1 finds the best response.
If users play the best responses using this algorithm in a
sequential manner, the cost of each user decreases or remains
unchanged every time a user updates its strategy. However, the
cost cannot decrease below a certain value since Cuser,n ≥ 0.
This implies that the algorithm converges to a fixed point, beyond which the cost can not be improved, for given strategies
of other users. To prove that this fixed point is an NE, let
∗
(g∗n , g∗−n ) as the cost for user n at the fixed
us denote Cuser,n
point, which corresponds to the strategy set (g∗n , g∗−n ) and let
0
0
Cuser,n (gn , g∗−n ) be the cost for the same user for the strategy
0
0
set (gn , g∗−n ) where g∗n 6= gn . By definition, if (g∗n , g∗−n ) is a
fixed point of the UCS game, then,
0
0
∗
Cuser,n
(g∗n , g∗−n ) ≤ Cuser,n (gn , g∗−n ),
0
0
⇒ u∗user,n (g∗n , g∗−n ) ≥ uuser,n (gn , g∗−n ).
Thus, the fixed point (g∗n , g∗−n ) is an NE of the game. If the
NE is unique, the algorithm converges to the unique NE. If
the game possesses multiple Nash equilibria, the algorithm
converges to one of them depending on the initially selected
strategies.
V. L OAD S CHEDULING AND U TILITY C OMPANY
S ELECTION : A J OINT A PPROACH
Scheduling in time is a conventional approach used to
reduce the Peak to Average Ratio (PAR) of a single UC which
consequently reduces the cost of generation for the (single) UC
and also the user bills. However, it should be noted that not
all users prefer saving on their bills by shifting their flexible
load. Such time-scheduling schemes are useful only for those
users who have a considerable amount of flexible load. In
fact, many consumers may not even have any shiftable load,
or they may have some flexible appliances but they may prefer
to use them at certain hours rather than scheduling them at any
time for the sake of reducing costs. In addition, the electricity
requirements of the commercial and the industrial consumers
are usually rather strict, and scheduling in time may not be a
preferable option for them. On the contrary, if the users have
shiftable appliances, their costs can be reduced significantly
by scheduling their shiftable appliances in different time-slots.
We accommodate the needs and goals of all these types of
consumers together into one framework in our proposed DRM
solution. To this end, we introduce a joint scheme where users
schedule their shiftable appliances in different time-slots and
also choose UCs in each time-slot.
As mentioned in Section I, we would like to emphasize that
the UCS scheme incorporating the uncertainty in the supply
side associated with the renewable energy resources, is one
of our main contributions in this paper. The joint scheme is
another contribution of this work which further reduces the
cost for the users. The joint scheme is not a straightforward
extension of the UCS game. The joint scheme is comprised
of two levels of games: the optimal SLS in time for the
whole day and the optimal UCS for all time-slots in a day.
It is worth noting that the two scenarios have rather different
application domains. The UCS game finds its extensive use
in a scenario where the end-users have relatively strict power
requirements such as commercial and/or industrial users who
have either no or very little flexible demands. On the other
hand, the scheduling in time is more useful for users with
higher proportion of shiftable appliances, such as residential
users. The joint scheme is very generic and is suitable for
residential, commercial as well as industrial users.
In addition to a non-shiftable demand in each time-slot, we
now incorporate some shiftable power requirement for each
h , d h be the non-shiftable demand in timeconsumer. Let dn,ns
n,s
slot h, and part of the shiftable demand of user n, scheduled in
time-slot h, respectively, for a given scheduling in time. Here,
the subscript s represents shiftable, n is the user index and ns
indicates non-shiftable. The total amount of shiftable power
per day is fixed but its scheduling in time and the selection
of UCs is the strategy of each consumer in this case. We
incorporate the coupling among different time-slots for the
2168-6750 (c) 2013 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE
permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TETC.2014.2335541, IEEE Transactions on Emerging Topics in Computing
7
renewable energy sources based UC k in time-slot h, the price
charged to the user will be
shiftable load of consumer n as
τn,2
∑
h
h
dn,s
= d¯n,s ; dn,s
≥ 0, ∀h ∈ [τn,1 τn,2 ],
h
h
h
h
Cuser,n
:= βk,2 Ssupp,
UC,k Savl, user,n + βk,1 Savl, user,n +
h=τn,1
where τn,1 , τn,2 are the beginning and the end time-slots,
respectively, for the shiftable demand of user n.
A. Pricing Mechanism: Cost Model for Users
For this scheme, the strategy of each consumer is {xn , gn },
where {xn } := {xnh , h ∈ H }, is a demand scheduling vector
h + dh ,
that incorporates its shiftable load d¯n,s , i.e., xnh = dn,ns
n,s
is the power requirement of user n in time-slot h, and gn :=
{ghn,k , h ∈ H , k ∈ K } is the UCS strategy of user n for one
complete day, for a given schedule of the shiftable appliances.
0
If a user selects a fossil-fuel based UC k ∈ K f , in time-slot
h, the price charged to the user for time-slot h will be


βk0 ,0

h
h h
h
(20)
Cuser,n
= βk0 ,2  ∑ xm
 xn + βk0 ,1 xn + h .
N0
m∈N h
k
If a user chooses renewable energy based UC k in time-slot
h, the price charged on user n is calculated as follows. The
amount of power supplied by UC k, if it is in state mk , is


h
 h
Ssupp,
UC,k,mk = min Savl, UC,k,mk ,
∑
xlh  .
(21)
l∈Nkh
h
where Savl,
UC,k,mk is the corresponding available power of UC
k in state mk . The amount of power that user n gets from UC
k is
!
xnh
h
h
h
(22)
Savl, UC,k,mk .
Savl, user,n,mk = min xn ,
∑l∈N h xlh
k
If the demand from the renewable energy based UC k in timeslot h is higher than the available power in state mk , the deficit
power for user n will be
h
h
h
Sdef,
user,n,mk = xn − Savl, user,n,mk .
(23)
h
(Ssupp,
UC,k ),
The average supply of UC k
the average power
h
that user n gets from it (Savl, user,n ), and the average deficit for
h
user n (Sdef,
user,n ), can be obtained using (21) in (9), (22) in
(10), and (23) in (12), respectively.
The total deficit power for renewable energy based UC k in
time-slot h if the source is in state mk , is


∑
h

xlh − Savl,
UC,k,mk , 0
(24)
l∈Nkh
The total amount of power to be generated by the extra
source to supply the deficit power for all the renewable sources
when the UCs are in states m1 , m2 , . . . , mKr , is denoted as
h
Sdef,
tot,{m1 ,...mKr } , and can be calculated by substituting (24)
into (13). Similarly, the total deficit power for all renewable
energy based UCs averaged over all the states can be obtained
h
by using Sdef,
tot,{m ,...mK } in (14). Therefore, if user n chooses
1
r
βres,0
.
h
Nres
(25)
B. Utility Company Selection and Shiftable Load Scheduling
Game among Users
The objective for a centralized approach would be to minimize the total daily cost of all users i.e.,
min
∑
{gn ,xn ,n∈N } n∈N
Cuser,n =
min
∑ ∑
{gn ,xn ,n∈N } n∈N h∈H
h
Cuser,n
(26)
τn,2
s.t.
∑
h
dn,s
= d¯n,s , ∀n ∈ N ,
(27)
ghn,k xnh ≤ Pk,max , ∀k ∈ K f .
(28)
h=τn,1
∑
n∈Nkh
k
0
h

Sdef,
UC,k,mk = max
h
h
h
+ βres,2 Sdef,
tot Sdef, user,n + βres,1 Sdef, user,n +
βk,0
Nkh
The objective of each user, is to minimize its own cost given
the strategy of other users, i.e., the optimization problem for
user n (OPuser,n ) in this case, is
min
∑
{gn ,xn } h∈H
h
Cuser,n
, s.t. (27), (28).
(29)
In the UCS game, the coupling of the strategies was only in
terms of the selected UCs. There was no coupling in time. In
the joint UCS and SLS game, although the UCs are chosen for
each time-slot in an independent manner, since each consumer
also optimizes its choice of the UCs based on the scheduling
of its shiftable load, the strategies of the consumers are thus
coupled in both time and the UCs. We design the interactions
as a joint game in this case, where the players are the
consumers: N , the strategies are {xn , gn }, ∀n ∈ K , and their
payoffs are uuser,n (xn , gn , x−n , g−n ) = −Cuser,n (xn , gn , x−n , g−n ).
C. Existence of Nash Equilibrium
For the joint UCS and SLS game, the NE is defined as the
strategy set {x∗ := {x∗n , x∗−n }, g∗ := {g∗n , g∗−n }} such that
Cuser,n (x∗n , x∗−n , g∗n , g∗−n ) ≤ Cuser,n (xn , x∗−n , gn , g∗−n ),
Theorem 3. The joint UCS and SLS game is a concave N-user
game and a Nash equilibrium exists for this game.
Proof: The payoff is given by the function uuser,n (x, g) =
uuser,n ({x1 , . . . , xn , . . . , xNk }, {g1 , . . . , gn , . . . , gNk }) for user n,
where (x1 , . . . , xn , . . . xNk ) and (g1 , . . . , gn , . . . gNk ) are the vectors that represent the demands and the UCS of users
1, . . . , n, . . . , Nk , all of whom get power from UC k, (k ∈ K ,
here k may indicate different UCs at different time-slots).
For a given scheduling of the shiftable load of the users, an
NE exists for the UCS game for each time-slot, as shown in
Section IV-C. Thus we need to prove that an NE exists only
for the scheduling game.
Since the cost function of each UC is increasing and
strictly convex, the payoff function uuser,n (xn , x−n ) is strictly
2168-6750 (c) 2013 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE
permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TETC.2014.2335541, IEEE Transactions on Emerging Topics in Computing
8
concave w.r.t. xn . For xn ∈ x, where x = x1 × x2 × . . . × xNk ,
uuser,n (x) is continuous in x and is concave in xn for given
(x1 , . . . , xn−1 , xn+1 , . . . , xNk ). This holds for all users n ∈ N
selecting different time-slots h ∈ H for their load scheduling
from the same UC. Hence, the load scheduling game is a
concave N-user game and the existence of an NE for such a
game is straightforward [13].
D. Distributed Algorithm to Find Nash Equilibrium
in an asynchronous manner, the cost of each user decreases
or remains unchanged every time a user updates its strategy.
However, the cost cannot decrease below a certain value because Cuser,n ≥ 0. This implies that the algorithm converges to
∗
a fixed point. Let us denote Cuser,n
(x∗n , x∗−n , g∗n , g∗−n ) as the cost
for user n at the fixed point, which corresponds to the strategy
0
0
0
space {x∗n , x∗−n , g∗n , g∗−n } and let Cuser,n (xn , x∗−n , gn , g∗−n ) be the
0
0
cost for the same user for the strategy space {xn , x∗−n , gn , g∗−n }
0
0
where {x∗n 6= xn , g∗n 6= gn }. By definition, if (xn , x∗−n , g∗n , g∗−n )
is a fixed point of the game, then,
0
Problem (26)-(28) can be solved in a centralized setting by
using convex programming techniques such as IPM. Notably,
(29) represents a local optimization problem for each user
given the strategies of other users. Therefore, each user can
reach the NE of the game starting from some arbitrary initial
selection vectors {xn , gn }. We devise a user-centric distributed
algorithm for each user to solve (29) locally in order to find
its optimal strategy. We name this algorithm, Algorithm 2,
which follows, and is described next.
Each user starts with a random scheduling of the shiftable
appliances and by randomly selecting a generator, for each
time-slot. To start the next iteration, a 1-bit control information
is sent to one of the users by the NAN-GW to update its
strategy. The user who gets the control signal solves the local
optimization problem (29) for given {x−n , g−n } and updates
its strategy {xn , gn }. Then, another user is chosen randomly
to update its strategy. The process stops when the strategies
of all users ({xn , gn , n ∈ N }) do not change.
Algorithm 2 : Distributed Algorithm for the Joint Scheme
1: t = 1; Arbitrarily choose xn,t = {xhn,t }, gn,t = {ghn,t },
∀h ∈ H , such that (27), (28) is satisfied ∀n ∈ N , and
announce the initial {xn,t , gn,t , n ∈ N }.
2: t = t + 1
3: Randomly choose user n, n ∈ N .
4: User n: Solve (29) to find optimal {xn,t , gn,t }
for given strategies of other users.
5: If {xn,t , gn,t } changes compared to {xn,t−1 , gn,t−1 },
6:
Send a control message to announce the new
{xn,t , gn,t } to other smart meters.
7: end
8: If any user has not updated its strategy for iteration t,
9:
Send a control message user l who has not
updated its strategy for iteration t, l ∈ N , l 6= n.
10: Update n = l. User n, go to Step 4.
11: end
12: If {xn,t gn,t } changes compared to {xn,t−1 , gn,t−1 }
for at least one user,
13:
Go to Step 2.
14: end
Theorem 4. Algorithm 2 converges to an NE as long as the
updating of the individual strategies is done sequentially.
Proof: Line 4 in Algorithm 2 finds the best response. If
users play the best responses sequentially using this algorithm
0
0
∗
Cuser,n
(x∗n , x∗−n , g∗n , g∗−n ) ≤ Cuser,n (xn , x∗−n , gn , g∗−n ).
This implies that the fixed point (x∗n , x∗−n , g∗n , g∗−n ) is an NE
of the game. If the NE is unique, the algorithm converges to
the unique NE. If there exists multiple Nash equilibria, the
algorithm converges to one of them depending on the initially
selected strategies.
If the energy consumption needs of all users is the same for
different days, theorems 3 and 4 imply that starting from any
initial point, Algorithm 2 converges to an optimal UCS and
SLS solution for each day, without having to recalculate the
solution everyday. If however, the energy consumption needs
for the users are different for different days, then Algorithm 2
will converge to different optimal UCS and SLS solutions for
different days. We note that even in the later case, the optimal
solutions can be computed at least a day before as long as the
users know their total demands and the shiftable demands for
the next day, which makes Algorithm 2 a practical approach.
The same applies to Algorithm 1 discussed in Section IV also.
Since both Algorithm 1 and Algorithm 2 require sequential
update of the strategies, a natural concern may arise regarding
the practicality of the algorithms for large population of
consumers. For this scenario, users can be grouped based on
their electricity consumption needs or geographical proximity
or some other relevant condition, and the optimal solutions
should be calculated for those groups. Within each group,
a predefined scheduling scheme can be deployed or further
optimizations can be carried out locally.
VI. N UMERICAL R ESULTS
In this section we show the convergence of Algorithms
1 and 2, and evaluate and compare the performance of the
algorithms. The equilibrium of the game will certainly depend
on the values of the parameters. Nonetheless, for the purpose
of illustration, we use a certain set of parameters in all analyses
in this section. Note that the value of the results may change
with a different set of parameters, but the pattern of the results
will be similar. We consider 30 users (N = 30) and 4 UCs
(K = 4). UC 1 and 2 are fossil-fuel based, and UCs 3 and 4
are renewable energy based. We divide the daily consumption
of users into 24 hours. The renewable energy based UCs have
three states: state 1 is the OFF state i.e. no power available,
state 2 means it has a power of 45 KW and state 3 means it
has a power of 90 KW. The capacity of UCs 1 and 2 used for
the analysis are P1,max = P2,max = 500KW .
Each user has some non-shiftable appliances such as refrigerators, bulbs, heating/cooling units, electric stoves, televisions, computers, etc., and some shiftable appliances such
2168-6750 (c) 2013 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE
permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TETC.2014.2335541, IEEE Transactions on Emerging Topics in Computing
for both renewable energy based UCs. We chose this matrix
such that it is closely related to the transition probability
matrix used in [10], but we slightly modified the values since
[10] employs the matrix for 2 states. Note that the transition
probability matrix was chosen as above for illustrating the
performance of the proposed schemes. Choosing different
values for the two renewable energy based UCs may alter the
cost values but the pattern of the results will persist.
A. Distributed Algorithm for Utility Company Selection
Fig. 3 depicts the daily cost paid by 10 randomly selected
consumers (out of 30), and shows that Algorithm 1 converges.
This figure represents the convergence of the cost of each user
to the NE for a 24 hours period without using shiftable load.
The convergence is within 3 iterations. Note that one iteration
here means one round of strategy updating for all users. Fig.
4 depicts the daily cost for 10 randomly selected users with
and without employing optimal UCS strategy. It is evident
that the cost decreases for each user when the UCS scheme is
employed. E.g., for consumer 3, the daily cost reduces from
21.2 to 14, i.e., the UCS scheme results in an improvement
of about 34% for this consumer.
B. Distributed Algorithm for Utility Company Selection and
Shiftable Load Scheduling
We added shiftable demands of 20 KWh to all the users such
that each user needs this amount of energy in 5 hours. Each
user schedules this demand in time. Fig. 5 shows the total cost
of each user for a 24-hour period calculated using Algorithm
2. The figure shows that the algorithm converges very fast,
in about 5 iterations. Note that one iteration here means one
5
10
5
10
5
10
5
10
5
10
Cost($) Cost($) Cost($) Cost($) Cost($)
15
10
5
0
30
20
10
0
15
10
5
0
20
10
0
0
15
10
5
0
20
15
10
5
0
20
15
10
0
10
8
6
0
30
20
10
0
15
10
5
0
5
10
5
10
5
10
5
10
5
Iterations
10
Fig. 3. Convergence of the costs of 10 randomly selected users to NE for a
30-users utility company selection game
30
Without UCS and SLS
25
Daily cost ($)
as dishwashers, laundry machines, electrical vehicles, Plug-in
hybrid electric vehicles (PHEVs), etc. The power requirement
for these appliances are typical standard values, e.g., 1.32 3.96 KWh for refrigerators (depending on the number of refrigerators a household has), 1-2 KWh for bulbs, 6.4-9.6 KWh
for heating, 3.9 KWh for electric stoves, 1.1-2.0 KWh for
televisions/computers, 1.44 KWh for a dish-washer, 3.43 KWh
for a laundry machine and 9.9 KWh for PHEVs. The power
requirement for the non-shiftable appliances was generated
using a uniform random distribution over the typical values mentioned above. For UCs 1 and 2, we used β1,2 =
0.3 cents, β1,1 = β1,0 = 0 as in [3], and β2,2 = 0.1 cents, β2,1 =
0.5 cents, β2,0 = 0 as in [18]. Similar values were assigned to
the pollution coefficients, i.e., αk,2 = βk,2 , αk,1 = βk,1 , αk,0 =
βk,0 for k = 1, 2. For the renewable energy based UCs, we
considered β3,2 = 0.1 cents, β3,1 = 0.3 cents, β3,0 = 0.3 cents,
β4,2 = 0.05 cents, β4,1 = 0.3 cents, β4,0 = 0.2 cents. unless otherwise stated. Note that we have chosen β1,2 , β2,2 ≥ β3,2 , β4,2 ,
and β1,0 , β2,0 β3,0 , β4,0 , assuming that the production cost
is dominant for fossil-fuel based generators and the initial
installation cost is higher for renewable energy sources. The
transition probability matrix used for the numerical analysis is


0.3 0.4 0.3
P = 0.2 0.4 0.4 ,
0.1 0.6 0.3
Cost($) Cost($) Cost($) Cost($) Cost($)
9
With UCS and SLS
20
15
10
5
0
2
4
6
User
8
10
Fig. 4. Comparison of the daily costs of 10 randomly selected users with and
without employing optimal utility company selection strategies
complete round of strategy updates for all consumers. Fig.
6 depicts the total cost of all users for a 24-hour period.
The figure shows four curves. The curve with ’+’ markers
is the cost when neither the optimal UCS nor the optimal SLS
is employed. The dashed line shows the scheme where only
optimal SLS is performed. The dotted line with square markers
shows the performance of the UCS scheme. The solid line
shows the performance of our joint scheme where both optimal
SLS and optimal UCS are employed. The fourth curve (solid
line) shows that Algorithm 2 converges within 5 iterations.
We observe that the SLS and the UCS schemes reduce the cost
compared to the case when no scheduling is done, from about
485$ to about 420$, and 280$, respectively, i.e., by 13.4% and
42.2%, respectively. Our joint scheme incurs the least total
cost compared to the remaining three schemes. It reduces the
cost paid by the users to about 230$, i.e., by 10.3% compared
to the UCS scheme, by 39.17% compared to the SLS scheme
and by 52.5% compared to the scheme without using UCS and
SLS. An interesting observation from this figure is that optimal
UCS can yield drastic savings even for those consumers who
do not have a big proportion of shiftable appliances, or those
who prefer not to shift their appliances, if the consumers have
options to choose different providers or UCs.
Fig. 7 shows the total daily cost paid by 10 randomly
selected consumers with and without using the joint scheme.
The figure clearly indicates that the daily cost is reduced
significantly for every user. E.g., for consumer 3, the daily
cost reduces from about 21.22 to about 11.5, a reduction of
about 45.8%.
2168-6750 (c) 2013 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE
permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TETC.2014.2335541, IEEE Transactions on Emerging Topics in Computing
10
10
5
10
5
10
5
10
5
10
20
10
0
0
20
10
0
0
10
8
6
0
40
20
0
0
15
10
5
0
5
10
5
10
5
10
5
10
5
Iterations
10
Total daily cost of all users ($)
450
300
Without UCS and SLS
UCS only
SLS only
With both UCS and SLS
250
200
0
5
10
Iterations
15
10
0
0
2
4
6
8
10
User
500
350
20
5
Fig. 5. Convergence of the costs of 10 randomly selected users to NE for a
30-users utility comapny selection and shiftable load scheduling game
400
Without UCS and SLS
With UCS and SLS
25
Daily cost ($)
5
Cost($) Cost($) Cost($) Cost($) Cost($)
Cost($) Cost($) Cost($) Cost($) Cost($)
30
15
10
5
0
30
20
10
0
15
10
5
0
20
10
0
0
15
10
5
0
15
20
Fig. 6. Total daily cost paid by 30 users for various schemes
VII. C ONCLUSION
We have formulated a utility company selection game,
incorporating multiple renewable and non-renewable energy
based utility companies. We have proved that there exists
a Nash equilibrium for the game and have proposed a distributed algorithm that converges to the Nash equilibrium.
Next, we improved the performance of our proposed scheme
by integrating optimal load scheduling with it, using a twostage game framework for analysis. This is a novel approach
exploiting both spatial and temporal dimensions for demand
response management in the smart grid. We have proven
the existence of a Nash equilibrium for this game also, and
have developed a distributed algorithm to converge to the
equilibrium. We have verified the convergence of the proposed
algorithms and have shown that while the utility company
selection scheme improves the costs of the users compared
to the case without any scheduling, the joint scheme yields
the lowest costs compared to three other schemes.
R EFERENCES
[1] H. Farhangi, The Path of Smart Grid, IEEE Power and Energy Magazine,
Vol. 8, No. 1, pp. 18-28, Jan.-Feb. 2010.
[2] A & B National Energy Technological Laboratory (2007-07-27), A Vision
for the Modern Grid, United State Department of Energy, May 2010.
[3] A. H. Mohsenian-Rad, V. W. S. Wong, J. Jatskevich, R. Schober and
A. Leon-Garcia, ”Autonomous Demand Side Management Based on Game
Theoretic Energy Consumption Scheduling for the Future Smart Grid,
IEEE Transactions on Smart Grid, Vol. 1, No. 3, pp. 320-331, December
2010.
[4] Z. Fan, Distributed Demand Response and User Adaptation in Smart
Grids, IEEE International Symposium on Integrated Network Management
(IM), 2011.
Fig. 7. Comparison of the daily costs of 10 randomly selected users with
and without employing optimal utility company selection and shiftable load
scheduling strategies for a 30-users game
[5] A. J. Conejo, J. M. Morales and L. Baringo, Real-Time Demand Response
Model, IEEE Transactions on Smart Grid, Vol. 1, No. 3, pp. 236-242,
December 2010.
[6] C. Wang and M. D. Groot, Managing End-user Preferences in the Smart
Grid, in ACM e-Energy, 2010.
[7] S. Caron and G. Kesidis “Incentive Based Energy Consumption Scheduling Algorithms for the Smart Grid”, SmartGridComm 2010, pp. 391-396.
[8] S. Maharjan, Q. Zhu, Y. Zhang, S. Gjessing and T. Başar, “Dependable
Demand Response Management in the Smart Grid: A Stackelberg Game
Approach,” IEEE Transactions on Smart Grid, Vol. 4, Issue 1, pp. 120-132,
March 2013.
[9] British Columbia Hydro Conservation Electricity Rates, https://www.
bchydro.com/youraccount/content/residential inclining block.jsp2009
[10] S. Bu, F. R. Yu and P. X. Liu, Stochastic Unit Commitment in Smart
Grid Communications, IEEE International Conference on Computer Communications (Infocom) 2011 Workshop on Green Communications and
Networking, 2011.
[11] H. Gintis, Game Theory Evolving, Princeton Univ. Press, May 2000.
[12] M. He, S. Murugesan and J. Zhang, Multiple Timescale Dispatch and
Scheduling for Stochastic Reliability in Smart Grids with Wind Generation
Integrations, IEEE Infocom 2011.
[13] J. B. Rosen, Existence and Uniqueness of Equilibrium Points for
Concave N-Person Games, Econometrica, Volume 33, Issue 3, pp. 520534, Jul. 1965.
[14] A. T. Al-Awamivand E. Sortomme, “Coordinating Vehicle-to-Grid Services With Energy Trading, ” IEEE Transactions on Smart Grid, Vol. 3,
No. 1, March 2012, pp. 453-462, 2012.
[15] S. Shao, M. Pipattanasomporn and S. Rahman, “Grid Integration of
Electric Vehicles and Demand Response with Customer Choice,” IEEE
Transactions on Smart Grid, Vol. 3, No. 1, March 2012, pp. 543-550,
2012.
[16] C. A. Hill, , M. C. Such and D. Chen, “Battery Energy Storage
for Enabling Integration of Distributed Solar Power Generation,” IEEE
Transactions on Smart Grid, Vol. 2, No. 2, June 2012, pp. 850-857, 2012.
[17] M. Fazeli, G. M. Asher, C. Klumner, L. Yao and M. Bazargan, “Novel
Integration of Wind Generator-Energy Storage Systems within Microgrids,”
IEEE Transactions on Smart Grid, Vol. 2, No. 2, June 2012, pp. 728-737,
2012.
[18] R. Deng, J. Chen, X. Cao, Y. Zhang, S. Maharjan, and S. Gjessing,
“Sensing-Performance Tradeoff in Cognitive Radio enabled Smart Grid”,
IEEE Transactions on Smart Grid, Vol. 4, Issue 1, pp. 302-310, March
2013.
[19] I. Koutsopoulos, and L. Tassiulas, “Optimal Control Policies for Power
Demand Scheduling in the Smart Grid”, IEEE Journal on Selected Areas
in Communications, Vol. 30, No. 6, pp. 1049-1060, July, 2012.
[20] C. J. Wong, S. Sen, S. Ha, and M. Chiang, “Optimized-Day Ahead
Pricing for Smart Grids with Device-Specific Scheduling Flexibility”, IEEE
Journal on Selected Areas in Communications, Vol. 30, No. 6, pp. 10751085, July, 2012.
[21] H. Nfaoui, H. Essiarab and A. Sayigh, “A Stochastic Markov Chain
Model for Simulating Wind Speed Time Series at Tangiers, Morocco,”
Renewable Energy, Vol. 29, No. 8, pp. 1407-1418, 2004.
[22] A. Shamshad, M.A. Bawadi, W.M.A. Wan Hussin, T.A. Majid, S.A.M.
Sanusi, “First and Second Order Markov Chain Models for Synthetic
Generation of Wind Speed Time Series,” Energy, Vol. 30, No. 5, pp. 693708, 2005.
2168-6750 (c) 2013 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE
permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TETC.2014.2335541, IEEE Transactions on Emerging Topics in Computing
11
[23] A. Zeroual and M. Ankrim and A. J. Wilkinson, “Stochastic modelling
of daily global solar radiation measured in Marrakesh, Morocco,” Solar
Energy, Vol. 6, No. 7, pp. 787-793, 1995.
[24] P. Poggi, G. Notton, M. Muselli, and A. Louche, “Stochastic Study
of Hourly Total Solar Radiation in Corsica Using a Markov Model,”
International Journal of Climatology, Vol. 20, No. 14, pp. 1843-1860,
Nov. 2000.
[25] J. S. G. Ehnberg and M. H. J. Bollen, “Simulation of Global Solar
Radiation Based on Cloud Observations,” Solar Energy, Vol. 28, No. 2,
pp. 157-162, 2005.
[26] D. Niyato, E. Hossain and A. Fallahi, “Sleep and Wakeup Strategies
in Solar-Powered Wireless Sensor/Mesh Networks: Performance Analysis
and Optimization,” IEEE Transactions on Mobile Computing, 2007, Vol.
6, No. 2., pp. 221-236, Feb. 2007.
Sabita Maharjan is currently a Postdoctoral Fellow in Simula Research Laboratory, Norway. She
received M.E. from Antenna and Propagation Lab,
Tokyo Institute of Technology, Japan, in 2008 and
Ph.D from University of Oslo and Simula Research
Laboratory, Norway, in 2013. Her research interests
include wireless networks, security in wireless networks, smart grid communications, cyber-physical
systems and machine-to-machine communications.
Danny H.K. Tsang (M’82-SM’00-F’12) received
the Ph.D. degree in electrical engineering from the
Moore School of Electrical Engineering at the University of Pennsylvania, U.S.A., in 1989. He has
joined the Department of Electronic & Computer
Engineering at the Hong Kong University of Science and Technology since summer of 1992 and
is now a professor in the department. He was a
Guest Editor for the IEEE Journal of Selected Areas
in Communications’ special issue on Advances in
P2P Streaming Systems, an Associate Editor for the
Journal of Optical Networking published by the Optical Society of America,
and a Guest Editor for the IEEE Systems Journal. He currently serves as
Technical Editor for the IEEE Communications Magazine. He was nominated
to become an IEEE Fellow in 2012. During his leave from HKUST in 20002001, Dr. Tsang assumed the role of Principal Architect at Sycamore Networks
in the United States. He was responsible for the network architecture design of
Ethernet MAN/WAN over SONET/DWDM networks. He also contributed to
the 64B/65B encoding proposal for Transparent GFP in the T1X1.5 standard
which was advanced to become the ITU G.GFP standard. The coding scheme
has now been adopted by International Telecommunication Union (ITU)’s
Generic Framing Procedure recommendation GFP-T (ITU-T G.7041/Y.1303)).
His current research interests include Internet quality of service, P2P video
streaming, cloud computing, cognitive radio networks and smart grids.
Yan Zhang received a PhD degree from Nanyang
Technological University, Singapore. From August
2006, he has been working with Simula Research
Laboratory, Norway. He is currently senior Research
Scientist at Simula Research Laboratory, Norway.
He is an adjunct Associate Professor at the University of Oslo, Norway. He is a regional editor, associate editor, on the editorial board, or guest editor
of a number of international journals. His research
interests include wireless networks and smart grid
communications.
Stein Gjessing is a Professor of Computer Science
in Department of Informatics, University of Oslo. He
received his the Cand. Real. degree in 1975 and his
Dr. Philos. degree in 1985, both form the University
of Oslo. He acted as head of the Department of Informatics for 4 years from 1987. From February 1996
to October 2001 he was the chairman of the national
research program Distributed IT-System, founded by
the Research Council of Norway. Gjessing’s original
work was in the field of programming languages
and programming language semantics, in particular
related to object oriented concurrent programming. He has worked with
computer interconnects and computer architecture for cache coherent shared
memory, with DRAM organization, with ring based LANs (IEEE Standard
802.17) and with IP fast reroute. His current research interests are transport,
routing and network resilience in Internet-like networks, in sensor networks
and in the smart grid.
2168-6750 (c) 2013 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE
permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.