Available online at www.sciencedirect.com
ScienceDirect
Procedia Computer Science 108C (2017) 675–684
International Conference on Computational Science, ICCS 2017, 12-14 June 2017,
Zurich, Switzerland
An
An Advanced
Advanced Software
Software Tool
Tool to
to Simulate
Simulate Service
Service ResRestoration
Problems:
a
case
study
on
Power
Distribution
Systems
An
Advanced
Software
ServiceSystems
Restoration
Problems:
a caseTool
study to
on Simulate
Power Distribution
toration
Problems: a case study on Power Distribution Systems
Richardson Ribeiro1, Fabrício Enembreck4, Douglas M. Guisi2, Dalcimar
4
Richardson Ribeiro1, Fabrício
M. Guisi2, Dalcimar
2 Enembreck , Douglas
1
3
Casanova22, Marcelo Teixeira
2, Fausto A. de4 Souza1 and André P.
1
2 Borges3
Casanova
,
Marcelo
Teixeira
,
Fausto
A.
de
Souza
and
André
P.
Borges
Richardson Ribeiro
FabrícioofEnembreck
, Douglas
M. Guisi , Dalcimar
1
Federal ,University
Paraná, Curitiba,
Parana, Brazil,
1
2 of Paraná, Curitiba, Parana,
Federal
University
Brazil,
Casanova2, Marcelo
Teixeira
, Fausto
A. de Souza1 and
André P. Borges3
richardsonr@ufpr.br,
faustoaugusto04@gmail.com
1 richardsonr@ufpr.br,
faustoaugusto04@gmail.com
of Informatics,
Federal University
of Curitiba,
Technology
of Paraná,
Pato Branco, Brazil
Federal University
of Paraná,
Parana,
Brazil,
Department of {dguisi,
Informatics,
Federal
University
of Technology of Paraná, Pato Branco, Brazil
richardsonr, casanova,
marceloteixeira}@utfpr.edu.br
richardsonr@ufpr.br,
faustoaugusto04@gmail.com
2 3Federal University
richardsonr,
marceloteixeira}@utfpr.edu.br
of Technology
Paraná, Ponta
Grossa, Brazil,
apborges@utfpr.edu.br
Department
of {dguisi,
Informatics,
Federalofcasanova,
University
of Technology
of Paraná,
Pato Branco, Brazil
3
Federal
University
of
Technology
of
Paraná,
Ponta
Grossa,
Brazil,
apborges@utfpr.edu.br
4
Pontifical
Catholic
University
of
Paraná,
Curitiba,
Brazil,
fabricio@ppgia.pucpr.br
{dguisi,
richardsonr,
casanova,
marceloteixeira}@utfpr.edu.br
4
3
Pontifical
Catholic
University of
Paraná, Ponta
Curitiba,
Brazil,
fabricio@ppgia.pucpr.br
Federal
University
of Technology
of Paraná,
Grossa,
Brazil,
apborges@utfpr.edu.br
4
Pontifical Catholic University of Paraná, Curitiba, Brazil, fabricio@ppgia.pucpr.br
2
Department
2
Abstract
Abstract
This
paper presents a software tool to simulate a practical problem in smart grid systems. A feature of
Thissmart
papergrid
presents
a software
tool to simulate
a practical
problem inof
smart
grid systems.
of
the
is a system
self-recovery
capability
in the occurrence
anomalies,
such asAafeature
recovery
Abstract
the
smart
grid
is
a
system
self-recovery
capability
in
the
occurrence
of
anomalies,
such
as
a
recovery
This
paper presents
a software
tool
to simulate
a practical
in smart
grid systems.
featurefor
of
of
a power
distribution
network
after
an occurrence
of a problem
fault. When
this system
has a A
capacity
of asmart
powergrid
distribution
network
after The
ancapability
occurrence
ofamong
aoccurrence
fault.areas
When
this
system
has aascapacity
for
the
system
self-recovery
in the
of computer
anomalies,
such
a recovery
self-recovery,
itisisa called
self-healing.
intersection
as
science,
telecommuself-recovery,
it is called
self-healing.
The
intersection
areas
as computer
science,
telecommunication,
automation
andnetwork
electrical
engineering,
has allowed
power
systems
to gain
newatechnologies.
of
a power
distribution
after
an occurrence
ofamong
a fault.
When
this system
has
capacity
for
nication,
automation
and
electrical
engineering,
has
allowed
power
systems
to
gain
new
technologies.
However,
because
it is aself-healing.
multi-area domain,
self-recovery
simulation
tools in smart
grids
are often
self-recovery,
it is called
The intersection
among
areas as computer
science,
telecommuHowever,
because
it and
is as
aelectrical
multi-area
domain,
self-recovery
simulation
tools
smart
are often
nication,
automation
engineering,
hasbyallowed
power systems
toingain
newgrids
technologies.
highly
complex
as well
presenting
low fidelity
using approximation
algorithms.
The
main
conhighly
complex
as
well
as
presenting
low
fidelity
by
using
approximation
algorithms.
The
main
conHowever,ofbecause
it isisaa multi-area
domain,
simulation
toolsininterms
smartofgrids
are often
tribution
this paper
simulator with
highself-recovery
fidelity and low
complexity
programming,
tribution
of
this
paper
is
a
simulator
with
high
fidelity
and
low
complexity
in
terms
of
programming,
usability
and semantics.
this simulator,
a computational
intelligence technique
derivative
highly complex
as well asInpresenting
low fidelity
by using approximation
algorithms.and
Thea main
conusabilityfor
and
semantics.
this simulator,
aencapsulated.
computational
intelligence
technique
andprogramming,
a derivative
tribution
of
this
paper isthe
aInsimulator
with
high
fidelity and The
low
complexity
in termstool
of
method
calculating
power
flow
were
result
is a software
with
high abmethod for
theInpower
flow
Theintelligence
result
a software
high
abusability
semantics.
this simulator,
computational
techniquetool
andwith
derivative
straction
andcalculating
easy
customization,
aimedwere
at aaencapsulated.
self-healing
system
for is
a reconfiguration
ofa an
electric
straction
andcalculating
easy network.
customization,
aimedwere
at aencapsulated.
self-healing system
for is
a reconfiguration
of anhigh
electric
power
distribution
method
for
the power flow
The result
a software tool with
abpower distribution
straction
and easy network.
customization, aimed at a self-healing system for a reconfiguration of an electric
©
2017
The
Authors. Tool,
Published
by Elsevier
B.V. Self-Healing Smart Grid, Reinforcement Learning
Keywords:
Software
Restoration
Problem,
power
distribution
network.
Peer-review under responsibility of the scientific committee of the International Conference on Computational Science
Keywords: Software Tool, Restoration Problem, Self-Healing Smart Grid, Reinforcement Learning
Keywords: Software Tool, Restoration Problem, Self-Healing Smart Grid, Reinforcement Learning
1 Introduction
1 Introduction
have been dedicated to different segments of electric power systems as generation, trans1 Studies
Introduction
Studies have been dedicated to different segments of electric power systems as generation, trans-
mission and distribution of energy. The need to make these segments more reliable, secure and effimission
distribution
of energy.
Theareas
needsuch
to make
these
segments
reliable,
secure
andtransefficient
hasand
increased
thededicated
intersection
of
as computing,
telecommunications,
automation
and
Studies
have
been
to different
segments
of
electric
powermore
systems
as generation,
cient
hasand
increased
the The
intersection
of
asthese
computing,
telecommunications,
automation
and
mission
distribution
of energy.
Theareas
needsuch
to
make
these
segments
reliable,
secure
and have
effielectrical
engineering.
new applications
with
areas
in
the more
electrical
infrastructures
electrical
engineering.
The
new
applications
withasthese
areas in
cient has the
increased
the energy
intersection
of areas
computing,
telecommunications,
automationhave
and
emerged
intelligent
systems,
calledsuch
smart
grids
(Liserre
etthe
al.,electrical
2010). infrastructures
emerged
intelligent
energy
systems,
called
grids
(Liserre
al.,electrical
2010).
In thethe
energy
distribution
segment,
smart smart
grids
include,
in particular,
the useinfrastructures
of distributed have
syselectrical
engineering.
The new
applications
with these
areas
inetthe
In artificial
thethe
energy
distribution
smart smart
grids
include,
in particular,
the use
of distributed
sysemerged
intelligent
energy
systems,
called
grids
et the
al., process
2010).
tems,
intelligence
andsegment,
power systems
as a way
to(Liserre
automate
of recovering
a distritems,
artificial
intelligence
and
power
systems
as
a
way
to
automate
the
process
of
recovering
a
distriIn the energy distribution segment, smart grids include, in particular, the use of distributed systems, artificial intelligence and power systems as a way to automate the process of recovering a distri-
1877-0509 © 2017 The Authors. Published by Elsevier B.V.
Peer-review under responsibility of the scientific committee of the International Conference on Computational Science
10.1016/j.procs.2017.05.248
676
Richardson Ribeiro et al. / Procedia Computer Science 108C (2017) 675–684
bution network in the event of an abnormality (e.g., failures in network devices, nature damage or
vandalism to the distribution system). Normally, these events generate a fault, causing the nonelectricity supply to consumers. One of the fundamental aspects in a smart grid power distribution
network is the ability of the system to identify and recover the network in the event of a fault. This
ability characterizes a self-healing system (Ghosh et al., 2007).
Several works have proposed self-healing mechanisms for the recovery of electrical networks (Lu
et al., 2009), (Zidan and El-Saadany, 2012), (ZakiEl-Sharafy and Farag 2016). Due the multidisciplinary problem domain, some proposed approaches often require human experts from the subareas of
computing (Distributed Systems, Programming and Artificial Intelligence), telecommunication (signal
transmission and reception), and electrical engineering (power and electrical systems). This multidisciplinary often makes simulation tools highly complex for smart grids, as well as they present low
fidelity, as they require the use of non-exact approaches to service restoration through switching operation.
Here we encapsulate, in a software tool framework for smart grid, the concepts of computational
intelligence to simulate a practical problem of reconfiguration of an electric energy distribution network. A well-known reinforcement learning algorithm and a method for calculating power flow are
simplified through a UML component diagram, while a graphical and semantic interface is used to
guide customization. The results are defined in terms of network quality, loss parameters and maintenance.
2 Learning Systems for Smart Grids
Electricity networks are being improved due to the demand increasing and the development of new
technologies. This can be seen in studies that show the use renewable sources of energy, generation
and distribution, energy efficiency, microgrids, consumer participation and generation of clean energy.
The smart grid concept proposes alternatives and innovations to conventional power grids, such as
allowing the recovery of a network after a fault. This capability comes from steps in a smart grid subarea, called self-healing. Self-healing can be used to restore a system, with techniques that generally
change the topology of the network considering the quality of the electrical energy (e.g., voltage stability, network restrictions, priority load, etc.) delivered to consumers (Lim et al., 2013). Figure 1
shows restoration process on the network.
Figure 1: Restoration process on the network
Detect and isolate fault locations and restoring service are important functions of distribution automation and form the cornerstone of strategies for developing smart grids (Mamo et al., 2009). Service restoration is defined as finding suitable feeders and laterals to transfer the loads in out-of-service
areas using operational criteria through a series of switching operations (Chen, 2010). Restoration is
achieved through the switching operation of tie switches (normally opened). Different restoration
methods thus entail different configurations, which may affect service quality. In addition, because the
task of restoration is usually performed under emergency conditions, time constraints can complexity
the problem (Chen, 2010) (Tsai, 2008).
Richardson Ribeiro et al. / Procedia Computer Science 108C (2017) 675–684
The ability to quickly and flexibly reconfigure the network to restore the loads previously deenergized by the fault represents the key component of the self-healing function. This automated fault
location, isolation and service restoration or self-healing function will: (i) minimize the workload for
the field human operators, (ii) provide almost immediate restoration for the customers, and (ii) improve reliability and robustness of distribution networks. Thus, one vital pillar of smart grids is selfhealing. This ability can be described as the quality of a system that enables it, when subjected to a
fault, automatically and intelligently perform corrective actions to restore the system to the best possible state, enabling it to perform basic functions without violating any system constraints.
Several approaches have been employed to generate self-healing systems in smart grids. Some of
them come from paradigms of learning systems, such as probabilistic, diffuse, genetic and derived
from by reinforcement learning. For instance, Das (2006) presents an algorithm for network reconfiguration based on fuzzy multi-objective approach. The objectives considered attempt to maximize the
fuzzy satisfaction of the load balancing among the feeders, minimization of power loss, deviation of
nodes voltage and branch current constraint violation subject to radial network structure in which all
loads must be energized. The effectiveness of the method was demonstrated through an example.
Learning methods based on collective behaviors also represent part of self-healing learning systems. Using an intelligent swarm-based methodology, Lu et al. (2009) proposes an ant colony optimization algorithm for a minimization problem of energy not supplied during restoration process. The
proposed algorithm based on a hypercube framework searches for an optimal switching sequence, and
the solution provides an effective service restoration strategy that improves system reliability. A similar approach can also be found in Lacerda and Medeiros (2007), who used an ACO algorithm to minimize the number of switching operations on a network. Vlachogiannis and Hatziargyriou (2004) used
reinforcement learning for optimal reconfiguration involves selection of the best set of branches to be
opened, such that the resulting radial distribution system has the desired performance. Among the
several performance criteria considered for optimal network reconfiguration, an important one is real
power losses minimization, while satisfying voltage limits. More specifically, the model-free learning
algorithm (Q-learning) learns by experience how to adjust a closed-loop control rule mapping operating states to control actions by means of reward values. Rewards were chosen to express how well
control actions cause minimization of power losses. The Q-learning algorithm was applied to the reconfiguration of 33-bus radial distribution system bus bar system.
Another approach is to use a multiagent system. In Zidan and El-Saadany (2012), a multiagent system is designed to locate and isolate faults, then decide and implement the switching operations to
restore the out-of-service loads. The proposed control structure has two layers: zone and feeder. The
function of zone agents in the first layer is monitoring, making simple calculations, and implementing
control actions. Feeder agents in the second layer are assigned to negotiation. The constraints include
voltage limits, line current limits, load variation and radial topology. The results show that cooperation
among agents through two-way communication provides a good solution for fault location and isolation and for building an effective restoration plan. In Tsai (2008), an expert system is developed by
utilizing its fast reasoning mechanism and object-oriented features. The feeder component and configuration data are organized in a hierarchy way using the object-oriented programming paradigm. The
simulation results indicate that more solutions can be obtained for service restoration problems when
load variation is considered. Another unique property of the proposed system is that it is capable of
proposing multiple restoration plans. By proposing multiple plans, the system operator can choose a
plan which is more suitable to the real situation for service restoration problems.
There are also several other methods for reconfiguring networks, which consider the minimization
of electrical losses using genetic algorithms (Awais et al., 2015); the use of selective cutting of system
loads to maintain the supply of priority regions (Ferreira et al., 2014); and models of quantitative decision making, using the analytical hierarchy process-based fuzzy-grey approach (Chen, 2010).
These are just some of the dedicated works in building techniques for self-healing systems in smart
grids. It is observed that, because it is a multidisciplinary domain, the tools proposed for the simula-
677
Richardson Ribeiro et al. / Procedia Computer Science 108C (2017) 675–684
678
tion of self-recovery in smart grids are, for the most part, highly complex, difficult to reproduce, as
well as presenting low fidelity due to the use of non-exact or approximation approaches, according to
the papers mentioned. In addition, the problem is characterized by a function with multiple objectives,
subject to operational and electric constraints, usually with a huge space of solutions, low scalability
and nondeterministic.
In the next section, we present a computational tool with high fidelity and low complexity in terms
of programming, usability and semantics. In this tool was encapsulated the concepts of self-healing
using a computational intelligence technique and a method for calculating the power flow. The result
is a tool of high abstraction and easy customization, aimed at a self-healing system for the reconfiguration of an electric power distribution network.
3 Contribution: A Software tool (framework) to simulate problems of service restoration in distribution systems
Before presenting the proposed framework*, the problem formulation is showed in subsection A,
and in subsection B the main constraints on service restoration. These subsections are supported by the
theoretical aspects presented in subsection C and by the practical aspects presented in the works of
Kumar et al. (2008), Chen (2010) and, Zidan and El-Saadany (2012). The restoration problem can be
formulated as a multi-objective multi-constraint optimization problem. The proposed objectives and
constraints considered are as follows:
A. Objective Functions
����
a. maximizing the load restored with consideration of load priority: �� ∑���
�� ∗ �� ∗ �� , (Eq. 1),
where Nbus: the number of energized buses after service restoration; Li: load at the ith bus; yi: status of
the load at the ith bus (i.e., equals 1 for a restored load and 0 for an unrestored one); and wi: priority or
importance level of the load at the ith bus;
b. minimizing the number of switching operations and thus reducing the time required for and the
��
operational cost of the restoration: �� ∑���
|�� ∗ ��� | , (Eq. 2), where Ns: the total number of switches;
xi: status of the ith switch in the restored network (i.e., equals 1 for a closed one and 0 for an opened
one); and xio: status of the ith switch immediately after the fault has been isolated;
�
�� �
c. minimizing losses during the restoration period: ��� ����
�� ∗ �� , (Eq. 3) where Nbr: total number
of branches; Ii: current in the ith branch; and Ri: resistance of the ith branch.
B. Constraints
1) radial network structure should be maintained (i.e., buses should be energized by only one feeder);
2) bus voltages at all buses should be kept within limits: ���� ≤ �� ≤ ���� , where Vi: voltage at the
ith bus, and Vmax, Vmin: maximum and minimum acceptable bus voltages (in this research, they are 1.02
p.u. and 0.9 p.u., respectively);
3) all branch currents should be kept within their limits: �� ≤ ���� , where Ij: (instant) current in branch
j, and Imax: maximum line current.
C. Issues related to the restoration problem in distribution systems
Here we highlight some of the important aspects related to the operational practices of the restoration problem, which were used for designing the tool described in subsection D. The related issues are
the follows:
*
download in: https://drive.google.com/open?id=0B4FCRZNOz4xQQ2NaeWQySkdNLVk
Richardson Ribeiro et al. / Procedia Computer Science 108C (2017) 675–684
 For locating and isolating the fault and coordinating the protection device, it is important to maintain the radial network topology during the service restoration process (Kumar et al., 2008).
 In the case of limited available capacity for restoration or partial restoration, the restoration should
begin with the highest priority customers first (i.e., hospitals, industrial loads, etc.) as formulated in
equation 1 (Kumar et al., 2008).
 Do not violate the operational constraints (voltage and current limits). This point is aligned with
the second objective in (B2) because considering load variation will prevent the need for further
reconfigurations during the future restoration, thus reducing the number of switching operations.
 Restoration is accomplished by transferring loads in the out-of-service area to other feeders
through the on-off control of tie-switches. Because the time required for the restoration process
depends on the number of switching operations (Chen, 2010) (Kumar et al., 2008), that number
should be kept as low as possible, as formulated in equation 2. For example, the typical operating
time of remote-controlled and manually controlled switches are 50s and 1200–1500s, respectively.
Reducing the number of switching operations also reduces the possibility of switching surges, the
risk of outages, and the number of transient disturbances in the system due to multiple switching
operations. The operational cost of the switching operations is also lowered (i.e., the cost of
maintenance and dispatching technicians for non-automated systems as well as the costs associated
with the reduced lifetime of the switches).
 If the restoration provides a network configuration with a topology closer to that of the pre-fault
configuration, it will be easier to return to the normal configuration after the fault is cleared.
 If any loads are still unrestored after the load transfer, shedding of the least priority loads can enable the remainder of the unrestored loads to be restored with the remaining limited capacity.
 For practical purposes, the loss reduction calculated according to (equation 3) is definitely to be
used in normal operating conditions and is not appropriate for service restoration during an emergency situation (Kumar et al., 2008) (Liu et al. 2000).
D. Contribution Analysis
In section 2 we present methods used to restore services from algorithms derived from reinforcement learning. Reinforcement learning is a computational paradigm in which a learning agent maximizes a measure of performance, receiving rewarding or punishing efforts while interacting with the
environment (Sutton and Barto, 1998). The agent (actuator) acts in the environment formed by a set of
states and can choose actions within a set of possible actions, indicating the immediate value of the
resulting state transition. The agent's task is to learn a control policy (sequence of actions) that maximizes the expected sum of these reinforcements, discounting (usually exponentially) the rewards or
punishments proportionally to their temporal delay (Sutton and Barto, 1998).
The tool proposed in this study uses the reinforcement learning algorithm, called Q-learning, to determine which switches should be opened or closed. Working detail of this algorithm can be found in
Watkins and Dayan (1992). To ensure the network constraints, it used the Newton-Raphson method to
calculate the power flow. The power flow employs a series of calculations to determine the current,
voltage and losses using the data of resistance, reactance, reactive power and active power. Details of
the implementation of this method can be found in (Grainger and Stevenson 1994).
Experiments using the framework show the performance of the Q-learning algorithm based on the
power flow with different scenarios and parameters. The framework is able to show, in an interactive
way, the impact of the variation of Q-learning parameters (learning rate, discount factor and greedy).
Figure 2 presents an overview of the software tool and the main components that make up the interface are detailed. It is possible to observe, to the right side of Figure 2, the Cartesian plane where
the network is positioned. In the upper part of the image, the learning parameters of the Q-learning
algorithm (α, γ, ε) are shown. The variable α represents the learning constant. The variable γ represents the discount factor, and finally, the variable ε represents the probability of the agent choosing an
action by the highest expected value.
679
680
Richardson Ribeiro et al. / Procedia Computer Science 108C (2017) 675–684
Figure 2: Software tool overview
Stop condition is the condition that will cause the agent to interact with the environment until it
reaches the given number. The Elements Details section reports the values of loads, feeders, and
branches. In the functions section, it is possible to visualize the network information, the power flow
and the objective function. Table 1 shows the corresponding values.
Network information(total no. of)
Feeders, Branches, Loads, switches, active/reactive power demand
Table 1: Software tool functions
Power flow
Loads supplied/not supplied,
Loads active/reactive power supplied,
Loads active/reactive power not supplied, Loads active/reactive power outof-service, Feeders used active/reactive
power, Feeders available active/reactive
power, Total active/reactive power lost
Objective Functions (%)
Active/reactive power lost
percentage, Supplied active/reactive power percentage,
Not supplied active/reactive
power percentage, Out-ofservice active/reactive power
percentage
In the options views, the graphs represent the network properties at each iteration of the agent using the Q-learning algorithm. In this options, the estimated values of the power flow are shown, such
as: power loss, supplied loads, load current voltage (p.u), reward (%loads supplied) and required
switch operations. The learning values of the Q-learning algorithm and the values of the objective
functions can be seen in the graphs policy change (figure 3a) and Q-values average (figure 3b).
(a) Policy change: number of times the agent changed the
status of the switch
Figure 3: Charts
(b) Q-values: reward value for convergence analysis
Richardson Ribeiro et al. / Procedia Computer Science 108C (2017) 675–684
The system input is a configurable file that contains the values of the network such as real power
PL (kW), reactive power QL (kVar), load priority, maximum branch current (A), resistance R (Ω),
reactance X), switch (open, close).
Figure 4: Component Diagram
Figure 4 shows the component diagram of the framework, illustrating the communication of the main components (agent, algorithm, network, power flow). The agent
uses the Q-Learning algorithm to learn what action should
be taken after the perception of the environment (network).
The network component, upon verifying a change made by
the agent, requests execution of the power flow.
The interaction of the agent in the network using the reinforcement learning algorithm is as follow. An action is
selected when the agent initiates interaction with the network. The selected action is then performed and a switching operation occurs. After this, the power flow is executed, generating the values of bus voltage, branch current
and power loss. The agent then checks whether or not the
switching operation has improved the target function values. This generates a reward value (positive or negative)
by modifying the Q-Learning learning table. These procedures are repeated until some stopping criterion is satisfied, such as convergence of the algorithm or number of
iterations of the agent.
4 Using the framework to evaluate network restoration with
reinforcement learning
The proposed Q-learning-based self-healing framework was tested on a distribution system having
2 substations, 4 feeders, and 70 nodes. Data for this system are given in (Das 2006). The experiments
evaluated the framework considering factors such as network quality, loss parameters and maintenance. The parameters that have been used in Q-learning are the same as those used in Guisi et al.
(2016): discount factor (y) of 0.99, e-greedy (ε) of 0.90 and learning rate (α) of 0.2.
The number of iterations required for the agent to find its best efficiency was 3000. Smaller values
are insufficient due to the number of combinations, since the policy is 2n, where n is the number of
switches. As a strategy to reduce state space, we limit the candidate switches to a maximum distance
from tie-switches. To do this, up to 11 switches were tested closer to each tie-switch.
Figure 5: Search space
Figure 5 illustrates this representation. The best results were achieved
using 5 candidate switches. Larger
values require a much larger number
of iterations, without guaranteeing
improved efficiency in terms of network quality. Values less than 5
maybe not guarantee the best solution, however, it ensures faster convergence.
681
Richardson Ribeiro et al. / Procedia Computer Science 108C (2017) 675–684
682
Outage area
Isolated area
(a) Before the fault
(b) Faulted and outage area
Outage
area
(c) Final radial configuration after network reconfiguration
Figure 6: Tested radial distribution system
Figure 6a shows pre-fault network configuration, in which the total real power loss of this
system is 0.2311 MW and minimum voltage is
Vmin = V67 = 0.92672 p.u. Figure 6b shows the
fault location (branch 58) and switching operation to isolate the fault (open switches: 53, 54,
60). In this figure is seen two outage area.
After network reconfiguration (Figure 6c),
number of loads supplied improved in 11.8%
and number of loads not supplied improved in
53.8%. The minimum voltage of the system
change from 0.9460 to 0.9065 p.u. It is to be
noted here that during the iterative process, the
proposed algorithm has considered only three
out of twenty tie-switches (64, 66 and 69) and
remaining tie-switches have been discarded.
Remember that, in general, it is not possible to
energize 100% of the loads after the reconfiguration. This is due to network limitations, mainly due to branch currents (�� ≤ ���� ) or due to
the search method itself (in this case, the reinforcement learning is a heuristic search method
that does not guarantee an optimal solution).
The restoration process consists of restoring
(totally or partially) the loads into the outage
area. Fig. 6c illustrates the configured network.
Richardson Ribeiro et al. / Procedia Computer Science 108C (2017) 675–684
Table 2 shows the open switches, losses, minimum voltage, switching operations, the load shed
during the restoration plan and others system information [Base: 11 KV, 1 MVA].
Before the fault (6a)
Faulted (6b)
System and power flow information
Total active power demand (MW)
4.47
4.47
Fault locations (branch)
58
Open switches to isolate the fault
53, 54, 60
Outage areas
0
2
Number of loads supplied
68
52
Number of loads not supplied
0
13
Loads active power supplied (MW)
4.847
3.568
Loads active power not supplied (MW)
0.0
1.279
Loads active power out-of-service (MW)
0.0
0.0
Feeders used active power (MW)
5.0779
3.7223
Feeders available active power (MW)
9.3220
10.6776
Total active power lost (MW)
0.2311
0.1592
minimum voltage (p.u)
V29 = 0.9267
V50=0.9460
Objective function
Active power lost (%)
4.76
4.32
Supplied active power (%)
100
79.65
Not Supplied active power (%)
0.0
20.35
Open Switches
69, 72, 70, 71
69, 72, 70, 71
Table 2: Open switches, loss, minimum voltage, switching operations, others
Reconfigured network (6c)
4.47
58
53, 54, 60
1
59
6
4.146
0.701
0
4.3920
10.0079
0.2461
V65=0.9065
5.96
92.55
7.45
69, 66, 64
Table 3 shows feeders used active power before and after network reconfiguration. From Table 3,
it is seen that feeders are more balanced after reconfiguration.
Feeder
Before reconfiguration
After reconfiguration
1
0.74
1.30
2
1.08
1.28
3
1.92
1.44
4
0.06
0.06
Table 3: Feeder used active power before and after reconfiguration
5 Conclusions
In this paper, we proposed a framework for the self-healing of intelligent systems to solve the network reconfiguration problem in a radial distribution system. The problem of reconfiguring electric
power distribution systems is a practical example of the intersection among areas as Computer Science, Telecommunication, Automation and Electrical Engineering. Because it is a multidisciplinary
problem domain, the approaches studied usually require interaction among human specialists from
these areas, which makes it difficult to converge to practical and understandable tools. Therefore, this
multidisciplinary makes the simulation tools for high complexity smart grids, as well as low fidelity
due to the problem presenting characteristics that require the use of approximation algorithms to find
near-optimal solution (approximate solutions).
The framework presented consists of abstracting these areas, considering the electrical magnitudes
(i.e., voltage, current, resistance and power) and the characteristics of the problem domain (voltage
limits, line current limits, and radial topology). The result of the proposal is a computational tool
framework of easy interaction, low complexity and programmable using a high level alternative environment. For service restoration, the framework implements the well-known reinforcement learning
algorithm, which is able to estimate action policies after an estimated number of interactions. In this
683
Richardson Ribeiro et al. / Procedia Computer Science 108C (2017) 675–684
684
work, the use of Q-learning discovered action policies that determined switching operations to restore
the out-of-service loads. The detailing of the operation of the Q-learning algorithm is beyond the scope
of this work, and can be found in Kaelbling et al. (1996) and Sutton and Barto (1998). Although we
use the Q-learning algorithm, the software tool is capable of encapsulating any approximation or exact
algorithm, since the network and power flow components are independent of the control algorithm
employed. This allows other methods to be easily tested.
The effectiveness of the proposed method was demonstrated through an example. Perspective of
future research include the use of algorithms with different paradigms to decrease the number of candidate switches. Due to the number of combinations to be 2n, a strategy, as presented in Guisi et al.
(2016), could be used for such. It is intended to further evaluate the framework with greater variations
of switches, loads and voltage.
References
Awais, M., Javaid, N., Shaheen, N. (2015). An Efficient Genetic Algorithm Based Demand Side Management
Scheme for Smart Grid. In 18th Int. Conf. on Network-Based Inf. Systems, Taipei, Taiwan, pp. 351-356.
Chen, W.-H. (2010). Quantitative decision-making model for distribution system restoration. In IEEE Transactions on Power Systems, 25(1), pp. 313–321.
Das, D. (2006). Reconfiguration of distribution system using fuzzy multi-objective approach. In International
Journal of Electrical Power & Energy Systems, 28(5), pp. 331–338.
Ferreira, L., Siebert, L. C., Aoki, A. R., Fernandes, T. S. P. (2014). Load shedding through optimal power flow to
support self-healing actions in distribution feeders. In IEEE PES Transmission & Distribution Conference
and Exposition Latin America (PES T&DLA), Medellin, pp. 1-6.
Ghosh, D., Sharman, R., Rao, R. and Upadhyaya, S. (2007). Self-healing systems - survey and synthesis. In Decision Support Systems, 42(4), pp. 2164–2185.
Grainger, J. J. and Stevenson, W. (1994). Power System Analysis. In New York: McGraw-Hill.
Guisi, D., Ribeiro, R., Teixeira, M.. Borges, A. P., Enembreck, F. (2016) Reinforcement Learning with Multiple
Shared Rewards. In: International Conference on Computational Science (ICCS), 2016, San Diego.
Kaelbling, L. P., Littman, M. L., and Moore, A. P. (1996). Reinforcement learning: A survey. In Journal of Artificial Intelligence Research, 4, pp. 237–285.
Kumar, Y., Das, B. and Sharma, J. (2008). Multiobjective, Multiconstraint service restoration of electric power
distribution system with priority customers. In IEEE Transactions on Power Delivery, 23(1), pp. 261–270.
Lacerda, E. G. M., and Medeiros, M. F. (2007). Minimum Number of Switching Operations Via Ant Colony
Optimization. In 19th International Conference on Electricity Distribution, Vienna, pp. 21-24.
Lim, K., Jung, W., Ko, Y., and Kim, Y. (2013). On the self-healing mechanism in smart grid networks. In International Journal of Smart Home, n. 2, pp. 27-38.
Liserre, M., Sauter, T. and Hung, J. (2010). Future energy systems: Integrating renewable energy sources into the
smart power grid through industrial electronics. In IEEE Industrial Electronics Magazine, pp. 18–37.
Liu, C., Lee, C. and Venkata, S. (2000). An expert system operational aid for restoration and loss reduction of
distribution systems. In IEEE Trans. Power Syst., no. 2, pp. 619–626.
Lu, Z., Wen, Y. and Yang, L. (2009). An improved ACO algorithm for service restoration in power distribution
systems. In Institute of Electrical and Electronics Engineers (IEEE), 3, pp. 2864-2871.
Mamo, X., Mallet, S., Coste, T. and Grenard, S. (2009). Distribution automation: The cornerstone for smart grid
development strategy. IEEE Power and Energy Society General Meeting, Calgary, Alb., Canada, 2009.
Sutton, R. S. and Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press.
Tsai, M.-S. (2008). Development of an object-oriented service restoration expert system with load variations.
In IEEE Transactions on Power Systems, pp. 219–225.
Vlachogiannis, J.G. and Hatziargyriou, N. (2004). Reinforcement learning (RL) to optimal reconfiguration of
radial distribution system (RDS). In Methods and Applications of A.I. Springer Nature, pp. 439–446.
Watkins, C.J.C.H. and Dayan, P. (1992) ‘Q-learning’, Machine Learning, 8(3-4), pp. 279–292.
ZakiEl-Sharafy, M. and Farag, H. E. (2016). Self-healing Restoration of Smart Microgrids in Islanded Mode of
Operation. Smart City 360°. In Social Informatics and Telecommunications Engineering. pp. 395-407.
Zidan, A. and El-Saadany, E.F. (2012). A cooperative Multiagent framework for self-healing mechanisms in
distribution systems. In IEEE Transactions on Smart Grid, 3(3), pp. 1525–1539.