Ain Shams Engineering Journal 15 (2024) 102899
Contents lists available at ScienceDirect
Ain Shams Engineering Journal
journal homepage: www.sciencedirect.com
A new adaptive droop control strategy for improved power sharing
accuracy and voltage restoration in a DC microgrid
Rifqi Firmansyah a, b, * , Makbul A.M. Ramli a, c, *
a
Department of Electrical and Computer Engineering, Faculty of Engineering, King Abdulaziz University, Jeddah 21589, Saudi Arabia
Department of Electrical Engineering, Faculty of Engineering, Universitas Negeri Surabaya, Surabaya, Indonesia
c
Research Center for New and Renewable Energy Engineering, Universitas Airlangga, Surabaya 60115, Indonesia
b
A R T I C L E I N F O
A B S T R A C T
Keywords:
DC microgrid
Power sharing
Sliding mode control
TD3
Voltage restoration
In a DC Microgrid, accurate power sharing and voltage restoration are two primary control goals to guarantee
power quality and reliable operation. Inaccurate power sharing is a significant concern due to discrepancies in
feeder line resistance, faulty equipment, lack of monitoring and control, and nonlinear load. Moreover, inaccurate power sharing may lead to overloading of converters and cause a cascade of failures throughout the entire
system. This study proposes a new adaptive droop control strategy to address these challenges. To enhance
accurate power sharing, error current sharing is formulated by considering bus current and total rated current.
This is regulated by the adaptive controller to adjust droop resistance. Additionally, the impact of droop voltage
because of feeder line resistance is considered in the proposed strategy. The primary control loop regulates the
output voltage of converter utilizing the proposed observer-based optimum sliding mode control (OOSMC). The
controller gain of the OOSMC is optimized using a gradient-based method to enhance transient and steady state
response of the converter output. In the secondary loop, the twin-delayed deep deterministic policy gradient
(TD3) for voltage restoration is employed with a new reward function to optimally tune the proportional-integral
(PI) controller. Finally, the superiority of the proposed strategy is evaluated through rigorous testing scenarios,
including the use of constant power load (CPL) and sudden failure in the converters. Moreover, the proposed
strategy is validated by simulations and laboratory-based experiments. The results show that the OOSMC outperforms GA and PSO while the TD3 has better performance than traditional PI. Furthermore, when the
equivalent power-rated converters are implemented in the system, the proposed strategy can transfer identical
power sharing of 4 W to the load and provide accurate current sharing of 0.167 A compared to conventional
droop control while the DC bus voltage is maintained at the 24 V reference value. In the case of different powerrated converters, the proposed strategy can achieve power sharing accuracy. The first, second and third converters supply 2 W, 4 W, and 6 W, respectively, to the load and provide accurate current sharing. Meanwhile the
DC bus voltage can be kept at the reference voltage of 24 V. In the scenario of various kinds of disturbances, the
proposed strategy can achieve accurate power and current sharing when the system is tested under load and
input voltage variations. Meanwhile the DC bus voltage always returns to the voltage reference. Moreover, the
proposed strategy can share power and current accurately when the converter occurs a sudden failure.
1. Introduction
Implementation of microgrids have been increasing due to electric
grid drawbacks such as investment cost, maintenance difficulties, and
environmental problems, and the growing interest in renewable energy
sources (RES) [1]. Direct current (DC) microgrids are small power distribution systems that consist of multiple interconnected dc power
sources and are often used to improve energy efficiency. It operates on
DC and typically comprises RES, energy storage systems (ESS), and loads
connected to a DC bus [2]. DC microgrids provide various benefits over
AC microgrids, including high power efficiency, ease of control,
increased dependability, and compatibility with a variety of consumer
electronic DC loads [3]. In the operation of DC microgrids, the reactive
power flow, power quality, and frequency remain unaffected [4]. As a
result, DC microgrids have been widely employed in various industries,
including telephony, data centers, ESS, solar array systems, electrical
boat systems, aircraft, and electric vehicles.
* Corresponding authors.
E-mail addresses: rifqifirmansyah@unesa.ac.id (R. Firmansyah), mramli@kau.edu.sa (M.A.M. Ramli).
https://doi.org/10.1016/j.asej.2024.102899
Received 10 November 2023; Received in revised form 25 March 2024; Accepted 29 May 2024
Available online 19 June 2024
2090-4479/© 2024 THE AUTHORS. Published by Elsevier BV on behalf of Faculty of Engineering, Ain Shams University. This is an open access article under the CC
BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
Nomenclature
Kp
Ki
e
Φ
u
η1
η2
Vref,i
Vbus,ref
uv
ωd,i
ii
Vc,i
Vbus
ec,i
Ibus
Ir,i
Ir,total
Ωd,i
ẋ1
ẋ2
x1
x2
d
R
L
C
Vin
δ
iL
Vc
Vout
̂
x˙
1
̂
x˙ 2
̂
x1
̂
x2
̂ in
V
̂˙ in
V
̂
δ
˙
̂
δ
K1
K2
̃
x1
̃
x2
̃
δ
̃ in
V
̃
x˙ 1
̃
x˙ 2
V
γ1
γ2
V̇
ILref
σ
σ̇
deq
dn
T
du
Proportional gain
Integral gain
Error signal
Constant value of PI controller
Control signal of adaptive PI
Constant value of parameter Kp
Constant value of parameter Ki
Voltage reference ith converter
Bus voltage reference
Control signal of TD3
Droop resistance value of the ith converter
Current of the ith converter
Output Voltage of the ith converter
Bus voltage
Current sharing error of the ith converter
Bus load current
Rated current of the ith converter
Total rated current from all converters
Adaptive droop gain
Derivative of inductor current
Derivative of capacitor voltage
Inductor current
Capacitor voltage
Duty cycle
Resistor
Inductor
Capacitor
Input voltage
Reverse of resistor
Inductor current
Capacitor voltage
Output voltage of the converter
α
s
a
N
μ
Q
θμ
θQ
θμʹ
θQʹ
τ
Qʹ
γ
r
LF
M
∇θ μ J
y
λ
β
ur
er
Kii
Kpp
uV
eV
Kpt
Kit
Derivative of switching surface
Switching Controller
Natural control
Gain of natural control
Control signal of OOSMC
Learning rate
State
Action
Random noise function
Actor
Critic
Actor update
Critic update
Target actor update
Target critic update
Contant of update
Target actor
Discount factor
Reward function
Loss function
Number of samples
Policy gradient
Temporal difference error
Gain of proposed reward function
Constant of reward function
Reward agent control action
Error of the reward agent input
Integral gain of neural network weight
Proportional gain of neural network weight
Control signal of secondary controller
Error between bus voltage reference and desired bus
voltage
Proportional gain for secondary loop control
Integral gain for secondary loop control
Abbreviations
ACO
Ant Colony Optimization
BWO
Beluga Whale Optimization
CPL
Constant Power Load
DC
Direct Current
DDPG
Deep Deterministic Policy Gradient
DNN
Deep Neural Network
DQN
Deep Q-Network
DRL
Deep Reinforcement Learning
ESS
Energy Storage Systems
FA
Fire-fly Algorithms
GA
Genetic Algorithms
ITAE
Integral Time Absolute Error
LFC
Load Frequency Control
MPC
Model Predictive Control
OOSMC Observer-Based Optimum Sliding Mode Control
PI
Proportional-Integral
PSO
Particle Swarm Optimization
RES
Renewable Energy Sources
RL
Reinforcement Learning
SMC
Sliding Mode Control
SFO
Sunflower Optimization
TD
Temporal difference
TD3
Twin-Delayed Deep Deterministic Policy Gradient
Estimation value of ẋ1
Estimation value of ẋ2
Estimation value of inductor current
Estimation value of capacitor voltage
Estimation value of input voltage
̂ in
Derivative of V
Estimation value of δ
Derivative of ̂
δ
Gain of error estimation of x1
Gain of error estimation of x2
error estimation of x1
error estimation of x2
error estimation of δ
error estimation of Vin
Derivative of ̃
x1
Derivative of ̃
x2
Quadratic Lyapunov function
Adaption gain of δ
Adaption gain of Vin
Derivative of V
Inductor current reference
Switching surface
2
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
Fig. 1. The proposed control of DC Microgrid.
while the SMC is to control the converter. The results displayed that the
converter has performed well under transient and steady-state conditions. Moreover, in [23], an adaptive SMC with an observer was proposed to control a boost converter by employing model predictive
control (MPC). The method facilitates the accurate estimation of the
converter output voltage. However, the recovery period needs
improvement because of the impact of disturbances.
Other issues in DC microgrids are accurate power sharing and
voltage restoration among parallel converters [24]. Parallel converters
can also experience harmful circulating currents which reduce energy
efficiency, lead to unbalanced power sharing, and can even damage
components. To overcome this issue, a current sharing strategy must be
proposed to ensure power sharing accuracy with minimum current
sharing error to the load. However, when applying this strategy, another
controller is required to prevent droop voltage at the DC bus. The most
utilized controllers for voltage restoration at the DC bus are the
proportional-integral (PI) controller. Several methods have been used to
optimize the parameters of the PI controllers and the design of energy
system, such as ant colony optimization (ACO) [25], genetic algorithm
(GA) [26], particle swarm optimization (PSO) [27], sunflower optimization (SFO) [28], beluga whale optimization (BWO) [29], fire-fly algorithms (FA) [30], and harmony search [31]. These strategies,
however, are typically suggested for systems with nonlinear constraints.
Reinforcement learning (RL) controller-based methods have been
implemented in smart grid systems in recent years. A comprehensive
analysis of the principle and function of deep RL (DRL) has been published in the previous research [32]. However, the degree of action
discretization is fundamental because a low-dimensional action region
may result in finite control performance. A grouping of deep learning
and RL can resolve these issues. The deep deterministic policy gradient
(DDPG) algorithm has been presented by Lillicrap et al. to address the
issues with continuous control [33], where discretization of states and
actions is unrequired. In [34], a twin-delayed DDPG (TD3) has been
implemented using DRL and PID controller parameter adjustment to
regulate load frequency control (LFC). TD3 can address DDPG limitations by updating delayed actors, double critics, and actors.
According to the literature, several conventional droop control
strategies have been used for power sharing accuracy. However, droop
voltage due to feeder line resistance was not considered in the system. In
the primary loop, the SMC has been used to control and maintain the
output voltage of the converter. Meanwhile, the conventional PI has
been used for voltage restoration in the secondary loop. The controller
The first objective of integrating and controlling multiple devices in
DC microgrids is to control and maintain the output voltage of each
converter around the set point [5]. The second objective is voltage
restoration and power sharing accuracy among converters. Many researchers have developed various methods to control DC microgrid,
such as master–slave, decentralized, centralized, hierarchical, and
distributed control [6].
The dependability of master–slave technique is poor since any issues
with the master control could result in system shutdown [7]. Another
fundamental issue of centralized control is its dependability as changing
the network requires redesigning the entire control system [8]. Decentralized control design avoids a central controller and long communication lines [9]. However, the main disadvantages of this scheme are the
ineffectiveness control and security [10]. Distributed control keeps
voltage close to the reference value and distributes loads fairly in a DC
microgrid [11,12]. As a result, this method eliminates the chance of one
object failure [13]. However, this approach is affected by transmission
lag and requires challenging mathematical analysis [14]. To address the
shortcomings, a hierarchical strategy has been suggested. This approach
involves mathematical analysis and the use of communication lines
[15].
The local signal is used for both the dynamic and steady-state voltage
regulation of each converter output. Furthermore, it aims to provide
suitable load sharing and to prevent imbalance at DC microgrids first
level. Previous research have introduced some techniques for these
purposes, such as DC bus signaling [16], fuzzy logic control [17], and
droop control [18]. The droop control approach is a popular method for
DC microgrids primary control due to its superior performance
compared to other techniques. In [19], the authors have proposed an
adaptive droop control utilizing a sliding mode control (SMC) to
improve the system output. The SMC can quickly respond to system
dynamics and is insensitive to modifications in system structure. It also
provides fast dynamic and steady-state response for nonlinear systems
including power converters [20].
To monitor some variables such as input voltage and load resistance
changes, the SMC relies on multiple sensors, which proved costly, needs
a substantial area, and require complex computations due to real-time
data recording. In [21], the authors have proposed SMC based on
observer design by considering the input voltage and load variety. In
[22], the authors have proposed two methods namely adaptive law with
observer-based method and SMC. The adaptive law with observer-based
method is to estimate input voltage uncertainties and load changes,
3
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
gain of the SMC is selected using trial and error. Moreover, inductor
current, intermittent input voltage, and load variety were not considered
in the controller design. The parameters of PI for voltage restoration
need to be optimized to improve the performance of the controller. The
research gap is covered in this study through new various control strategies. The main contributions of this study are:
output of the system. In this approach, parameters of the controller are
not constant but are adjusted by using the recent behavior. The formulation of adaptive PI is expressed as follows [35]:
⎛
⎞
∫t
u(t) = Φ⎝Kp (t)e(t) + Ki (t)e(t)dt ⎠
(1)
• A new strategy of adaptive droop control for power sharing accuracy
is proposed. Accurate power sharing is achieved by minimizing
current sharing errors. Current sharing errors are determined using
bus current and total rated current. Afterwards, an adaptive PI
controller is employed to control the current sharing error and adjust
the droop resistance. This new strategy considers the droop voltage
due to feeder line resistance in each converter.
• In the primary loop, an observer-based optimum sliding mode control (OOSMC) is proposed to regulate the output voltage of each
converter. To enhance the effectiveness of the proposed method, the
controller gain is tuned using a gradient-based approach. This proposed method improves the robustness of system against disturbances. The control design also considers the inductor current
estimation, intermittent input voltage, and load variety disturbance
as observers.
• In the secondary loop, a novel reward function is proposed in the
design of TD3 to optimize the parameters of PI controller for voltage
restoration. Furthermore, rigorous testing scenarios, such as constant
power load (CPL) and generator or converter malfunction, is
employed to evaluate the performance of the proposed strategy.
where u, Kp , Ki , Φ, and e are the control signal of adaptive PI, the proportional gain, the integral gain, the constant value, and the error signal,
respectively. Kp and Ki are constantly fine-tuned based on the current
error, as illustrated below [35]:
∫t
Kp (t) = e2 (t) + η1
e2 (t)dt
(2)
0
0
Ki (t) = η2
∫t
0
e2 (t)dt
(3)
Where η1 and η2 are the constant values. Rising the gain Φ can speed up
the system response, while involvement of the PI actions can be tuned by
adjusting η1 and η2 .
2.2. The new strategy of adaptive droop control
The gain of the proposed droop strategy is regulated by the PI
controller that works adaptively to improve accurate power sharing
through reduction of current sharing error. This study proposes a new
adaptive droop control strategy defined as follows:
))
(
(
Vref,i = Vbus,ref + uv − ωd,i *ii + Vc,i − Vbus
(4)
The rest of this study is structured as follows. In Section 2, the proposed adaptive control strategy is explained. A primary and secondary
loop control design are given in Section 3. Section 4 shows numerical
simulation results, while section 5 is the results of experimental hardware. Conclusions are revealed in Section 6.
where Vref,i is the voltage reference ith converter, Vbus,ref defines the
reference value of dc bus voltage, and uv denotes control signal that
comes from secondary controller. This signal is employed for voltage
restoration whilst maintaining the power sharing accuracy of each
converter in the DC microgrid. ωd,i denotes the droop resistance value of
the ith converter determined adaptively, while ii , Vc,i , and Vbus are the
current of the ith converter, the capacitor voltage of the ith converter, and
the DC bus voltage, respectively. The droop voltage, Vc,i − Vbus in each
converter because of feeder resistances are considered in the proposed
adaptive droop control strategy. The secondary controller that generates
uv is employing a PI controller tuned by TD3. The detailed explanation of
the primary and secondary control design will be presented in section 3.
To improve power sharing accuracy, the current sharing error must be
reduced as small as possible. The expression of current sharing error is
formulated as follows [36]:
(
)
Ir,i
ec,i = ii − Ibus
(5)
Ir,total
2. The proposed adaptive control strategy of DC microgrid
DC microgrid is constructed to provide an efficient and reliable
electrical source for specific areas, such as small communities or
buildings. The DC microgrid utilizes DC-based technologies and some
components such as RES, DC-DC converters, and the loads. Fig. 1 presents the proposed adaptive droop control strategy of the DC microgrid.
To achieve accurate power sharing, the current sharing errors are
minimized using adaptive PI controller. The current sharing errors are
obtained from bus current and total rated current. Then, the adaptive PI
controller adjusts the droop resistance. In addition, this new strategy
applies the droop voltage due to feeder line resistance in each converter.
In the primary loop, OOSMC is presented to control the converter output
and the inductor current as well. To enhance the performance of the
proposed controller, the controller gain is tuned by using a gradientbased approach. The proposed controller also considers the inductor
current, intermittent input voltage, and load variety as observers. The
voltage reference of OOSMC is obtained from the current sharing
strategy and control signal of secondary control. In the secondary loop,
the TD3 is employed for tuning the gains of PI controller to enhance the
voltage restoration in the DC bus. For improvement of accurate power
sharing in the DC microgrid, a new current sharing strategy is detailed in
the following subsection, while the use of OOSMC for primary loop and
TD3 for secondary loop are presented in detail in section III.
where ec,i and Ir,i denote the current sharing error and rated current of
the ith converter. Ibus indicates the bus load current, and Ir,total represents
the total rated current from all converters used in the system. Ibus and
Vbus are needed for communication among the converters to carry out
the proposed strategy. The adaptation strategy must be applied for Ωd,i
to decrease error of current sharing as small as possible. The adaptive PI
is employed to update Ωd,i and expressed as follows [19]:
⎛
⎞
∫t
⎝
Ωd,i = Φ Kp (t)ec,i (t) + Ki (t)ec,i (t)dt ⎠
(6)
2.1. The adaptive PI
0
Traditional PI controllers utilize fixed parameters. However, in
extreme conditions, these parameters may require to be adapted.
Consequently, traditional PI can fail to meet the required conditions.
Adaptive PI controllers are valuable in conditions where the dynamic of
the regulated system changes over time or when disturbances influence
ωd,i = Ωd,i + ωd,i,old
(7)
where Φ is a constant value, Ωd,i is the adaptive droop gain, and ec,i is the
current sharing error of the ith converter. In this study, the proposed
design can decrease executed data in low-bandwidth line because the
4
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
̂ in , and ̂
x2, V
δ define the estimation of x1 , x2 , Vin , and δ,
where ̂
x1, ̂
respectively, while K1 > 0 and K2 > 0 denote the gain of the observer. It
̃ in is =
is assumed that ̃
x1 is x1 − ̂
x1, ̃
x2 is x2 − ̂
x2, ̃
δ is = δ − ̂
δ, and V
̂
Vin − V in . Subsequently, (8) is subtracted from (9) to obtain (10) [20].
̃ in
V
(1 − d)
̃
̃
x1
x2 +
x˙ 1 = −
+ K1 ̃
L
L
̃
(1 − d)
δ
̃
̃
x1 − x2 + K2 ̃
x˙ 2 =
x2
C
C
̃ in , and ̃
where ̃
x1 , ̃
x2 , V
δ are the error estimation of x1 , x2 , Vin , and δ. The
quadratic Lyapunov function is utilized to gain the following adaptation
law [20].
)
(
1
1 2 1̃ 2
2
2
V=
L̃
x1 + C̃
x2 + ̃
(11)
δ + V in
2
γ1
γ2
Fig. 2. DC-DC boost converter equivalent circuit.
design only requires bus current and voltage variable for communication
among converters.
where γ 1 and γ 2 are the adaption gain of δ and Vin . The adaptation gains
are expressed by γ 1 > 0 and γ 2 > 0. Using the time derivative of (11), the
following formulation can be obtained.
(
)
(
)
1˙
1 ̂˙
2
2
̃ in ̃
V̇ = − K1 L̃
x1 −
(12)
δ +V
V in
x1 − K2 C̃
x2 − ̃
δ x2 ̃
x2 + ̂
γ1
γ2
3. Primary and secondary loop control design
In this study, OOSMC as shown in Fig. 1 is presented to regulate
output voltage of the converter in primary loop control. The OOSMC
uses an observer to estimate input voltage variations and load changes
accurately. The sliding surface of the controller is constructed using the
DC-DC boost converter model. The controller gain of OOSMC is then
tuned using a gradient-based approach to increase the efficiency of the
proposed technique. Furthermore, in the secondary loop control design,
a TD3 with a new reward function is proposed to provide voltage
restoration.
The negative definite must meet (12) to stabilize the observer design.
Therefore, the formula should be zero inside the bracket, and the
adaptation laws are defined in (13).
˙
̂
δ = − γ 1 x2 ̃
x2
3.1. DC-DC boost converter model
̂˙ in = γ 2 ̃
x1
V
Kirchhoff’s voltage and current law are used to determine the
modelling of the converter. The converter operates in either an open or
closed switch state [20]. Fig. 2 depicts the basic circuit of the boost
converter. The converter equation in state space expression is formulated as follows [37].
By substituting (13) into (12), (14) is obtained.
ẋ1 = −
2
2
(14)
Because (14) is negative definite, the stability of direct Lyapunov is
satisfied. It can be determined that ̃
x1 →0 and ̃
x2 →0 which is asymptotically stable.
3.3. OOSMC
(8)
An indirect control mechanism is used in the design since the converter possesses a non-minimum phase feature. In this scenario, the iL
should manage the Vout . The following equation is used to calculate the
set point signal of the inductor current ILref [21].
where d is the duty cycle, Vin indicates the input voltage, L describes
the inductor, iL represents the inductor current, C denotes the capacitor,
R denotes the resistor, Vc is the capacitor voltage, Vout denotes output
voltage, ẋ1 is the derivative of inductor current, ẋ2 is the derivative of
capacitor voltage, and δ is defined as the inverse of the resistor. The iL
and Vc are defined as x1 and x2 , respectively.
ILref =
2
Vref
̂
δ
̂ in
V
(15)
where Vref and ILref indicate the voltage and inductor current reference.
Subsequently, the switching surface is expressed as follows [38].
3.2. Observer design
For varying Vin and R estimation, a nonlinear observer controller is
constructed. The state variables x1 and x2 are provided to determine
̂ in and ̂
these quantities. V
δ or 1 are estimations of the unknown Vin and R,
̂
R
̂ in and ̂
respectively. Thus, V
δ are variables because their quantity are
changes depend on the conditions of input voltage supplied by the
source and the load that comes from the consumer side. The observer
design is, therefore, expressed as follows [20].
σ=̂
x1 −
2
Vref
̂
δ
̂ in
V
(16)
The fundamental regulation of SMC involves reducing the state
switching surface to zero and maintaining it. The following step involves
substituting derivative (16) into (15), yielding the following equation.
σ̇ = ̂
x˙ 1 −
̂ in
V
(1 − d)
̂
̂
x˙ 1 = −
x2 +
x1)
+ K1 (x1 − ̂
L
L
̂
δ
(1 − d)
̂
̂
x˙ 2 =
x 1 − x2 + K2 (x2 − ̂
x2)
C
C
(13)
V̇ = − K1 L̃
x1 − K2 C̃
x2
1
(1 − d)
x2 + Vin
L
L
δ
(1 − d)
x1 − x2
ẋ2 =
C
C
(10)
2
2
Vref
˙ Vref ̂ ̂˙
̂
δ V in
δ+
̂ in
̂ 2
V
V
in
(17)
By using (9), (13) and (17), the switching controller can be obtained in
(18).
(9)
5
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
Fig. 3. Network diagram: (a) Actor; (b) Critic.
Fig. 4. TD3-based controller for a secondary loop in DC microgrid.
Table 1
Converter, switching surface, and controller parameters.
System
Parameter
Symbol
Value
Units
Boost Converter
Inductor
Capacitor
Resistor Load
L
C
R
100
47
5
μH
μF
Adaptation Gains
γ1
γ2
K1
K2
T
100
2.104
5.104
5.104
0.0309
−
−
−
−
−
SMC Switching Surface
Observer Gains
OOSMC
Controller gain
Table 3
Parameters of GA and PSO.
kΩ
Methods
Gains
Value
TD3
Kp
Ki
Φ
1.0271
995.1308
103
60
2x106
η1
η2
Parameters
Value
GA
Population size
Crossover percentage
Mutation percentage
Mutation rate
100
0.7
0.3
0.1
PSO
Population size
Personal leading coefficient c1
Global learning coefficient c2
Inertia weight
Inertia weight damping ratio
100
1.5
2
1
0.99
⎛
Table 2
PI gains of for TD3 and adaptive method.
Adaptive
Method
2
γ1 LVref
2
γ2 LVref
⎞
̂ in + K1 L̃
̂x1 ⎟
x1 +
x ̃
x +
2 δ̃
⎜V
̂V in 2 2
̂V in
⎟
⎜
deq = 1 − ⎜
⎟
̂
⎠
⎝
x2
(18)
By modifying the signal control of (18), a better performance can be
found by considering Lyapunov function V = 12σ2 . For ensuring the
stability of the system, time derivative should be negative definite V̇ =
σσ̇ < 0. The optimal natural control is proposed as follows.
dn = − T σ
6
(19)
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
By combining (18) and (19), the overall control equation is formulated
in (20) [21].
⎞
⎛
2
2
γ1 LVref
γ2 LVref
̂ in + K1 L̃
̂x1 ⎟
x1 +
x2 ̃
x2 +
2 δ̃
⎜V
̂V in
̂V in
⎟
⎜
du = deq + dn = 1 − ⎜
(20)
⎟ − Tσ
̂
⎝
⎠
x2
where T is the gain of natural control, which is positive. Using the
control signal (20), σ̇ = − T ̂xL2 σ therefore σ σ̇ = − T ̂xL2 σ 2 < 0. Thus, (20) is
asymptotically stable. In this study, the optimal controller gain T is
proposed by applying the gradient-based technique to enhance the
proposed method. Gradient methods are used to sole the issues of the
form minn f(x) during optimization, with the gradient of the function at
x∊R
the current location defining the search direction. In this case, x is the
integral time absolute error (ITAE) between Vref and Vout , while f(x)
represents the converter in (8), state estimate in (9), error estimate in
(10), adaptation laws in (13), and formulation of SMC controller in (20).
The ITAE is calculated by adding the regions above and below the
reference signal and process value, then multiplying by time.
3.4. TD3
A sizable network of the activity and critics in a DDPG algorithm is
illustrated in Fig. 3. The critic employs the temporal differentiation
method to update its settings. On the other hand, the actor is modified
(
)
through α = μ s|θμ + N. To update the parameters of actor θμ and critic
θQ , exponential smoothing is employed, and is formalized as follows
[39]:
Fig. 5. The performances of the proposed OOSMC, GA, and PSO.
Fig. 6. The performances of proposed TD3 and traditional PI.
Fig. 7. Comparison response of traditional and proposed droop control strategy
for power sharing among three converters with equal power rating.
Table 4
Performances of TD3 and traditional PI.
Methods
TD3
Traditional PI
Starting
24.076 V
24.322 V
Load Change
Voltage input change
12 W
15 W
13 W
V1 (− 8 V)
V2 (+8 V)
V3 (+8 V)
0.655 V
0.656 V
0.528 V
0.672 V
0.264 V
0.332 V
0.283 V
0.379 V
0.286 V
0.354 V
0.361 V
0.382 V
7
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
Fig. 8. The response of DC Bus Voltage under equivalent powerrated converters.
Fig. 10. Response of proposed droop control strategy for power sharing among
three converters with unequal power rating.
Fig. 9. Comparison response of conventional and proposed droop control
strategy for current sharing among three converters with equal power rating.
Fig. 11. The performance of DC Bus Voltage under unequal powerrated converters.
θμʹ = τθμ + (1 − τ)θμʹ (actor)
where γ is the discount factor with γ≪1 while r is the reward function.
The temporal difference (TD) error uses y = r +γQʹ(sʹ, aʹ) and decreases
the loss function to update the critics’ parameters, as expressed in (23)
[33].
θQʹ = τθQ + (1 − τ)θQʹ (critic)
(21)
where θμ , θQ , and τ are the update of actor, critic and constant. Using the
critic network, the Bellman equation and action value estimation are
obtained [40].
Qʹ(s, a) = E[r(s, a) + γQʹ(sʹ, aʹ) ]
LF =
(22)
M
1 ∑
(yi − Q(si , ai ) )2
M i=1
(23)
where M is the number of samples and y is the temporal difference error.
8
Ain Shams Engineering Journal 15 (2024) 102899
R. Firmansyah and M.A.M. Ramli
Fig. 12. The response of proposed droop control strategy for current sharing
among three converters with unequal power rating.
Fig. 14. The performance of DC Bus Voltage under disturbances.
Fig. 15. Performance of proposed droop control strategy for current sharing
under disturbances.
Fig. 13. Response of proposed droop control strategy for power sharing with
the variations of disturbance.
policies is negatively influenced if this overestimate persists during
training. To mitigate this issue, the double Q-learning and double deep
Q-network (DQN) techniques have been employed. These strategies
separate the Q-value and the updates of action selection using two
networks.
The following state quantity, which forms the double Q-value networks described in the double Q-learning method, is calculated in (25)
[34].
(
(
))
y1 = r + γQθʹ1 sʹ, μʹ sʹ|θμʹ
The policy gradient denoted in (24) is applied to maximize the expected
discounted reward [33].
∇θ μ J ≈
M [
( ⃒ )⃒ ]
1 ∑
∇a Q(s, a) |s=si ,a=μ(si |θμ ) ∇θμ μ s⃒θμ ⃒s
i
M i=1
(24)
The TD3 algorithm is an upgrade to the DDPG approach. TD3 computation is comparable to the DDPG algorithm, where value function
overestimation impacts the performance of Q-learning. The updating of
9
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
Fig. 16. Performance of the proposed droop strategy for accurate power
sharing during fault.
Fig. 18. Performance of the proposed droop control strategy for current
sharing during fault.
clip(N(0, σ ), − c, c )
(28)
where N is a random noise function.
3.4.1. Environment construction to train the agent.
In this study, the PI parameters are trained to be optimized by the
TD3 agent. To choose the best course of action, the agent finds information regarding the environmental states at each time step. First, it is
necessary to establish the observation formula e to build the training
environment. e is the error between the output voltage and Vref . This
error is multiplied by negative, and penalty then will be added with
signal control that comes from the system output. This expression is
called reward function. The RL agent block is next attached to the signal
through this formula. The RL agent’s negative reward function, which
offers feedback regarding the system convergence, is then established.
The agent is directed to pursue activities that maximize these values.
The DC microgrid control is the output agent. Its value is set using a
strategy that optimizes the reward. For optimal PI parameter determination, selecting the right incentive function is crucial.
A reward function that produces quick convergence, has high performance, and requires little processing must be chosen because the
agent updates its parameters continuously. The following is the proposed formulation for the new reward function.
[
]
r = − λ |er |β + |ur |β
(29)
Fig. 17. The response of DC Bus Voltage during fault.
(
(
))
y2 = r + γQθʹ2 sʹ, μʹ sʹ|θμʹ
where λ and β are the constants and ur represents the r agent control
action, while er is the error of r agent. In this expression, the negative
sign results in the highest reward and the lowest mistake.
(25)
The TD-error is formulated in (26) [34].
ʹ ʹ
y = r + γmin
i=1,2 Qi (s , aʹ)
3.4.2. Creating TD3 agent
In utilizing actor model, TD3 chooses a subsequent action after
acquiring the observed information. Deep neural network (DNN) receives the action output. Then, the observed input is the first stage in
creating this actor. To model a neural network for PI controller, the
system is linked to the layer with error and integral error as described as
follows [37].
(26)
The application of clipped typical distribution noise is used. The
outcome in the updated target is expressed in (27) and (28) [39].
(
(
)
)
(27)
y = r + γQθ sʹ, μʹ sʹ|θμʹ + ε
10
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
Fig. 19. Hardware setup.
Fig. 20. Bus voltage waveform when the input voltage of DC source 1 is decreased.
11
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
Fig. 21. Power sharing waveform when the input voltage of DC source 1 is decreased.
ur =
[
∫
er dt
] [
er * Kii
Kpp
]ʹ
4. Simulation results
(30)
A numerical simulation is carried out by using MATLAB/Simulink to
verify the efficacy of the proposed droop control strategy. The DC
microgrid comprises three converters, with the rating of the second
converter being double that of the first converter, and the rating of the
third converter being triple that of the first converter. Table 1 provides
details of the switching surface, Gain of OOSMC, and DC-DC boost
converter characteristics employed in the research. Table 2 lists the
parameters of the PI controller for adaptive method and TD3. For RF, the
constant values of λ and β are 0.1 and 0.6, respectively. The starting
computation of the system has the following parameters: Vbus,ref = 24V,
f = 100kHz, simulation time t = 0.2s. The initial conditions chosen for
the system to ensure the presence of the sliding mode control are as
̂ i (0) = 10̂
follows: ̂
x 1 (0) = 0.8, ̂
x 2 (0) = 6, V
φ (0) = 0.0048.
where Kpp and Kii denotes the neural network weights. The agent estimates the long-term reward. To build the criticism, a DNN is constructed
using an output and two inputs.
The discount factor used in this study is 0.99. Fig. 4 indicates the TD3
algorithm to control the DC microgrid.
3.4.3. Training and validating the agent
One thousand episodes of the training program developed for this
study are conducted, with each episode consisting of 100-time steps. The
agent’s training is terminated when the average cumulative reward over
100 subsequent sessions exceeds − 355. The agent can now successfully
control the converter’s output. Subsequently, a simulation is used to
verify the learned agent. The absolute weights of the actor representation are the parameters of PI controller. The weights and controller gains
are derived from the actor’s learnable parameters. The formula of the PI
controller is expressed as follows [41].
uV = Kpt (t)eV (t) +
∫t
Kit (t)eV (t)dt
In this study, the efficacy of the presented droop resistance strategy is
examined in several conditions, including scenarios with equivalent
power-rated converters, different power-rated converter, various system
disturbances, and sudden converter failures.
(31)
4.1. Performance of OOSMC
0
To check the effectiveness of the proposed OOSMC which uses the
gradient-based technique, various disturbances such as input voltage
and load resistance variations are implemented into the system.
Furthermore, the proposed OOSMC is compared against GA and PSO.
For the best result, parameters of GA and PSO algorithm utilized in this
where uV is the control signal that comes from secondary controller, Kpt
is the proportional gain of secondary loop control, Kit is the integral gain
of secondary loop control, and eV is the error signal between bus voltage
reference and desired bus voltage.
12
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
Fig. 22. Bus voltage waveform when the input voltage of DC source 2 is increased.
study are listed in Table 3. The controller gains of GA and PSO are
0.0203 and 0.0261, respectively. Fig. 5 shows the performances of the
proposed OOSMC, GA, and PSO. As observed in the figure, the total
simulation time is 0.5 s. In the beginning of simulation, the proposed
OOSMC has smaller overshoot of 24.303 V compared to GA that has
overshoot of 25.391 V, and PSO that has overshoot of 25.473 V. At t =
0.1s, the voltage input is decreased from 12 V to 4 V. The voltage deviation of all methods is 0.525 and the recovery time is 0.0061 s. At t =
0.2s, the voltage input is increased from 4 V to 12 V. In this state, the
proposed OOSMC has the smallest voltage deviation of 25.495 V
compared to GA that has voltage deviation of 26.371 V and PSO that has
voltage deviation of 26.423 V. Meanwhile the recovery time of the
proposed OOSMC is 0.014 s, GA is 0.019 s and PSO is 0.02 s. The load
change is implemented at t = 0.3s. In this case, the load is decreased
from 5kΩ to 100Ω. The voltage deviation and recovery time of all
methods are similar. At t = 0.4s, the load is increased from 100Ω to 5kΩ.
The voltage deviation of the proposed method is identical, 0.465 V. But
the recovery time of the proposed method is faster than GA and PSO.
Furthermore, the ITAE of the proposed method is 0.006564, while GA is
0.01057 and PSO is 0.01082. According to the results, in comparison to
GA and PSO, the proposed method generates smaller overshoot, smaller
voltage deviation, and faster recovery time.
controller. To verify the performance of TD3, the traditional PI applied
in [42] is used for comparison. The parameter of proportional is 0.7651,
while the parameter of integral is 1159. In addition, some disturbances
such as load variation and voltage input change are applied to verify the
superiority of the proposed method. The performances of the proposed
method and traditional PI are shown in Fig. 6. It can be observed that the
overall simulation time is 0.4 s. In the starting condition, the proposed
method has smaller overshoot than the traditional PI. Then, at t = 0.05s
the 12 W load change is applied to the system. The voltage deviation of
all methods is similar. At t = 0.1s, the strategy is changed to the proposed adaptive droop control. The voltage deviation of the proposed
TD3 is smaller than the traditional PI. The load resistance variation is
changed to 15 W and 13 W at t = 0.15s and t = 0.2s, respectively. In this
condition, the proposed method has lower voltage deviation than the
traditional PI. The voltage input change is applied to the system at t =
0.25, t = 0.3, and t = 0.35. In this case, the performance of TD3 is
better than the traditional PI for all input voltage changes. The detailed
performances of the TD3 and the traditional PI are presented in Table 4.
The results show that the proposed TD3 outperforms traditional PI.
4.3. Equivalent power-rated converters
To verify the superiority of the proposed droop control strategy,
three converters with equivalent power ratings are used. The investigation involves three distinct conditions. In the first condition, the system operates using conventional droop control for 0 ≤ t < 0.1s. Then, at
4.2. Performance of TD3
The TD3 method is employed to select the parameters of the PI
13
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
Fig. 23. Power sharing waveform when the input voltage of DC source 2 is increased.
t ≥ 0.05s the power load is increased to 12W. In the last condition at
t ≥ 0.1s, the proposed adaptive droop control strategy is applied in the
DC microgrid. Fig. 7 depicts the power generated by each converter and
the power consumed by the load. When the conventional droop strategy
is implemented, the power load is 11.53 W and sources 1, 2 and 3 deliver
3.66W, 3.84 W and 4.03 W, respectively. At t ≥ 0.05s the power load is
increased to 12W and the power outputs of sources 1, 2, and 3 are
3.82W, 4W, and 4.18W, respectively. The proposed droop control is
activated at t ≥ 0.1s. Fig. 7 illustrates that each source can transfer 4 W
to the load, achieving an identical power sharing objective. Despite
there are some minor oscillations in the response of all converters, the
system quickly reaches the desired value due to the performance of the
proposed controller. The maximum overshoot of the first, second and
third converter are 2.8 W, 0.2 W, and 2 W, respectively. The second
controller has the smallest overshoot in the response because its initial
condition is close to identical power sharing objective of 4 W.
Fig. 8 displays DC bus voltage when both traditional droop control
and the proposed droop control are applied. At t ≥ 0.05s, a voltage deviation occurs due to the load change. The voltage deviation is 0.4 V
below the voltage reference. However, in this case the proposed secondary controller will work to make the response return to the voltage
reference. Another voltage deviation is observed at t ≥ 0.1s, where the
voltage deviation occurs because of the change from conventional droop
control to the proposed adaptive droop control strategy. In this moment,
the voltage deviation is 2.9 V due to adaptation of power load of three
converters. However, with the use of the proposed secondary loop
controller, the DC bus voltage can be maintained at the reference value
of 24V. This performance indicates that the proposed secondary
controller has parameter tuning ability which is important for voltage
restoration and can improve controller stability. Fig. 9 presents the
current sharing among three converters with identical power ratings and
the load current in DC microgrid. As observed in the figure, in classical
droop control for 0 ≤ t < 0.1s, the current waveforms of all converters
are unequal. Then, the proposed droop strategy is applied for t ≥ 0.1s. In
this case, there is an oscillation due to the change of control. The current
change of converter 2 is smaller than other converters because its initial
current value is close to 0.167 A. Afterwards, the current of all converters has equal magnitude of 0.167A, indicating that the proposed
strategy can provide accurate current sharing.
4.4. Different power-rated converters
To further verify the efficacy of the proposed strategy, converters
with unequal power ratings are considered. In this scenario, the second
converter has a power rating double that of the first converter and the
third converter has a power rating triple that of the first converter.
Fig. 10 presents the power consumed by the load and the power produced by each converter for different power ratings of the converters. As
shown in the figure, for 0 ≤ t < 0.1s, all converters have equal power
ratings, and each converter can produce an equal power of 4 W to satisfy
the 12 W load demand. Furthermore, when the power ratings of converters are changed at t ≥ 0.1s, and the proposed droop control is
applied, accurate power sharing among three converters with nonidentical power ratings can be achieved. As observed in Fig. 10 the
14
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
Fig. 24. Bus voltage waveform when the load resistance is increased.
load consumes 12W, while the first, second and third converters
generate 2W, 4W, and 6W, respectively.
The performance of the DC bus voltage using the proposed secondary
controller with converters of unequal power ratings is depicted in
Fig. 11. As demonstrated in the figure, at t ≥ 0.05s and t ≥ 0.1s the DC
bus voltage experiences deviations because of load changes and the
transition from equal power ratings to unequal power ratings. The first
deviation is 0.38 V, and the second deviation is 1.4 V. All converters
must satisfy the accurate power sharing for different power rating.
However, by applying the proposed secondary loop control specifically
TD3-PI, the voltage deviation can be eliminated, and the DC bus voltage
can be kept at 24V while the sources can deliver accurate power to the
load. This performance indicates that the proposed controller can provide parameter tuning ability which is important for voltage restoration.
Fig. 12 depicts the performance of the proposed strategy for current
sharing with non-identical power rating. As shown in the figure, at t >
0.05s converters with identical power ratings can accurately deliver a
current of 0.167A, resulting in total load current of 0.5A. Additionally,
at t > 0.1s, the power ratings of the converters are changed from being
equal to different. To produce a load current of 0.5A, each converter
delivers 0.082A, 0.167A and 0.251A for the first, second and third
converter, respectively.
such as load and input voltage change, as shown in Fig. 13. The load
demand change is considered as a disturbance since the load demand on
the consumer side may vary throughout the day. In this study, the type
of load demand is CPL, which is a nonlinear load. At the beginning of the
simulation (0 ≤ t < 0.4s), the proposed droop control is applied with
converters of different power ratings. At t = 0.05s, t = 0.1s, and t =
0.15s the power loads are changed to 12W, 15W, and 13W, respectively.
As observed in Fig. 13, for the power load is 12 W, the first, second, and
third converter provide 2 W, 4 W, and 6 W, respectively. For the power
load is 15 W, the first, second, and third converter generate 2 W, 4 W,
and 6 W, respectively. Meanwhile the 2.16 W, 4.33 W, 6.5 W are
delivered by converter 1, 2, and 3, respectively when the power load is
13 W. The sources consistently deliver accurate power to the load
despite variations in the load. Another disturbance that affects the
performance is input voltage changes. The input voltage may vary,
especially in systems that utilize RES. In the simulation, at t = 0.2s, the
input voltage of the source 1 is changed to 4V, while at t = 0.25s and t =
0.3s, the Vin of the sources 2 and 3 are changed to 16V and 6V respectively. As observed in Fig. 13, all converters experience small change
even only one converter that deals with input voltage change. This reaction occurs because other converters will satisfy the power sharing to
the load requirement. So, accurate power sharing can be achieved using
the proposed droop control strategy when the system encounters variations in the input voltage of all converters. Furthermore, at t ≥ 4s, the
proposed droop control strategy is deactivated. As displayed in the
figure, each converter cannot share accurate power sharing.
Fig. 14 displays the performance of DC bus voltage under load
4.5. Performance of the proposed strategy with various disturbances
This case is conducted to investigate the performance of the proposed
adaptive droop control strategy when the system encounters disturbance
15
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
Fig. 25. Power sharing waveform when the load resistance is increased.
changes and input voltage variations. The DC bus voltage experiences
deviations at t = 0.1s and t = 0.15s due to the load changes. The voltage
deviation of 2.1 V at t = 0.1s is bigger, than at t = 0.15s, because the
load change of 3 W at t = 0.1s is bigger, than at t = 0.15s. Similarly, at
t = 0.2s, t = 0.25 and t = 0.3s the voltage deviation occurs because of
the input voltage changes. The biggest voltage deviation because of
input voltage change occurs at t = 0.3s. The most significant voltage
deviation occurs at t = 0.4s when the proposed strategy is switched to
classical droop control. This event occurs because the change of droop
control strategy and the response must change to inaccurate power
sharing. However, by using the secondary loop control, TD3-PI, the DC
bus voltage can return to the desired value, whilst the converters
maintain power sharing accuracy. Its superior performance in maintaining stability and fast recovery under varying conditions further
confirms its applicability and reliability. The performance of the proposed droop strategy for current sharing under disturbances is shown in
Fig. 15. As mentioned earlier, the system is tested with load variations
and input voltage changes. When the proposed droop control is used, the
converters deliver accurate current sharing even in the presence of
disturbances. However, when the proposed strategy is disabled, and
traditional droop control is activated, the system fails to deliver proper
current sharing.
converter 1. From 0.05s ≤ t ≤ 0.1s, all converters are activated. All
converters accurately share power to the load. Converters 1, 2, and 3
deliver 2W, 4W, and 6W, respectively to meet 12W power load. However, at t ≥ 0.1s a sudden failure of converter 1 occurs. As observed in
the figure, the power output of converter 1 drops to zero, while the
converters 2 and 3 provide 4.8W and 7.2W, respectively to meet the
12W power load. There are small changes in converters 2 and 3 since
these converters must increase the power to meet the power load.
Fig. 17 presents the DC bus voltage when the proposed secondary
loop control is implemented. There is a voltage deviation at t = 0.1s
when the failure of converter 1 is applied. The voltage deviation of 2.1 V
is high, because one converter is off and other converters must increase
the power to meet the load power requirement. However, the proposed
secondary controller can handle the voltage deviation and return the
voltage to the reference. Furthermore, the performance of the proposed
droop control strategy for current sharing during fault is shown in
Fig. 18. As displayed in the figure, in the beginning of simulation all
converters can share accurate current sharing to meet 0.5 A current load.
Then, at t ≥ 0.1s, the converter 1 is off and converters 2 and 3 increase
the current to meet the requirement. The responses indicate that the
converters can share accurate current for failure condition of converter
1.
4.6. Performance of the proposed strategy during faults
5. Experimental validation
To further verify the consistency of the proposed strategy, a sudden
failure of converter 1 is investigated. Fig. 16 presents a performance
comparison of the proposed strategy before and after the failure of
A laboratory-based hardware design shown in Fig. 19 has been built
to further validate the efficacy of the proposed strategy. The hardware
comprises a DC power supply (Siglent SPD3303X-E), two converters, a
16
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
Fig. 26. Bus voltage waveform when the load resistance is decreased.
DC electronic load (Siglent SDL1020X-E), a two-channel digital oscilloscope (Siglent SDS1202X-E), a digital multimeter (Siglent
SDM3045X), a data acquisition device (NI Elvis II + ), and a Dell personal computer. The converters employed in this paper utilize the parameters in Table 1. The switching component is IRFZ44N. Initially,
voltage reference, frequency, DC source 1, DC source 2, and load resistance are 24 V, 100 kHz, 10 V, 12 V, and 3 kΩ, respectively. In this study,
the second converter has a power rating double that of the first
converter.
The input voltage changes of DC source 1 are the first test during
experimental validation. As previously mentioned, this system is applied
since the input voltage changes continuously. The experimental waveform of the bus voltage is presented in Fig. 20. In this case, the input
voltage is reduced from 10 V to 8 V. Although there is a change in the
input voltage, the bus voltage returns to 24 V voltage reference with
1.43 V voltage deviation and 1.75 s recovery time. Fig. 21 shows the
experimental waveform of accurate power sharing. To achieve accurate
power sharing of 12 W load consumption, the DC source 1 and 2 deliver
4 W and 8 W, respectively. There is 1.4 W power deviation of load power
when the input voltage of DC source 1 is decreased. However, because of
the proposed strategy, the load power can return to the desired value.
The input voltage changes of DC source 2 are also used to examine
the effectiveness of the method. The input voltage is increased from 10 V
to 12 V. The experimental waveform of the bus voltage is shown in
Fig. 22. When the input voltage change is applied, the bus voltage has
1.79 voltage deviation and 1.5 s recovery time. The waveform of accurate power sharing is shown in Fig. 23. When the load power is 12 W, the
sources 1 and 2 deliver accurate power sharing of 4 W and 8 W,
respectively. The power of source 1 experiences 0.6 W power deviation,
while the power of source 2 experiences 1.2 W power deviation.
To assess the superiority of the proposed strategy, the load resistance
variation is implemented. In this case, the load resistance is changed
from 3kΩ to 7kΩ. The bus voltage waveform is depicted in Fig. 24. When
the load resistance is increased, the bus voltage has voltage deviation of
2.3 V and recovery time of 2.25 s. Fig. 25 presents the waveform of
accurate power sharing. When the load power consumes 12 W, the
sources 1 and 2 supply 4 W and 8 W, respectively. Because the load
resistance is increased, the load power encounters 2.2 W power deviation. However, the load power can return to 12 W load power.
Another load resistance variation is used to examine whether the
proposed strategy can overcome the load change. In this case, the load
resistance is reduced from 7kΩ to 3kΩ. Fig. 26 depicts the bus voltage
waveform. When the load resistance is reduced, the voltage deviation of
bus voltage is 2.3 V. Meanwhile recovery time is 2.25 s. Fig. 27 shows
the waveform of accurate power sharing. The 12 W load power is supplied by 4 W source 1 and 8 W source 2. The power of source 1 occurs
0.8 W power deviation while the power of source 2 occurs 1.2 W power
deviation.
6. Conclusions
This study proposes a new adaptive droop control strategy for accurate power sharing and voltage restoration. To enhance accurate
power sharing, the error current sharing is expressed by utilizing bus
17
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
CRediT authorship contribution statement
Rifqi Firmansyah: Conceptualization, Methodology, Software,
Validation, Writing – original draft, Writing – review & editing. Makbul
A.M. Ramli: Methodology, Validation, Writing – original draft, Writing
– review & editing.
References
[1] Awad H, Bayoumi EHE, Soliman HM, Ibrahim AM. Invariant-set design of robust
switched trackers for bidirectional power converters in hybrid microgrids. Ain
Shams Eng J 2023;14. https://doi.org/10.1016/j.asej.2023.102123.
[2] Ullah Z, Wang S, Lai J, Azam M, Badshah F, Wu G, et al. Implementation of various
control methods for the efficient energy management in hybrid microgrid system.
Ain Shams Eng J 2023;14. https://doi.org/10.1016/j.asej.2022.101961.
[3] Abou El-Ela AA, Mosalam HA, Amer RA. Optimal control design and management
of complete DC- renewable energy microgrid system. Ain Shams Eng J 2023;14.
https://doi.org/10.1016/j.asej.2022.101964.
[4] Tazy A, Samy MM, Ghalib MA. Enhancement and Control of Reactive Power
Sharing Utilizing Circulating Current in AC Microgrids. 2022 23rd Int. Middle East
Power Syst. Conf. MEPCON 2022, 2022. DOI: 10.1109/
MEPCON55441.2022.10021751.
[5] Wang C, Duan J, Fan B, Yang Q, Liu W. Decentralized high-performance control of
DC microgrids. IEEE Trans Smart Grid 2019;10:3355–63. https://doi.org/10.1109/
TSG.2018.2825199.
[6] Abhishek A, Ranjan A, Devassy S, Kumar Verma B, Ram SK, Dhakar AK. Review of
hierarchical control strategies for DC microgrid. IET Renew Power Gener 2020;14:
1631–40. https://doi.org/10.1049/iet-rpg.2019.1136.
[7] Lai J, Lu X, Yu X, Yao W, Wen J, Cheng S. Distributed multi-DER cooperative
control for master-slave-organized microgrid networks with limited
communication bandwidth. IEEE Trans Ind Informatics 2019;15:3443–56. https://
doi.org/10.1109/TII.2018.2876358.
[8] Mehdi M, Kim CH, Saad M. Robust centralized control for DC islanded microgrid
considering communication network delay. IEEE Access 2020;8:77765–78. https://
doi.org/10.1109/ACCESS.2020.2989777.
[9] Mosayebi M, Sadeghzadeh SM, Khooban MH, Guerrero JM. Decentralised nonlinear I-V droop control to improve current sharing and voltage restoration in
DCNG clusters. IET Power Electron 2020;13:248–55. https://doi.org/10.1049/ietpel.2019.0263.
[10] Mojica-Nava E, Rey JM, Torres-Martinez J, Castilla M. Decentralized switched
current control for DC microgrids. IEEE Trans Ind Electron 2019;66:1182–91.
https://doi.org/10.1109/TIE.2018.2833020.
[11] Xing L, Mishra Y, Guo F, Lin P, Yang Y, Ledwich G, et al. Distributed secondary
control for current sharing and voltage restoration in DC microgrid. IEEE Trans
Smart Grid 2020;11:2487–97. https://doi.org/10.1109/TSG.2019.2956515.
[12] Ahsan M, Alsenani TR. Distributed consensus control for voltage tracking and
current distribution in DC microgrid. Ain Shams Eng J 2023. https://doi.org/
10.1016/j.asej.2023.102363.
[13] Guo F, Xu Q, Wen C, Wang L, Wang P. Distributed secondary control for power
allocation and voltage restoration in islanded DC microgrids. IEEE Trans Sustain
Energy 2018;9:1857–69. https://doi.org/10.1109/TSTE.2018.2816944.
[14] Guo F, Wang L, Wen C, Zhang D, Xu Q. Distributed voltage restoration and current
sharing control in islanded DC microgrid systems without continuous
communication. IEEE Trans Ind Electron 2020;67:3043–53. https://doi.org/
10.1109/TIE.2019.2907507.
[15] Vandoorn TL, Vasquez JC, De Kooning J, Guerrero JM, Microgrids VL. Hierarchical
control and an overview of the control and reserve management strategies. IEEE
Ind Electron Mag 2013;7. https://doi.org/10.1109/MIE.2013.2279306.
[16] Panda M, Devara VB, Maity T. A novel dc bus-signaling based power management
strategy for dc microgrid. Int Trans Electr Energy Syst 2021;31. https://doi.org/
10.1002/2050-7038.12758.
[17] Al SKA, Sulaiman N, Wahab NIA, Hizam H. Energy management and voltage
control in microgrids using artificial neural networks, PID, and Fuzzy Logic
Controllers. Energies 2022;15. https://doi.org/10.3390/en15010303.
[18] Gorijeevaram Reddy PK, Dasarathan S, Krishnasamy V. Investigation of adaptive
droop control applied to low-voltage DC microgrid. Energies 2021;14. https://doi.
org/10.3390/en14175356.
[19] Mokhtar M, Marei MI, El-Sattar AA. An adaptive droop control scheme for DC
microgrids integrating sliding mode voltage and current controlled boost
converters. IEEE Trans Smart Grid 2019;10:1685–93. https://doi.org/10.1109/
TSG.2017.2776281.
[20] Muktiadji RF, Ramli MAM, Bouchekara HREH, Milyani AH, Rawa M,
Seedahmed MMA, et al. Control of boost converter using observer-based
backstepping sliding mode control for DC microgrid. Front Energy Res 2022;10.
https://doi.org/10.3389/fenrg.2022.828978.
[21] Pandey SK, Patil SL, Phadke SB. Comment on PWM-based adaptive sliding-mode
control for boost DC-DC converters. IEEE Trans Ind Electron 2018;65. https://doi.
org/10.1109/TIE.2017.2764872.
[22] Pandey SK, Veeranna K, Patil SL, Phadke SB. Uncertainty estimator based sliding
mode control schemes for multimode noninverting buck-boost DC-DC converter.
IFAC-PapersOnLine, vol. 53, 2020. DOI: 10.1016/j.ifacol.2020.06.093.
Fig. 27. Power sharing waveform when the load resistance is decreased.
current and total rated current. It is controlled by the adaptive controller
to adjust droop resistance. Furthermore, the proposed design considers
the droop voltage in the feeder line resistance. In the primary loop,
OOSMC is proposed to regulate the output voltage of each converter. In
the controller design, factors like the load disturbance, intermittent
input voltage, and inductor current estimation are considered as observers. Subsequently, the optimum controller gain of OOSMC is proposed utilizing a gradient-based technique to enhance the performance
of the proposed strategy. Moreover, TD3 with a novel reward function is
proposed to regulate the voltage restoration in the secondary loop
control. The TD3 is used to optimally tune the parameter of PI controller.
The superiority of the proposed strategy is tested using various testing
scenarios such as sudden failure and CPL. The results show that the
proposed OOSMC has better performance compared to GA and PSO
while the performance of TD3 is better than the traditional PI. By
employing the proposed adaptive droop control strategy, converters
with equivalent power ratings can accurately deliver equal 4 W power to
the load compared to classical droop control that cannot transfer power
to the load equally. In current sharing, the proposed strategy can provide
accurate current sharing of 0.167 A for each converter, while the conventional droop control cannot share accurate current sharing equally.
Additionally, the proposed droop control strategy can also provide accurate power sharing of 2 W, 4 W, and 6 W to the load in different power
ratings, for the first, second and third converter, respectively. The proposed strategy can also achieve accurate power sharing for other cases
such as various disturbances and sudden failure in the converter.
Furthermore, the proposed secondary loop controller, TD3-PI, can keep
the voltage output of DC bus at the desired value in all conditions
including equivalent power-rated converter, different power-rated
converter, various kinds of disturbance and a sudden failure of converter. These findings collectively provide improvements in grid stability, energy efficiency, and overall system reliability. They are crucial
for advancing the implementation and integration of DC microgrid
technology in various applications. Additional research is recommended
to enhance its performance by improving the switching surface of SMC.
RL techniques, such as model-based policy optimization agents and
proximal policy optimization agents should be used for optimizing
specific SMC parameters.
18
R. Firmansyah and M.A.M. Ramli
Ain Shams Engineering Journal 15 (2024) 102899
[38] Oucheriah S, Guo L. PWM-based adaptive sliding-mode control for boost DC-DC
converters. IEEE Trans Ind Electron 2013;60. DOI: 10.1109/TIE.2012.2203769.
[39] Nguyen TT, Nguyen ND, Nahavandi S. Deep reinforcement learning for multiagent
systems: a review of challenges, solutions, and applications. IEEE Trans Cybern
2020;50. https://doi.org/10.1109/TCYB.2020.2977374.
[40] Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, et al. Humanlevel control through deep reinforcement learning. Nature 2015;518. https://doi.
org/10.1038/nature14236.
[41] Muktiadji RF, Ramli MAM, Seedahmed MMA, Uswarman R, Endryansyah. Power
Sharing Control and Voltage Restoration in DC Microgrid Using PI Fuzzy. 2022
Fifth Int. Conf. Vocat. Educ. Electr. Eng., 2022, p. 130–5. DOI: 10.1109/
ICVEE57061.2022.9930441.
[42] Mohammed N, Callegaro L, Ciobotaru M, Guerrero JM. Accurate power sharing for
islanded DC microgrids considering mismatched feeder resistances. Appl Energy
2023;340. https://doi.org/10.1016/j.apenergy.2023.121060.
[23] Talbi B, Krim F, Laib A, Sahli A, Krama A. PI-MPC switching control for DC-DC
boost converter using an adaptive sliding mode observer. 2020 Int Conf Electr Eng
ICEE 2020. https://doi.org/10.1109/ICEE49691.2020.9249934.
[24] Ghalib MA, Samy MM. Magnification of Performance Operation for Low Voltage
DC Microgrids Based on Adaptive Droop Control Technique. 22nd Int. Middle East
Power Syst. Conf. MEPCON 2021 - Proc., 2021. DOI: 10.1109/
MEPCON50283.2021.9686201.
[25] Wongkhead S, Tunyasrirut S. Implementation of a dsp- tms320f28335 based state
feedback with optimal design of pi controller for a speed of bldc motor by ant
colony optimization. Prz Elektrotechniczny 2021;97. https://doi.org/10.15199/
48.2021.07.02.
[26] Belgaid Y, Helaimi M, Taleb R, Youcef MB. Optimal tuning of PI controller using
genetic algorithm for wind turbine application. Indones J Electr Eng Comput Sci
2019;18. https://doi.org/10.11591/ijeecs.v18.i1.pp167-178.
[27] Faisal SF, Beig AR, Thomas S. Time domain particle swarm optimization of PI
controllers for bidirectional VSC HVDC light system. Energies 2020;13. https://doi.
org/10.3390/en13040866.
[28] Hussien AM, Hasanien HM, Mekhamer SF. Sunflower optimization algorithmbased optimal PI control for enhancing the performance of an autonomous
operation of a microgrid. Ain Shams Eng J 2021;12. https://doi.org/10.1016/j.
asej.2020.10.020.
[29] Ali S, Xia Y, Aurangzeb K, Khan ZA, Anwar MS. Design of optimized flatness based
active disturbance rejection controller for communication time delayed hybrid
microgrid. Ain Shams Eng J 2024;102664. https://doi.org/10.1016/j.
asej.2024.102664.
[30] Gupta DK, Soni AK, Jha AV, Mishra SK, Appasani B, Srinivasulu A, et al. Hybrid
gravitational-firefly algorithm-based load frequency control for hydrothermal twoarea system. Mathematics 2021;9. https://doi.org/10.3390/math9070712.
[31] Samy MM, Almamlook RE, Elkhouly HI, Barakat S. Decision-making and optimal
design of green energy system based on statistical methods and artificial neural
network approaches. Sustain Cities Soc 2022;84. https://doi.org/10.1016/j.
scs.2022.104015.
[32] Arwa EO, Folly KA. Reinforcement learning techniques for optimal power control
in grid-connected microgrids: a comprehensive review. IEEE Access 2020;8.
https://doi.org/10.1109/ACCESS.2020.3038735.
[33] Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, et al. Continuous control
with deep reinforcement learning. 4th Int. Conf. Learn. Represent. ICLR 2016 Conf. Track Proc., 2016.
[34] Khalid J, Ramli MAM, Khan MS, Hidayat T. Efficient load frequency control of
renewable integrated power system: a twin delayed ddpg-based deep
reinforcement learning approach. IEEE Access 2022;10:51561–74. https://doi.org/
10.1109/ACCESS.2022.3174625.
[35] Fan JC, Kobayashi T. A simple adaptive PI controller for linear systems with
constant disturbances. IEEE Trans Automat Contr 1998;43. https://doi.org/
10.1109/9.668848.
[36] Nasirian V, Davoudi A, Lewis FL, Guerrero JM. Distributed adaptive droop control
for DC distribution systems. IEEE Trans Energy Convers 2014;29. https://doi.org/
10.1109/TEC.2014.2350458.
[37] Muktiadji RF, Ramli MAM, Milyani AH. Twin-delayed deep deterministic policy
gradient algorithm to control a boost converter in a DC microgrid. Electronics
2024;13. https://doi.org/10.3390/electronics13020433.
Rifqi Firmansyah received the B.Eng. and M.Eng. in electrical
engineering from the Institut Teknologi Sepuluh Nopember,
Surabaya, Indonesia, in 2011 and 2013. He is currently pursuing the Ph.D. degree with the Department of Electrical and
Computer Engineering, King Abdulaziz University. He is also a
Lecturer with the Department of Electrical Engineering, Universitas Negeri Surabaya, Indonesia. His research interests
include microgrid optimization and control.
Makbul A.M. Ramli received the B.Eng. degree in electrical
engineering from the University of Tanjungpura, Indonesia, in
1995, the M.Eng. degree in electrical engineering from the
Bandung Institute of Technology (ITB), Indonesia, in 2000, and
the Dr.Eng. degree from the Nagaoka University of Technology
(NUT), Japan, in 2005. He is currently a Professor with the
Department of Electrical and Computer Engineering, King
Abdulaziz University (KAU). His research interests include
renewable and alternative energy, distributed generation, energy management systems, and smart grid.
19