Decentralized Coalition Formation of Multiple UAVs
in an Uncertain Region
'
'
'
Syed Arsalan Ali , Gao Xiaoguang , FU Xiaowei
I School of Electronics and Information
Northwestern Polytechnical University
Xi'an, China
arsaliengr@gmail.com, cxg20 12@nwpu.edu.cn, fxw@nwpu.edu.cn
Abstract- In cooperative missions,
not
have
sufficient
resources
to
resources out of which some may deplete by the use and
passage of time. The environment in which the UAVs are
operating is highly uncertain and no prior information is
available for the exact position and number of targets, no
centralized communication link with the UAVs is possible, and
also the number of UAVs used for this task may vary in
number due to the unpredictable nature of the environment.
The mission objective of all the UAVs taking part in this
search and attack task is to improve the overall operational
effectiveness of the mission by neutralizing maximum number
of targets in minimum time. If a single UAV after detecting a
target does not have sufficient resources to neutralize that
target completely, then a coalition out of the team of UAVs is
required to be formed which not only fulfills the target resource
requirement but can also help to achieve the global objective of
overall mission accomplishment in minimum time. The UAV
which detects a target becomes the coalition leader (CL) and
the other UAVs in the fmal coalition are the coalition
members. Therefore, there is a need of coalition formation
algorithms that are computationally less complex and can
achieve the overall mission objective requirement.
if an individual UA V does
neutralize
a target then
a
coalition of UA Vs may needs to be formed that fulfills the target
resource requirement. This paper proposes an algorithm for the
decentralized
coalition
formation
of
multiple
heterogeneous
UA Vs that cooperatively perform a search and attack task to
neutralize the static ground targets. The main objective of the
proposed algorithm is to determine the smallest coalition that
would successfully destroy the target in minimum time. First, the
responding UA Vs are sorted in the ascending order of their
resource difference based cost, and then the eligible sets of UAVs
with required total resources that can fulfill the target resource
criteria are determined.
the
algorithm
Second, from the eligible sets of UAVs,
determines
the
set
with
minimum
time
to
neutralize the target based on the time of arrival of UAVs on
target. Simulation tests to study the performance of proposed
approach are carried out and the results are compared with one
of the reference sub optimal decentralized coalition formation
approach. The results clearly depict that the proposed approach
is
more
effective
and
gives
near
optimal
solution
for
the
decentralized coalition formation.
Keywords-VA Vs;
decentralized
heterogeneous; resources
I.
coalition
formation;
This paper addresses the coalition formation or task
allocation problem of UAVs considering all the realistic
concerns of uncertain environment and heterogenetic UAVs.
The algorithm presented in this paper forms a coalition of
UAVs on the basis of two main objectives, which are (i) by
minimizing the coalition size, and (ii) neutralizing the target in
minimum time. Objective (i) is the first priority of the
algorithm as it allows more UAVs and resources to remain
available for the detection of other targets, thus reducing the
total search time of other UAVs and also enhance the
possibility of parallel coalition formation for neutralizing
multiple targets with the availability of more resources. In total
it can reduce the overall mission time and helps in achieving
the global mission objective. Objective (ii) selects the best
possible coalition which can neutralize the target in minimum
time out of multiple coalitions that satisfy the target's resource
requirement. This approach is contrary to the one presented in
[ 1]. In [ 1], Objective (i) has the second priority, while first
priority is to form a coalition satisfying target resource
requirement by adding resources of member UAVs from the
list which is based on earliest time of arrival (ETA) of member
UAVs at target location. The sub-optimal approach presented
in [ 1], sometime leads to the formation of coalition with more
number of UAVs, thus reducing the chances of search of other
INTRODUCTION
In recent times, the unmanned aerial vehicles (UAVs)
technology has shown its effectiveness in the limited and
targeted strikes military operations, and the flexibility in the
operational use of UAVs in such operations has raised the
demands for the use and acquires of UAVs by the top
air/military forces of the world to carry out various military
operations including border patrol, surveillance and
reconnaissance, target search and attack, battle damage
assessment, etc., and urged the combat planners and defense
systems designers to expand the role of UAVs for future
combat needs. As combat systems, more independent and
autonomous UAVs are envisioned which can execute their
actions especially in the form of cooperative teams to enhance
the overall performance of the mission in terms of mission
completion time and robustness.
One of the key military operations using mUltiple UAVs is
the target search and destroy mission in which a team of UAVs
cooperatively search and attack the targets in a highly uncertain
region. The UAVs in these missions are heterogeneous in
nature and carries different types and amounts of limited
978-1-4673-9613-4/16/$31.00 ©2016 IEEE
916
targets and parallel coalition formation to neutralize those
targets.
with use, while others like sensors do not deplete. Assume that
UAV Uj can carry n types of resources which can be
This paper is organized in the following sections as:
Related work is discussed in section II. Section III describes
the problem statement. The proposed coalition formation
algorithm is presented in section IV. Simulation results are
given in section V and section VI concludes this paper.
represented by a resource capability vector
II.
i
of the form:
( 1)
r - (I] ,. .,rp ) i=I, . . . , N
Where r/; , p = 1, . . . , n represents the number of type-p
resources held by UAV
For example, rU; = (2, 3) implies
Hi
RELATED WORK
that UAV
resources.
There have been lots of works, mostly in the area of
robotics and UAVs that study task allocation problems of
multiple autonomous agents. A centralized UAV task
assignment problem is addressed in [2], by developing an
optimal task assignment algorithm using Mixed Integer Linear
Programming (MILP) for a small sized problem. A strategic
routing problem for a fleet of UAVs to attack predetermined
targets is considered in [3]. The problem is modeled as a
MILP to which heuristic algorithm is proposed. In [4],
assuming global communication between UAVs a network
flow optimization model using linear programming method is
developed to allocate UAVs to targets. In [5], a multi-UAV
task assignment for predetermined targets is formulated as a
combinatorial optimization problem and a genetic algorithm to
solve the problem is proposed.
A decentralized control model of cooperative search problem
for a homogeneous UAV team and a path planning algorithm
based on a heuristic multi-objective cost function method is
presented in [6]. In [7], an algorithm for decentralized task
allocation is developed using a linear programming model
based on the information received from other agents.
These and many similar studies [8� 10] for task allocation of
multiple UAVs that consider some ideal conditions like UAVs
are homogeneous, not limited in resources, can prosecute the
targets with any resource, single task assignment to single
UAV, or have global communication. But these assumptions
become unrealistic when UAVs operate in uncertain
environments with many unknowns. Task assignment of
UAVs becomes a complex problem especially when UAVs
face the scenarios of unknown regions. In a real scenario,
communication between UAVs in the operating region can be
limited and only a set of UAVs can communicate with each
other, targets can be of various types and different types of
resources may be required to destroy them.
In [ 1], two decentralized coalition formation algorithms, the
first one is suboptimal polynomial time and the second is
optimal, for multiple heterogeneous UAVs operating in
uncertain environment are presented, assuming UAVs can
communicate with the coalition leader. In [ 1 1], a distributed
task allocation scheme based on resource welfare concept
from economics is proposed for multiple heterogeneous UAVs
with limited communications.
III.
rU
Uj
_
tt·
Hi
I
Uj
,
•
has two type-I resources and three type-2
Assume that there are M static targets whose positions and
the resource requirements are not known in advance as the
UAVs are operating in an uncertain environment with limited
sensor ranges. When a UAV Uj detects a target it can assess the
types and quantity of resources required to neutralize the target
Tj. If m types of resources are required by target Tj, then the
target resource requirement vector is
I]J,.
rTJ = (T
Where
.,rT)
qJ ,
j=I, . . . , M
(2)
r/J , q = 1, . . . , m represents the quantity of type-q
resources required to neutralize the target Tj. The UAVs in this
scenario have limited communication ranges and we assume
that the UAVs operating in a search region that are within the
immediate communication range of the detecting UAV can
communicate directly while other UAVs who are not in the
immediate communication range of detecting UAV can
communicate indirectly through a sequence of communication
links.
Once the UAV Uj detects a target Tj, and it does not have
sufficient resources to attack a target, it assumes the role of a
coalition leader and broadcasts the target resource requirement
and the other related information associated with the target to
form a coalition of UAVs which satisfies the target resource
requirement. The UAVs in the search region, that are within
the communication range of Uj, possessing at least one of the
required resources to destroy the target Tj will respond to Ui
with their cost and resource capabilities. A coalition c with
combined resources
as;
rC
�
rTJ , to destroy Tj can be represented
m =I, . . . , p
(3)
The global objective is to accomplish the overall mission in
minimum time. Assume that the UAV team of N agents takes
r time units to destroy all the targets, then the global objective
can be given as;
Global objective
This total time
PROBLEM STATEMENT
targets
Consider a team of N UAVs performing search and attack
task in an unknown region. The UAVs are heterogeneous and
can carry different types of resources in limited numbers. Some
of these resources are consumable, that is, the resources deplete
target
rs
r;J .
r
=
min r
(4)
depends on the time taken to search the
and the time taken to form coalitions and destroy the
Minimum member coalition allows more UAVs
remain available for the search of other targets thus reducing
917
the total search time
Ts
p,uq
Tj
and selection of coalition with
minimum time to destroy the target ultimately results in the
reduction of r:J .
IV.
When a VAV
Ui
ru·
Ui
B.
p
)
2
, for all p = 1, . . . , m
(5)
Coalition Formation Algorithm
Once the coalition leader receives all the responses from
the other VAVs within the time period allowed for the response
gathering, the coalition leader implements the following
coalition formation algorithm to form a finalized coalition to
neutralize the target.
Algorithm Steps:
1.
2.
list
e" ,
4.
u
q
q
from the sorted
to the eligible coalition
and add its resources
resource vector rC
3.
u
Pick the first VAV candidate
list RU , add the VAV
r
ruq
to the coalition
•
rC with
T
the target resource requirement vector r J by checking
C T,
the condition r > r
C
If r 2: r then stop otherwise continue the process of
Compare the updated coalition resource vector
T
}
including
the
e"
list RU in
next
VAV
U
q
from
the
sorted
, add its resources in coalition
resource rC , and repeat step 3.
For the first case when coalition leader has insufficient
resources, the coalition leader has to find the most suitable
coalition members on the basis of their cost to form a coalition.
For this a coalition fonnation algorithm is developed which
determines the smallest size coalition and selects the coalition
with minimum time to neutralize the target. The cost
calculation by the potential coalition members and the coalition
formation algorithm are described below.
Cost Calculation by Potential Coalition Members
Once a VAV Ui detects a target Tj and assumes the role of a
5.
The process of step 4 will continue until the target
resource constraint is met, and if the target resource
constraint is not met even after adding the resources of
all the VAVs in the sorted list of potential coalition
members, then the successful coalition formation is not
possible. If so, the coalition leader Ui will tenninate the
algorithm and rebroadcast the coalition formation
request.
6.
After step 4 condition
rC 2: rT.J is met, subtract one by
one the individual resources
coalition leader, it broadcasts the location and target resource
r
uq
of all the member
e" from the coalition resource set rC and
C rT, , then the VAV U E eu is removed
check if r
VAVs of
requirement rTJ to the other VAVs. On the basis of this
information the coalition leader itself and other VAVs
individually calculate their cost and ETA on target. The cost P
is calculated as the average sum of squared resource difference
between the target resource requirement vector
U including
q
J
detects a target Tj with a
rTj
Arrange the list RU of responding VAVs
coalition leader Ui in ascending order of their cost p;/q
resource
and '2: TJ , and if the detecting
VAV is not a part of any other coalition, then VAV Ui would
attack target Tj without requesting a coalition with other VAVs.
In this case VAV Ui will form a single member coalition and
broadcast this single member coalition infonnation along with
the necessary target information to other VAVs as information
update only.
A.
p
p=i
r Tj
the potential coalition members.
Ui
assumes the
role of coalition leader and requests for coalition fonnation by
broadcasting the information about the target Tj. The
information broadcasted by the VAV Ui includes the location
and required resources of target Tj, to the other VAVs. This is
the case when the detecting VAV has insufficient resources to
neutralize the target. The other VAVs who receive this
information from the VAV Ui and possess at least one of the
required resources to destroy the target Tj will respond to Ui
with their cost, resource capabilities and ETA on target. These
responding VAVs can be called as potential coalition members.
All these responses from the possible member VAVs are
considered by the coalition leader, which then determines
whether a coalition can be formed. If a coalition cannot be
formed, then the coalition leader sends a "discard coalition"
broadcast and after sometime, it will rebroadcast the coalition
formation request. Otherwise, the coalition is formed by the
coalition leader and the coalition information is broadcasted.
The responding VAVs who are not part of the final coalition
will continue their search task, and the selected members will
re-plan as per their strike mechanism to reach and destroy the
target.
j
requirement rT ,
m
_
P;/q and
J
ETA on target is communicated back to the coalition leader by
detects a target Tj with a resource
In the other case, when VAV
m
The information including resource based cost
COALITION FORMATION
j
requirement rT , and rTJ 2: rU; , then the VAV
( ruq
= _1 �
�
>
from the list
from
and
potential member VAV resource vector rUq , i.e.,
r C.
e" and its resources are also subtracted
This is to ensure the minimum members set
satisfying the
918
q
rC 2: rTJ
condition.
7.
total
8.
ETAs on target of the coalition leader and responding
Count the number of VAVs e in the [mal e" and
sort all the possible e member combinations out of the
n
VAVs as given in [ 1] are
member VAVs of the sorted list RU .
D;4I
172,
D;5I
207,
=
D;6I
=
123,
D;2I
=
=
63,
96. In the simulation, the initial
=
requirement r
After collecting the cost and ETA information from all the
potential coalition members, coalition leader Uj implements the
proposed coalition formation algorithm.
J
If more than one e member qualified combinations
are provided by step 8, calculate the simple sum of
each resource set of the available qualified
combinations. Select the minimum value resource sum
and keep only those combinations whose simple
resource sum is equal to the minimum value and
discard other combinations, e.g., if the resources of the
three valid two member combinations from step 8 are
(5,3), (2,4), and (4,2), then the simple resource sum
will be 8, 6, and 6 respectively for the three
combinations. The lowest sum value 6 is selected,
therefore the combinations with resources (2, 4) and (4,
2) will be selected and the other one is discarded.
then select the one with minimum ETA
As per the algorithm, the sorted list of VAVs in ascending
order of their cost is RU
{5, 1, 2, 4, 3, 6} . Following the
algorithm steps 2 to 5, the eligible coalition list which fulfills
=
the target resource requirement is e"
resource sum
V.
c
r
=
{5, 1} with coalition
{5, 3} . Now, as per step 6, the individual
=
resources of the coalition members in e" are subtracted one by
one from
Uq E
c
r
to check that the resources of member VAVs
e" are required for the coalition or not. The first VAV
in ell is VAV-5, therefore first we remove its resources from
c
r
D;' .
J
c
r
that results in
=
r.
c
r 2: r }
(2, 1). The condition
case with the next member VAV of e" . Hence the ell
remains same and no VAV is removed from e" . From step 7
and 8, only the possible two member combinations (as e =2)
SIMULATION RESULTS
out of
In this section, the performance of the proposed coalition
formation algorithm for VAVs performing a search and attack
mission is studied via simulations. Initially, an example along
with the simulation results is presented to show how the
coalition formation is carried out and then on the basis of
simulation results, performance of the proposed algorithm is
discussed.
n
member from the sorted list RU satisfying the
c
Tf
condition r 2: r are kept. In this case, only single two
member combination i.e. {5, 1} qualifies from step 8. Here the
output of step 8 is a single coalition set, therefore output of step
9 and step 10 will be the same coalition set. The final coalition
is {5, I} with resources (5, 3), as shown in Fig. 2.
I
For illustration and comparison of the proposed Algorithm,
we consider the same example case as given in [ 1]. The
example scenario is with six VAVs with constant speeds and a
static target T,. Consider a VAV Uj detects a target T, as shown
in Fig. 1. The target resource requirement is rTi
(5, 3), with
r2Ti
3, and since
=
Uj
�/
'..._...'
does not have enough
Resources
2,1
1,3
1,1
2,0
3,2
0,2
I
21
,
' 6, '
LIST OF UAVS WITH THEIR RESOURCES AND COST
I
I -�\
Cost
6.5
8
10
9
2.5
13
(�;\
'
, -,'
t
'...._ .."
.....
/',. \
r
,
6,51
'....... '
X-Distance
Fig. I. UAV
91 9
u/
Target
UAV
--T-75.61
'. _ 4,
\..... "
,
,
... .. ... ...
,
,
,
'... _ ..."
"A \
/�'T1
resources, it broadcasts the proposal for coalition. The
resources of U I and responding VAVs along with the cost
calculated using (5), are given in Table l.
UAV
r',
,�.
=
TABLE I.
!
I
A. Example
5 and
is not
met, so the VAV-5 cannot be removed from e" . Similar is the
Minimum ETA for the combination with more than
one member is determined by the combination member
with the highest ETA.
=
D;3I
positions of the VAVs are generated randomly, therefore ETA
on target of VAVs are different in our case than the ETAs
given in [ 1].
10. If more than one combination is available from step 9,
ljTi
47,
Out of the total sorted e member combinations, keep
only those combinations whose resource sum are
greater
than or equal to target resource
T.
9.
=
D;II
detects target TI
,
I
!.
..... .....�,..
i
,'..Ss,
....._..'.
I
�1
I
,
-1
' 'l
... _ ...
..... ...
l�t,
I
_
\,
'"
,-�
,
,
,--
4
,1
I
overall search time
,
(b
... -,�
' ""
VI.
�'
5J
...... _-,-"
X-Distance
Fig. 2. A coalition ofu] and
B.
U5
is formed to destroy target TJ
Performance of Algorithm
The above sequence of events shows how the coalition
leader determines best possible coalition using the proposed
coalition formation algorithm to destroy the detected target.
The performance of the proposed coalition formation algorithm
is evaluated by a comparison study with the algorithm of the
same domain. Table 2 presents the performance comparison of
the proposed coalition formation algorithm against the sub­
optimal coalition formation algorithm presented in [ 1].
TABLE II.
Total Available UAVs
Final Coalition
Coalition Resources
UAVs in Coalition
Remaining UAVs
.
CONCLUSION
ACKNOWLEDGMENT
This work was supported by the National Natural Science
Foundation of China under Grant NO.6 1573285.
REFERENCES
PERFORMANCE COMPARISON
Proposed Algorithm
6
{5, l}
(5,3)
2
4
of targets to achieve the global
A decentralized coalition formation algorithm for multiple
heterogeneous UAVs operating in an uncertain region to
neutralize static targets is proposed in this paper. The proposed
algorithm forms a minimum size coalition, which allows more
UAVs and resources to remain available for the search of other
targets, thus reducing the overall search time of targets for
other UAVs and selects the coalition with lowest time to
prosecute the target, thus reducing the target prosecution time.
The simulation results of the proposed algorithm for the sample
scenario and performance comparison with one of the sub­
optimal coalition formation algorithm clearly show that the
proposed algorithm achieves the global objective of
minimizing the overall mission time by minimum size and
minimum time coalition formation, and gives near optimal
solution. For future work, we aim to extend this decentralized
coalition formation approach to neutralize the dynamic targets
along with the development of multiple UAVs target strike
mechanism for both static and dynamic targets.
"'-
�
r5
objective of minimizing the overall mission time r
I
,
""
'b.'21
'
I
\
,
Target
UAV
Algorithm in [II
6
{3,6,I,4}
(5,4)
4
2
For the sample scenario, the resulted [mal coalition by the
proposed algorithm is a two member coalition {5, I} with
resources (5, 3). Thus out of six UAVs, four UAVs remain
available for the search of other targets.
While the final coalition provided by the algorithm in [ 1]
for the same example is a four member coalition {3, 6, 1, 4}
with coalition resources (5, 4). Thus out of six UAVs, only
two remains available for the search of other targets. This
shows that the proposed algorithm finds the minimum size
coalition more effectively and allows more UAVs and
resources to remain available for the search task.
As per the ETA of UAVs provided for the sample scenario
in [I], the ETA of the [mal coalition by the algorithm in [ 1] is
172 sec. In the case of proposed algorithm, the initial positions
of UAVs are generated randomly in simulation for the sample
scenario; therefore ETA of individual UAVs and the ETA of
final coalition are different from [ 1] as per the location of
member UAVs. Even if we consider the same ETAs of the
UAVs as in [ 1], then the ETA of the final coalition by the
proposed algorithm is 205 sec, which is more than the ETA of
final coalition in [ 1]. The proposed algorithm overcomes this
by forming minimum member coalition and reducing the
[I]
J. Manathara, P. Sujit, and R. Beard, "Multiple UAV coalition for a
search and prosecute mission," 1. Intell. Robot. Syst. Vol. 62(1), pp.
125-158,2011.
[2]
C. Shumacher, P. Chandler, M. Pachter, and L. Pachter, "UAV task
assignment with timing constraints via mixed-integer linear
programming,"
Proceedings of AlAA 3rd Unmanned Unlimited
Systems Technical Conference,Workshop and Exhibit,Chicago, 2004.
[3]
V. Shelty, M. SUdit, and R. Nagi, "Priority-based assignment and
routing of fleet of unmanned combat aerial vehicles," Comput. Oper.
Res. Vol. 35,pp. 1813-1828,June 2008.
[4]
K.E. Nygard, P.R. Chandler, and M. Pachter, "Dynamic network flow
optimization models for air vehicle resource allocation," In: Proc. of the
American Control Conference,Texas,vol. 3,2001,pp. 1853-1858.
[5]
T. Shima, S. Rasmussen, A. Sparks, and K. Passino, "Multiple task
assignments for cooperating uninhabited aerial vehicles using genetic
algorithms," Comput. Oper. Res. Vol. 33,pp. 3252-3269, 2006.
[6]
Y. Yang, A. Minai, and M. Polycarpou, "Decentralized cooperative
search by networked UAVs in an uncertain environment," In: Proc. of
the American Control Conference,Boston,2004,pp. 5558-5563.
[7]
D. Dionne, and C.A. Rabbath, "Multi-uav decentralized task allocation
with intermittent communications: the dtc algorithm," In: Proc. of the
American Control Conference ,New York,2007,pp. 5406-5411.
[8]
P. Sujit, and R. Beard, "Distributed sequential auctions for multiple
UAV task allocation," In: Proc. of the American Control Conference,
New York City,2007.
[9]
P. Gurfil, " Evaluating UAV flock mission performance using Dudek's
taxonomy," In: Proc of the American Control Conference, Portland,
2005,pp. 4679 - 4684 .
[10] P. Sujit,G. George, and R. Beard, "Multiple UAV coalition formation,"
In: Proc. of the American Control Conference, Seattle, Washington,
USA, 2008,pp. 2010-2015.
[II] Min-Hyuk Kim, Hyeoncheol Baik, and Seokcheon Lee, "Resource
Welfare Based Task Allocation for UAV Team with Resource
Constraints," 1. Intel I. Robot. Syst. Vol. 77, pp. 611-627, 2015.
920