Lecture III:
Collective Behavior of Multi
-Agent Systems: Analysis
Zhixin Liu
Complex Systems Research Center,
Academy of Mathematics and Systems Sciences, CAS
In the last lecture, we talked about
Complex Networks


Introduction
Network topology
Average path length
Clustering coefficient
Degree distribution

Some basic models





Regular graphs: complete graph, ring graph
Random graphs: ER model
Small-world networks: WS model, NW model
Scale free networks: BA model
Concluding remarks
Lecture III:
Collective Behavior of Multi
-Agent Systems: Analysis
Zhixin Liu
Complex Systems Research Center,
Academy of Mathematics and Systems Sciences, CAS
Outline




Introduction
Model
Theoretical analysis
Concluding remarks
What Is The Agent?
From Jing Han’s PPT
What Is The Agent?


Agent: system with two important capabilities:

Autonomy: capable of autonomous action – of
deciding for themselves what they need to do in order
to satisfy their objectives；

Interactions: capable of interacting with other agents
- the kind of social activity that we all engage in every
day of our lives: cooperation, competition, negotiation,
and the like.
Examples: Individual, insect, bird, fish, people,
robot, …
From Jing Han’s PPT
Multi-Agent System (MAS)

MAS



Many agents
Local interactions between agents
Collective behavior in the population level



More is different.---Philp Anderson, 1972
e.g., Phase transition, coordination, synchronization, consensus,
clustering, aggregation, ……
Examples:





Physical systems
Biological systems
Social and economic systems
Engineering systems
……
Biological Systems
Flocking of Birds
Bee Colony
Ant Colony
Bacteria Colony
Engineering Systems
From Local Rules to Collective Behavior
Phase transition, coordination, synchronization,
consensus, clustering, aggregation, ……
pattern
scale-free,
small-world
A basic problem: How locally interacting
agents lead to the collective behavior of the
overall systems?
swarm intelligence
Crowd Panic
Outline




Introduction
Model
Theoretical analysis
Concluding remarks
Modeling of MAS




Distributed/Autonomous
Local interactions/rules
Neighbors may be dynamic
May have no physical connections
This lecture will mainly discuss
A Basic Model
Assumption
Each agent
• makes decision according to local
information ;
• has the tendency to behave as other
agents do in its neighborhood.
Vicsek Model
(T. Vicsek et al. , PRL, 1995)
http://angel.elte.hu/~vicsek/
r
A bird’s Neighborhood
Alignment: steer towards the
average heading of neighbors
Motivation:
 to investigate properties in nonequilibrium systems
 A simplified Boid model for flocking behavior.
Notations
xi(t) : position of agent i in the plane at time t
i (t ) : heading of agent i, i= 1,…,n. t=1,2, ……
v: moving speed of each agent
r: neighborhood radius of each agent
Neighbors:
N i (t )  { j : xi (t )  x j (t )  r}
r
Vicsek Model
Neighbors:
N i (t )  { j : xi (t )  x j (t )  r}
Position:
xi (t  1)  xi (t )  v(cos  i (t  1), sin  i (t  1))
Heading:

sin  (t ) 

j 
 j  N (t )


i
i (t  1)  arctan 

cos

(
t
)


j 
 j  Ni (t )

Vicsek Model
Neighbors:
N i (t )  { j : xi (t )  x j (t )  r}
Position:
xi (t  1)  xi (t )  v(cos  i (t  1), sin  i (t  1))
Heading:
tan i (t  1) 

jN i ( t )
cos  j (t )
 cos (t )
jN i ( t )
j
tan i (t )
Vicsek Model
Neighbors:
N i (t )  { j : xi (t )  x j (t )  r}
Position:
xi (t  1)  xi (t )  v(cos  i (t~  1), sin  i (t  1))
P (t )  { ~
pij (t )},
Heading:
cos  j (t )


~
pij (t )    jN (t ) cos  j (t )
i

0

~
tan (t  1)  P (t ) tan (t ),
~
P (t ) is the weighted average matrix.
if i ~ j
otherwise
Vicsek Model
http://angel.elte.hu/~vicsek/
Some Phenomena Observed
(Vicsek, et al. Physical Review Letters, 1995)
n = 300
v = 0.03
r=1
Random initial
conditions
a) ρ= 6, ε= 1 high density, large noise
c)
b) ρ= 0.48, ε= 0.05 small density, small noise
d) ρ= 12, ε= 0.05 higher density, small noise
Synchronization

Def. 1: We say that a MAS reach synchronization
if there exists θ, such that the following equations
hold for all i,

Question: Under what conditions, the whole
system can reach synchronization?
Outline




Introduction
Model
Theoretical analysis
Concluding remarks
Interaction and Evolution
 (0)
 (1)
G(0)
 (2)
G(1)
……
G(2)
 (t-1)
 (t)
G(t-1)
……
……
x (0)
x (1)
x (2)
……
x (t-1)
x (t)
• Positions and headings are strongly coupled
• Neighbor graphs may change with time
Some Basic Concepts
Adjacency matrix:
Degree:
1 If i ~ j
A  {aij}, aij  
0 Otherwise
d i , i  1,, n, d max  max d i , d min  min d i
i
n
Volume:
Vol (G)   d j
j 1
Degree matrix: T  diag (d1, d2 ,, d n )
Average matrix: P  T 1 A
Laplacian:
L T  A
i
Connectivity of The Graph
Connectivity:
There is a path between any two vertices of the
graph.
Joint Connectivity of Graphs
G1
G2
G 1 ∪G 2
Joint Connectivity:
The union of {G1,G2,……,Gm} is a connected graph.
Product of Stochastic Matrices
Stochastic matrix A=[aij]: If ∑j aij=1; and aij≥0
SIA (Stochastic, Indecomposable, Aperiodic) matrix A
t

lim
A

1
c
, where 1n  [1, 1] .
If t 
n
Theorem 1: (J. Wolfowitz, 1963)
Let A={A1,A2,…,Am}, if for each sequence Ai1, Ai2, …Aik of
positive length, the matrix product Aik Ai(k-1) … Ai1 is SIA.
Then there exists a vector c, such that
lim Aik  Ai 2 Ai1  1n c .
k 
The Linearized Vicsek Model
1

i (t  1) 
 j (t ),


N i (t ) jNi (t )

 x (t  1)  x (t )  v(cos  (t  1), sin  (t  1))  .
i
i
i
 i
P (t )  { pij (t )},
 1
if i ~ j

pij (t )   | N i (t ) |
 0
otherwise
(t  1)  P(t )(t ),



x
(
t

1
)

x
(
t
)

v
(cos

(
t

1
),
sin

(
t

1
))
.

A. Jadbabaie , J. Lin, and S. Morse, IEEE Trans. Auto. Control, 2003.
Theorem 2 (Jadbabaie et al. , 2003)
Joint connectivity of the neighbor graphs on
each time interval [th, (t+1)h] with h >0
Synchronization of the linearized
Vicsek model
Related result: J.N.Tsitsiklis, et al., IEEE TAC, 1984
The Vicsek Model
Theorem 3:
If the initial headings belong to (-/2, /2),
and the neighbor graphs are connected,
then the system will synchronize.
 Liu and Guo (2006CCC), Hendrickx and Blondel (2006).
 The constraint on the initial heading can not be removed.
Example 1:
n  12, r  0.8, 0  v  0.1,
 x1 (0)  (1,0), 1 (0)  0;

 x2 (0)  ( 3 2 ,1 2),  2 (0)  5 6 ;

 x3 (0)  (1 2 , 3 2),  3 (0)  5 3;
 x4 (0)  (0,1),  4 (0)   2

 x5 (0)  ( 1 2 , 3 2),  5 (0)  4 3;

 x6 (0)  ( 3 2 ,1 2),  6 (0)   6 ;

 x7 (0)  (1,0),  7 (0)  
 x (0)  ( 3 2 , 1 2),  (0)  11 6 ;
8
 8
 x9 (0)  ( 1 2 , 3 2),  9 (0)  2 3;

 x10 (0)  (0,1), 10 (0)  3 2

 x11 (0)  (1 2 , 3 2), 11 (0)   3;
 x (0)  ( 3 2 , 1 2),  (0)  7 6
1
 12
Connected all the time, but synchronization does
not happen.
• Differences between with VM and LVM.
•
Example2:
n  24, r  0.3, v  0.1,
 x1 (0)  (1,0); 1 (0)  0;
 x (0)  (0.966,0.259);  (0)  13 12 ;
1
 2
 x (0)  ( 3 2,1 2);  (0)   6 ;
3
 3
 x4 (0)  ( 2 2, 2 2);  4 (0)  5 4 ;

 x5 (0)  (1 2 , 3 2);  5 (0)   3 ;
 x (0)  (0.259,0.966);  (0)  17 12 ;
6
 6
 x7 (0)  (0,1);  7 (0)   2 ;

 x8 (0)  (0.259,0.966); 1 (0)  19 12 ;
 x (0)  (1 2 , 3 2);  (0)  2 3 ;
9
 9
 x (0)  ( 2 2, 2 2);  (0)  7 4 ;
10
 10
 x11 (0)  ( 3 2,1 2); 1 (0)  5 6 ;

 x12 (0)  (0.966,0.259); 1 (0)  23 12 ;

 x13 (0)  (1,0); 13 (0)   ;
 x14 (0)  (0.966,0.259); 14 (0)   12 ;

 x15 (0)  ( 3 2,1 2); 15 (0)  7 6 ;

 x16 (0)  ( 2 2,  2 2); 16 (0)   4 ;

 x17 (0)  (1 2 ,  3 2); 17 (0)  5 3 ;
 x18 (0)  (0.259,0.966); 18 (0)  5 12 ;

 x19 (0)  (1,0); 19 (0)  3 2 ;
 x (0)  (0.259,0.966);  (0)  7 12 ;
20
 20
 x21 (0)  (1 2 ,  3 2);  21 (0)  5 3 ;

 x22 (0)  ( 2 2,  2 2);  22 (0)  3 4 ;

 x23 (0)  ( 3 2,1 2);  23 (0)  11 6 ;
 x (0)  (0.966,0.259);  (0)  11 12 ;
24
 24
The neighbor graph does not converge
May not likely to happen for LVM
 How
to guarantee connectivity?
 What
kind of conditions on
model parameters are needed ?
Random Framework
Random initial states:
1) The initial positions of all agents are uniformly
and independently distributed in the unit square;
2) The initial headings of all agents are uniformly
and independently distributed in [-+ε, -ε] with ε∈
(0, ).
Random Graph
G(n,p): all graphs with vertex set V={1,…,n} in which
the edges are chosen uniformly and independently
with probability p.
Theorem 5
P (G p is connected )  e
Corollary:
, then
Let
e c
c 
Not applicable to neighbor graph !
P.Erdős,and A. Rényi (1959)
Random Geometric Graph
Geometric graph G(V,E) :
V  {1,2,  , n},
E  {( i, j ) : xi  x j  r , i, j  V }
Random geometric graph:
If {xi ,1  i  n} are i.i.d. in unit cube
uniformly, then geometric graph G(V , rn ) is
called a random geometric graph
*M.Penrose, Random Geometric Graphs, Oxford University Press,2003.
Connectivity of Random Geometric Graph
Theorem 6
Graph G (n, r (n))
with
log( n)  c(n)
r ( n) 
n
is connected with
probability one as n   if and only if c(n)  .
G (n, r (n))
( P.Gupta, P.R.Kumar,1998 )
Analysis of Vicsek Model




How to deal with changing neighbor graphs ?
How to estimate the rate of the
synchronization?
How to deal with matrices with increasing
dimension?
How to deal with the nonlinearity of the model?
Dealing With Graphs With
Changing Neighbors
1) Projection onto the subspace spanned by 1n  [1,1] .
2) Stability analysis of TV systems (Guo, 1994)
3) Estimation of the number of agents in a ring
(1   )r
r
(1   )r
Ci  { j : (1  )r  x  xi  (1  )r}
Estimating the Rate of
Synchronization
The rate of synchronization depends on the spectral gap.
Normalized Laplacian: T 1/ 2 LT 1/ 2
Spectrum :
0  0  1    n1
Spectral gap:
  max( 1  1 , 1  n1 )
Rayleigh quotient
1  inf
z T 1n
2
(
z

z
)
i~ j i j
2
z
 jV j d j
n 1  sup
z
2
(
z

z
)
i ~ j i j
2
z
 jV j d j
The Upper Bound of
n1 (0)
Lemma1: Let edges of all triangles be “extracted” from a complete
graph. Then there exists an algorithm such that the number of residual
edges at each vertex is no more than three.
Example:
=
+
Lemma 2: For large n, we have


1
n1 (0)  21 
(1  o(1)) 
2
 4(1  2 3 )

( G.G.Tang, L.Guo, JSSC, 2007 )
The Lower Bound of 1 (0)
Lemma 3: For an undirected graph G, suppose there exists
a path set P joining all pairs of vertexes such that each path
in P has a length at most l, and each edge of G is contained in
at most m paths in P. Then we have
2
1  nd min / d max
ml
Lemma 4: For random geometric graphs with large n ,
d max  nrn2 (1  o(1)),
d min
nrn2

(1  o(1)).
4
( G.G.Tang, L.Guo, 2007 )
The Lower Bound of 1 (0)
1 (0) 
 r2
512(r  6 )
4
(1  o(1))
( G.G.Tang, L.Guo, 2007 )
Estimating The Spectral Gap of G(0)
Proposition 1: For G(n,r(n)) with large n

n1 (0)  21 

1 (0) 

1

(
1

o
(
1
))
2

4(1  2 3 )

 r2
512(r  6 )
 (0)  1 
4
(1  o(1))
 r2
512(r  6 )
4
(1  o(1))
( G.G.Tang, L.Guo, 2007 )
Analysis of Matrices with Increasing Dimension
Estimation of multi-array martingales
m
max max
1 m n 1 k  n
where
f
j 1
j
(k , n) w j 1
n
S n  max  f j2 (k , n),
1 k  n
Moreover, if
j 1
max max
f
j 1
j


Cw  sup E w2j 1 Fj (k , n) .
S n 4 Cw1 log n ,
m
1 m n 1 k  n
3Cw

S n  3 log n, a.s.
4
1 k , j  n
then we have
(k , n) w j 1  3 Cw S n log n
a.s.
Analysis of Matrices with Increasing Dimension
Using the above corollary, we have for large n
1) max
1i  n


2
sin

(
0
)

O
nr
a.s.;
 j
n log n ,
jN i ( 0 )


sin  

2
2) max   cos  j (0) 
  O nrn log n , a.s.;
1i  n

jN i ( 0 ) 


3) max tan i (1)  O log n nrn2 , a.s.;
1i  n


4) max cos i (1)  1  O log n nrn2 , a.s.
1i  n
Dealing With Inherent Nonlinearity
A key Lemma: There exists a positive constantη,
such that for large n, we have :
1) d ij (t )  d ij (0)  r (1  o(1)),
2)  (0) 
 1
with
min
  sup P ( s )  P( s )   
~
 1 s t
  / 64 ,

r 2 / 4 r 2
3  512(r  6 )
4


(1  o(1)).

min 64 /  , r 2 / 4 r 2
 64 4 
  max  , 2 ,  
3  512(r  6 ) 4
  r 
Theorem 7
High Density Implies Synchronization
For any given system parameters v  0
and r  0, when the number of agnets n
is large, the Vicsek model will synchronize
almost surely.
This theorem is consistent with the simulation result.
Theorem 8
High density with short distance interaction
16
Let
 log n 
rn  o(1), 

 n 
satisfy
 o( rn ),
and the velocity
 r6 n 
vn  O n 3 / 2 .
 log n  


Then for large population, the MAS will
synchronize almost surely.
Concluding Remarks



In this lecture, we presented the synchronization
analysis of the Vicsek model and the related models
under deterministic framework and stochastic
framework.
The synchronization of three dimensional Vicsek
model can be derived.
There are a lot of problems deserved to be further
investigated.
1. Deeper understanding of self-organization,




What is the critical population size for
synchronization with given radius and
velocity ?
Under random framework, dealing with the
noise effect is a challenging work.
How to interpret the phase transition of the
model?
……
2. The Rule of Global Information
If some sort of global interactions are
exist for the agents, will that be helpful?
Random connections
are allowed
Edges formed by
the neighborhood
3. Other MAS beyond the Vicsek Model
Nearest Neighbor Model
G (n,  (n))
Each node is connected
with the nearest  (n)
neighbors
Remark:
For G (n,  (n)) to be asymptotically connected, (log n)neighbors are
F.Xue, P.R.Kumar, 2004
necessary and sufficient.
Boid Model: Craig Reynolds(1987):
http://www.red3d.com/cwr/boids/applet
A bird’s Neighborhood
Separation: steer to avoid
crowding neighbors
Alignment: steer towards the
average heading of neighbors
Cohesion: steer to move toward the
average position of neighbors
In the next lecture, we will talk about
 Collective
Behavior of Multi-Agent
Systems: Intervention
 References:

J. Han, M. Li M, L. Guo, Soft control on collective behavior of a group of autonomous
agents by a shill agent, J. Systems Science and Complexity, vol.19, no.1, 54-62, 2006.

Z.X. Liu, How many leaders are required for consensus? Proc. the 27th Chinese
Control Conference, pp. 2-566-2-570, 2008.
Thank you!