From Local Network Motifs
to Global Invariants
Victor M. Preciado and Ali Jadbabaie
Department of Electrical and Systems Engineering
University of Pennsylvania
Third Year Review, October 29, 2010
ONR MURI: NexGeNetSci
Preciado
Local Motifs and
Global Invariants
Theory
• First principles
• Rigorous math
• Algorithms
• Proofs
Data
Analysis
• Correct
statistics
• Only as good
as underlying
data
Lab
Numerical
Experiments Experiments
• Simulation
• Synthetic,
clean data
• Stylized
• Controlled
• Clean,
real-world
data
Field
Exercises
Real-World
Operations
• SemiControlled
• Messy,
real-world
data
• Unpredictable
• After action
reports in lieu
of data
Outline
•
•
•
•
•
Motivation and context
The role of local structural information
Spectral analysis from local structural information
Bounds on spectral properties via optimization
Implications in dynamical processes
ONR MURI: NexGeNetSci
Complex Network: Properties
• Generic features:
– Large number of nodes
– Sparse connectivity
– Lack of regularity
• Examples:
– Comm networks (e.g. Internet)
– Social networks (e.g. Facebook)
– Biological networks
• We assume limited structural
information:
??
??
– Privacy and/or security concerns
– Storage/computing limitations
ONR MURI: NexGeNetSci
Complex Networks: Some Challenges
• Challenges when only local structural information is available:
– Estimation: How could we aggregate local measurements to
infer global properties of the network?
– Inference: What could we say about the behavior of a dynamical
process in the network from local measurements?
– Actuation: How could we modify the structure of a network to
induce a desired global behavior?
Actuation
Estimation
Inference
Usual Approach in “Network Science”
• Overly focused on random graph models and degree distributions, but
we can have very different networks with the same degree distribution
[Li et al., 2005]:
• Main drawbacks:
1. Degree distributions are a zero-th order approximation of the
network structure, by far not enough
2. Random models are difficult, if not impossible, to justify from an
engineering perspective
ONR MURI: NexGeNetSci
More Structured Random Models
•
We also find random graph models capturing increasingly richer structural
properties [Mahadevan et al., 2006]
Average degree
Degree distribution
Distribution of Triangles
•
Joint Degree Distribution
Original HOT model
Main drawbacks:
– Visual inspection is clearly not enough to measure similarity
– What structural measurements are relevant in the behavior of dynamical
processes in networks?
ONR MURI: NexGeNetSci
Networks = Graphs + Dynamics
• Our framework: We consider the dynamical behavior of networks
• Since the eigenvalues and eigenvectors are closely related with the
network dynamical behavior, spectral graph theory is a convenient
framework to study network dynamics
800
0
500
600
1000
400
1500
r(A)
200
2000
0
500
1000
1500
nz = 45572
2000
0
-40
-20
0
20
40
60
• Some relationships between spectra and dynamics are:
– Spreading processes
– Synchronization
– Diffusion/Consensus
 adjacency spectral radius
 (combinatorial) Laplacian eigenratio
 (normalized) Laplacian eigenvalues
ONR MURI: NexGeNetSci
80
Inference from Local Measurements
 Our problems:
1. - What measurements are most relevant in the behavior of
dynamical processes?
2. - How can we aggregate local measurements to say something
about the global dynamical behavior?
Aggregation of
local structural
measurements
•
Dynamical
Implications
We study those problems in the framework of spectral graph theory
and convex optimization, without making any assumption on the
global network structure (i.e., no random models)
ONR MURI: NexGeNetSci
Structure of our Approach
Use algebraic graph theory to relate the frequency of
certain small subgraphs, or motifs, with the so-called
spectral moments of the network
II. Propose a distributed technique to compute the
frequency of subgraphs from the distribution of local
network measurements
III. Use convex optimization to extract relevant spectral
information from a sequence of spectral moments
IV. Study implications on dynamical processes
I.
ONR MURI: NexGeNetSci
I. From Subgraphs to Spectral Moments
• Algebraic graph theory allows us to compute spectral moments
from local structural information. We use the following result:
1 n k 1
mk ( A)   i  # closed walks of length k 
n i 1
n
• Low-order moments: For k≤3 we have the following expressions
k=2
k=3
i
i
m2 
1
2e
d

 i n
n i
m3 
1
6
2
t

 i n
n i
ONR MURI: NexGeNetSci
From Subgraphs to Moments
• Higher-order moments: As we increase the order of the moments, a
variety of more and more complicated subgraphs come into the
picture. For k=4, we have the following types of closed walks:
i
i
i
i
Moments from
local structural
measurements
j
m4 
1
2qi

n i
m4 
1
( 8Q
n

(d i  1)d i

 (d i  1)d i
4
 di

8e

Moments from
subgraphs
frequencies
• In the first expression, we observe that local measurements can be
aggregated via distributed consensus to compute spectral moments
ONR MURI: NexGeNetSci
Moments from Subgraphs Frequencies
• Key observation: The spectral moments are linear combinations of
subgraphs embedding frequencies [Preciado, Jadbabaie, 2010].
The coefficients for all nonisomorphic connected subgraphs with 4
or less nodes are
mk
k=4
2
4
-
-
-
-
8
-
-
k=5
-
-
30
-
-
10
-
-
-
k=6
2
12
24
12
6
-
48
36
-
k=7
-
-
126
-
-
84
-
112
-
k=8
2
28
168
72
32
64
264
464
528
• For example, the fifth moment can be computed as:
1
m5  ( 30  10   30 e 
n
ONR MURI: NexGeNetSci
II. Distributed Computation of Moments
• We propose a distributed technique to compute subgraph
frequencies. Note that each subgraph can be ’discovered’ by a
number of its nodes. For example, for 1-hop neighborhoods:

 1
 i i
n n
3
 1
1
 i  i  i  i (d i  2)
n n
n
Q 1
  ???
n n i
• In general, if each node have access to its r-hops neighborhood, we
can discover all subgraphs involved in moments of order up to 2r+1
(and part of the subgraphs involved in moments of higher order)
ONR MURI: NexGeNetSci
Empirical Example
• Subgraph with 2,404 nodes and 22,786 edges obtained from crawling
the Facebook graph in a breadth-first search around a particular node
Node Degree Distribution
-1
Distribution of Triangles
-1
10
4
Degree-Triangle Scatter Plot
10
10
3
10
-2
-2
10
t,Triangles
P(t)
P(k)
10
-3
-3
2
10
10
10
1
10
-4
-4
10
0
10
1
2
10
10
k,Degree
3
10
10
0
0
10
1
10
2
10
t,Triangles
3
10
• We can compute the relevant quantities
which allow us to compute moments
ONR MURI: NexGeNetSci
4
10
10
0
10
1
2
10
10
k,Degree
3
10
III. Extracting Spectral Info from Moments
• So far, we have
Counting
subgraph
frequencies
Computing
spectral
moments
?
• We now present an SDP-based approach to extract information from
the spectral moments that
1. It is agnostic, in the sense that it does not make any assumption
on the global network structure (no random model)
2. It allows to study the effect of arbitrarily complicated structural
measurements in the network spectral properties
ONR MURI: NexGeNetSci
The Classical Moment Problem
• How can we extract information from spectral moments? The
following problem, called the classical moment problem, is
closely related to ours:
Given a sequence of moments (m1,…,mk), and Borel
measurable sets T  W  R, we are interested in computing
where m in M(W), M(W) being the set of positive Borel
measures supported by W.
• Generalization of Markov and Chebyshev´s inequalities from
probability theory, when a sequence of moments is available
ONR MURI: NexGeNetSci
Moment Problems, SOS and SDP
• Using duality theory, we obtain the following formulation
[Bertsimas, 2005]:
This dual problem is a sum-of-squares program (SOSP) and
can be formulated as a semidefinite program [Parrilo, 2006].
• We define the spectral distribution of a graph as
and define the r.v. XmG. Hence, using SOS, we can compute
optimal bounds on Pr(X  T)=#{i  T}/n when we have
access to a sequence of spectral moments
ONR MURI: NexGeNetSci
Numerical Results
• From the set of spectral moments, we compute optimal bounds
on #{i  [a,b]}/n,
and
#{i  [-c,c]}/n
B
b
1
a*
a
C
c
• Notice that only those intervals [a,b] in region B and [-c,c] in C
are able to support the whole set of eigenvalues. Hence,
– We have a lower bound on the spectral radius r(A)>a*
– We can also compute a bound on the Laplacian eigenratio from
the Laplacian spectral moments
ONR MURI: NexGeNetSci
IV Dynamical Implications: Spreading Processes
 We study a stochastic dynamical model of viral dissemination:
 - Each node has two possible
b
b
d
states:
 0. Susceptible (blue)
 1. Infected (red)
 - Spreading parameters:
 b probability of contagion
 d probability of recovering
 Spectral results [Draief et al., 2008]:
 - r(A)>d/b is a necessary condition for a small infection to infect
a significant part of the network
 - The larger r(A), the better a network disseminate a virus/rumor
ONR MURI: NexGeNetSci
Spreading Processes: Simulations
Counting
subgraph
frequencies
Computing
spectral
moments
Bound on
r(A)>45.0
0.2% initial infection
d/b=65 > r(A)=60
300
300
250
250
# of infections
# of infections
0.05% initial infection
d/b=35 < 45 < r(A)=60
200
150
100
50
0
t
0
20
40
60
80
100
120
140
Implications
on Spreading
160
ONR MURI: NexGeNetSci
200
150
100
50
0
t
0
10
20
30
40
50
Decentralized Network Design
• Ongoing work: Design incentives for each individual to modify their
local neighborhood in order to achieve a particular global spectral
property. Some preliminaries results [Preciado et al., 2010]:
ONR MURI: NexGeNetSci
Future Work
• Adapt our framework to:
1. Evolving networks: Tracking the evolution of
subgraphs frequencies and model their interactions
2. Links with weights and directions: Since the
eigenvalues become complex, we have to work with
2D support
3. Nodes with attributes
• Incentive design: How can we drive nodes to
take local actions that improve the global
dynamical behavior of the network?
ONR MURI: NexGeNetSci
Conclusions
• Our work is devoted to study local structural properties and
dynamical processes in large-scale complex networks
• There is a direct relationship between many dynamical
processes in networks and the eigenvalues of the underlying
graph
• There is plenty of information about the eigenvalue spectra
from the distribution of local network measurements
• Our approach is agnostic, in which we do not assume any
global structure (no random graphs)
• Our results can be of interest to analyze and design largescale networks from a spectral point of view
ONR MURI: NexGeNetSci
Some References
•
•
•
•
•
•
•
•
•
•
F. Chung, L. Lu, and V. Vu, "The Spectra of Random Graphs with Given Expected Degrees," Internet
Mathematics, vol. 1, pp. 257-275, 2003.
M. Draief, A. Ganesh, and L. Massoulié, "Thresholds for Virus Spread on Networks," Annals of
Applied Probability, vol. 18, pp. 359-378, 2008.
L. Li, D. Alderson, J.C. Doyle, and W. Willinger, " Towards a Theory of Scale-Free Graphs,“ Interner
Math, vol. 2, pp. 431-523, 2005.
P. Parrilo, Algebraic Techniques and Semidefinite Optimization, Massachusetts Institute of
Technology: MIT OpenCourseWare, Spring 2006.
L.M. Pecora, and T.L. Carroll, "Master Stability Functions for Synchronized Coupled Systems,"
Physical Review Letters, vol. 80(10), pp. 2109-2112, 1998.
I. Popescu and D. Bertsimas, "An SDP Approach to Optimal Moment Bounds for Convex Classes of
Distributions," Mathematics of Operation Research, vol. 50, pp. 632-657, 2005.
V.M. Preciado, and G.C. Verghese, "Synchronization in Generalized Erdös-Rényi Networks of
Nonlinear Oscillators," IEEE Conference on Decision and Control, pp. 4628-4633, 2005.
V.M. Preciado, and A. Jadbabaie, "Spectral Analysis of Virus Spreading in Random Geometric
Networks," IEEE Conference on Decision and Control, pp. 4802-4807, 2009.
V.M. Preciado, M.M. Zavlanos, A. Jadbabaie, and G.J. Pappas, “Distributed Control of the Laplacian
Spectral Moments of a Network,” American Control Conference, 2010.
V.M. Preciado and A. Jadbabaie, " From Local Measurements to Network Spectral Properties:
Beyond Degree Distributions, " IEEE Conference on Decision and Control, 2010.
ONR MURI: NexGeNetSci
• QUESTIONS?
ONR MURI: NexGeNetSci
IV.b Dynamical Implications: Synchronization
• We study a collection of resistively coupled nonlinear oscillators
g
g
 Network dynamics:
g
g
g
 Question: What values of g do
make the network synchronize?
 Spectral results [Pecora and Carrol]: L=D-A, and ki are its eigenvalues
 For stability of the synchronous state we need the Laplacian
eigenration kn/k2 < t , where t depends on the individual oscillator
dynamics
ONR MURI: NexGeNetSci
Laplacian Moments
•
•
•
Using our SDP approach, we can also bound the Laplacian eigenratio
from local structural properties via the Laplacian moments
In the Laplacian moments, not only the frequencies of subgraphs are
important, but also the degrees of the nodes involved. For example, for the
4th moment the following substructures are involved
The Laplacian moments are functions of the frequencies of these
structures and the degrees of the nodes involved:
m4 ( L) 
(

1
2
2
8P4c   6P (k1  k 2  k 3    4P   4PE k1  k 2  k1k 2  1

n k1 ,k2 ,k3 ,k4
k1 , k 2 , k3
k1 , k 2 , k3
k1 , k 2 , k3
ONR MURI: NexGeNetSci
Synchronization: Simulations
• We simulate a network of 200 resistively coupled Rossler oscillators
Counting
frequencies of
substructures
Computing
Laplacian
moments
Bound on
kn/k2
ONR MURI: NexGeNetSci
Implications on
Synchronization