A Proximal Gradient Algorithm for Tracking
Cascades over Networks
Brian Baingana, Gonzalo Mateos and Georgios B. Giannakis
Acknowledgments: NSF ECCS Grant No. 1202135 and NSF AST Grant No. 1247885
May 8, 2014
Florence, Italy
Context and motivation
Contagions
Infectious diseases
Buying patterns
Popular news stories
Network topologies:
Unobservable, dynamic, sparse
Propagate in cascades
over social networks
Topology inference vital:
Viral advertising, healthcare policy
Goal: track unobservable time-varying network topology from cascade traces
B. Baingana, G. Mateos, and G. B. Giannakis, ``A proximal-gradient algorithm for tracking cascades over
2
social networks,'' IEEE J. of Selected Topics in Signal Processing, Aug. 2014 (arXiv:1309.6683 [cs.SI]).
Contributions in context
 Structural equation models (SEM) [Goldberger’72]
 Statistical framework for modeling relational interactions (endo/exogenous effects)
 Used in economics, psychometrics, social sciences, genetics… [Pearl’09]
 Related work
 Static, undirected networks e.g., [Meinshausen-Buhlmann’06], [Friedman et al’07]
 MLE-based dynamic network inference [Rodriguez-Leskovec’13]
 Time-invariant sparse SEM for gene network inference [Cai-Bazerque-GG’13]
 Contributions
 Dynamic SEM for tracking slowly-varying sparse networks
 Accounting for external influences – Identifiability [Bazerque-Baingana-GG’13]
 First-order topology inference algorithm
D. Kaplan, Structural Equation Modeling: Foundations and Extensions, 2nd Ed., Sage, 2009.
3
Cascades over dynamic networks
 N-node directed, dynamic network, C cascades observed over
 Unknown (asymmetric) adjacency matrices
 Example: N = 16 websites, C = 2 news events, T = 2 days
Event #1
Event #2
 Node infection times depend on:
 Causal interactions among nodes (topological influences)
 Susceptibility to infection (non-topological influences)
4
Model and problem statement
 Data: Infection time of node i by contagion c during interval t:
un-modeled dynamics
external influence
Dynamic SEM
 Captures (directed) topological
and external influences
Problem statement:
5
Exponentially-weighted LS criterion
 Structural spatio-temporal properties
 Slowly time-varying topology
 Sparse edge connectivity,
 Sparsity-promoting exponentially-weighted least-squares (LS) estimator
(P1)
 Edge sparsity encouraged by
-norm regularization with
 Tracking dynamic topologies possible if
6
Topology-tracking algorithm
 Iterative shrinkage-thresholding algorithm (ISTA) [Parikh-Boyd’13]
 Ideal for composite convex + non-smooth cost
 Let
gradient descent
(P2)
Solvable by soft-thresholding operator [cf. Lasso]
-γ
γ
 Attractive features
 Provably convergent, closed-form updates (unconstrained LS and soft-thresholding)
 Fixed computational cost and memory storage requirement per
 Scales to large datasets
7
Recursive updates
 Sequential data terms in
recursive updates
: row i of
 Each time interval
Recursively update
Acquire new data
Solve (P2) using (F)ISTA
8
Simulation setup
 Kronecker graph [Leskovec et al’10]: N = 64, seed graph
 Non-zero edge weights varied for

1
0.5
 Uniform random selection from
0
0
20
40
60
80
100
120
140
160
180
200
0
20
40
60
80
100
120
140
160
180
200
0
20
40
60
80
100
120
140
160
180
200
0
20
40
60
80
100
120
140
160
180
200
1
edge weight
0.5
0
1
0.5
0
1
0
 Non-smooth edge weight variation

cascades,
−1
time
,
9
Simulation results
 Algorithm parameters

20
20
40
40
60
60
20
40
actual, t=20
60
20
40
inferred, t=20
20
20
40
40
60
60
20
40
actual, t=180
60
60
20
40
60
inferred, t=180
 Error performance
10
The rise of Kim Jong-un
 Web mentions of “Kim Jong-un” tracked from March’11 to Feb.’12
Kim Jong-un – Supreme leader of N. Korea
 N = 360 websites, C = 466 cascades, T = 45 weeks
Increased media frenzy following Kim
Jong-un’s ascent to power in 2011
t = 10 weeks
t = 40 weeks
Data: SNAP’s “Web and blog datasets” http://snap.stanford.edu/infopath/data.html
11
LinkedIn goes public
 Tracking phrase “Reid Hoffman” between March’11 and Feb.’12
 N = 125 websites, C = 85 cascades, T = 41 weeks
US sites
t = 5 weeks
t = 30 weeks
 Datasets include other interesting “memes”: “Amy Winehouse”, “Syria”, “Wikileaks”,….
Data: SNAP’s “Web and blog datasets” http://snap.stanford.edu/infopath/data.html
12
Conclusions

Dynamic SEM for modeling node infection times due to cascades
 Topological influences and external sources of information diffusion
 Accounts for edge sparsity typical of social networks

Proximal gradient algorithm for tracking slowly-varying network topologies
 Corroborating tests with synthetic and real cascades of online social media
 Key events manifested as network connectivity changes
 Ongoing and future research





Dynamical models with memory
Identifiabiality of sparse and dynamic SEMs
Statistical model consistency tied to
Large-scale MapReduce/GraphLab implementations
Kernel extensions for network topology forecasting
Thank You!
13
ISTA
iterations
PG iterations
with equality constraints yield the (pseudo) real-time tracking algorithm:
T
als.
R equir e: Y t t= 1, X , β.
ˆ 0 = 0N × N , Bˆ 0 = Σ 0 = I N , Y¯ 0 = 0N × C , λ 0.
1: Initialize A
2: for t = 1, . . . , T do
3:
Σ t = βΣ t− 1 + Y t (Y t )⊤
Recursive Updates
4:
Y¯ t = β Y¯ t− 1 + Y t
5:
Initialize A [0] = Aˆ t− 1, B [0] = Bˆ t− 1, and set k = 0.
6:
while not converged do
7:
for i = 1. . . N (in parallel) do
8:
a− i [k + 1] = Sλ t / L f a− i [k] − (1/ L f )∇ a− i f [k]
Parallelizable
9:
bi i [k + 1] = bi i [k] − (1/ L f )∇ bi i f [k]
10:
a⊤
i [k + 1] = [a− i ,1[k + 1] . . . a− i ,i − 1[k + 1] 0 a− i ,i [k + 1] . . . a− i ,N [k + 1]]
11:
end for
12:
k = k + 1.
13:
end while
14:
r et ur n Aˆ t = A [k], Bˆ t = B [k].
15: end for
A t t r act ive feat ur es of t he algor it hm :
for
1. Provably guaranteed convergence
14
ADMM iterations
 Sequential data terms:
,
,
can be updated recursively:
denotes row i of
15
ADMM closed-form updates
 Update
with equality constraints:
,

:
 Update
by soft-thresholding operator
16
Outlook: Indentifiability of DSEMs
a1) edge sparsity:
a2) sparse changes:
a3) error-free DSEM:
Goal: under a1)-a3), establish conditions on
to uniquely identify
 Preliminary result (static SEM)
If
, with
and diagonal matrix
and i)
, ii)
non-zero entries of
are drawn from a continuous distribution, and iii) Kruskal
rank
, then
and
can be uniquely determined.
J. A. Bazerque, B. Baingana, and G. B. Giannakis, "Identifiability of sparse structural equation models for directed, cyclic, and time-varying
networks," Proc. of Global Conf. on Signal and Info. Processing, Austin, TX, December 3-5, 2013.
17