R ANDOM WALK M ODELS OF S PARSE G RAPHS
Peter Orbanz · Columbia University
Collaborators
Benjamin Reddy · Columbia
Daniel M. Roy · Toronto
Balázs Szegedy · Alfréd Rényi Institute
G RAPH -VALUED DATA
Peter Orbanz
2 / 13
E XCHANGEABLE G RAPHS
π
random graph exchangeable
m
adjacency matrix jointly exchangeable
π
Representation theorem (Aldous, Hoover)
Any exchangeable graph can be sampled as:
1. Sample measurable and symmetric function
Θ : [0, 1]2
0
[0, 1]
Ui
2. Sample U1 , U2 , . . . ∼iid Uniform[0, 1]
Uj
0
Ui
Uj
3. Sample edge i,j ∼ Bernoulli(Θ(Ui , Uj ))
1
Peter Orbanz
1
3 / 13
C ONVERGENCE
Law of large numbers (Kallenberg, ’99; Lovász & Szegedy, ’06)
weakly
≡
−−−→
weakly
−−−→
Lifting Theorem (O. & Szegedy, 2011)
There exists a measurable mapping
b
ξ: W
W
such that
ξ(θ̂) ∈ [θ̂]≡
b .
for all θ̂ ∈ W
set of equivalence classes
Peter Orbanz
4 / 13
A PPLICATIONS IN S TATISTICS
Model ⊂ {Pθ | θ : [0, 1]2 → [0, 1] measurable }
Semiparametric
Nonparametric (Bayesian)
Nonparametric ML
Estimate functional of θ
Prior on θ
ML estimator of θ
Parametric
e.g. ERGMs
Peter Orbanz
Bickel, Chen & Levina (2011)
Lloyd et al (2012)
Wolfe & Olhede (2014)
Gao, Lu & Zhou (2014)
(too many)
5 / 13
A PPLICATIONS IN S TATISTICS
Model ⊂ {Pθ | θ : [0, 1]2 → [0, 1] measurable }
Semiparametric
Nonparametric (Bayesian)
Nonparametric ML
Estimate functional of θ
Prior on θ
ML estimator of θ
Parametric
e.g. ERGMs
Peter Orbanz
Bickel, Chen & Levina (2011)
Lloyd et al (2012)
Wolfe & Olhede (2014)
Gao, Lu & Zhou (2014)
(too many)
5 / 13
M ISSPECIFICATION P ROBLEM
Dense vs sparse
A random graph is called
I
dense if #edges = Ω((#vertices)2 ).
I
sparse if #edges = O(#vertices).
Most real-world graph data ("networks") is sparse.
Exchangeable graphs are dense (or empty)
Z
p=
Θ(x, y)dxdy
p̂n =
# edges observed in Gn
= p · Ω(n2 ) = Ω(n2 )
# edges in complete graph
Sparsification (Bollobas & Riordan)
Generating sparse graphs: Choose decreasing rate function ρ : N → R+
1. U1 , U2 , . . . ∼iid Uniform[0, 1]
2. edge(i, j) ∼ Bernoulli(ρ(#vertices)θ(Ui , Uj ))
Peter Orbanz
6 / 13
S AMPLING FROM AN E XCHANGEABLE M ODEL
Input data
Exchangeable sampling
(rate ρ(n) = 1)
Peter Orbanz
Estimate of parameter function
Rate ρ(n) =
1
log(n)
Rate ρ(n) =
1
n
7 / 13
S AMPLING IN S PARSE G RAPHS
Sampling with random walks
I
Independent subsampling only adequate in dense case
I
Sparse case: Need to follow existing link structure
I
This means edges are not conditionally independent
A simple random walk model
For n = 1, . . . , #edges:
1. Choose a vertex vn uniformly at random from the current graph.
2. With probability α, create a new vertex attached to vn .
3. With probability 1 − α: Start a simple random walk at vn .
Stop after Poisson(λ) + 1 steps.
Connect vn to terminal vertex.
Peter Orbanz
8 / 13
PARAMETER E FFECTS
Increasing probability of new vertex
Increasing probability of long random walk
Peter Orbanz
9 / 13
PARAMETER E STIMATION
(with Benjamin Reddy)
Probability of random walk in graph G
degree matrix
P(v → u|G, λ) =
∞
X
−λ λ
e
k=0
graph Laplacian
k
k!
(D
−1
A)
k+1
vu
= (I −
∆)e−λ∆ vu
adjacency matrix
Poisson weight
Parameter estimation
I
Case 1: History observed
P(GN |α, λ) =
N−1
Y
P(Gn+1 |Gn , α, λ)
n=1
Maximum likelihood estimator derivable from random walk probability.
I
Peter Orbanz
Case 2: Only final graph observed
Impute history by sampling. Importance sampling possible.
10 / 13
R ESULTS : C ARTOON
Input data
Reconstruction
exchangeable model
Peter Orbanz
preferential attachment model
random walk model
11 / 13
I NVARIANCE U NDER A G ROUP
all invariant measures
(e.g. all exchangeable graph laws)
extreme points: ergodic measures
(e.g. all graphs sampled from graphons)
every invariant measure
is mixture of extreme points
Consequences
I
Representation theorem (Aldous-Hoover, de Finetti, etc).
I
Invariant statistical models are subsets of extreme points.
I
Law of large numbers: Convergence to extreme points.
Open problem
What is a statistically useful notion of invariance in sparse graphs?
Peter Orbanz
12 / 13
C ONCLUSIONS
Approach
Statistical theory of graphs as statistical theory of random structures.
What we have
I
Exchangeable graphs: Some technical problems aside, statistics carries over.
I
“Dependent” edge case:
I
I
I
I
Inference possible in principle.
No general theory.
Excellent prediction results with very simple model.
We cannot prove much about the model.
Well-studied invariance properties
1. Exchangeability: Only dense case.
2. Involution invariance: Too weak for statistical purposes.
Peter Orbanz
13 / 13