Boulder 2008
Applications and fundamental results on random
Vandermonde matrices
Øyvind Ryan
July 2008
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Free probability
I
I
I
I
I
Sheds new light on results in random matrix theory: If the
eigenvalue distributions of independent n × n-random matrices
An , Bn exhibit some form of convergence, and one of them
exhibits unitary invariance, then the eigenvalue distribution of
An + Bn can be computed in the large n-limit.
The limit eigenvalue distribution of An + Bn is the additive
free convolution of the limiting eigenvalue distributions of An
and Bn .
Free convolution can be expressed nicely in terms of the limit
moments of the matrices.
Free convolution can also be expressed nicely in terms of
equations involving the Stieltjes transform of the involved
measures.
The Poisson distribution has its analogue in the free Poisson
distribution (the Marchenko Pastur law) in free probability
theory.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Does other types of random matrices (i.e. non-unitarily
invariant) t into a similar framework?
We have investigated this for Vandermonde matrices [1, 2], which
are widely used. They have the form


1
··· 1
 x1

· · · xL

V=

 ..
..
..
 .

.
.
N −1
N −1
x1
· · · xL
It is straightforward to show that square Vandermonde matrices
have determinant
Y
(xl − xk ).
det(V) =
1≤k <l ≤N
In particular, V is nonsingular if the xk are dierent.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Various results exist on the distribution of the determinant of
Vandermonde matrices (Gaussian entries (Metha), entries with
B-distribution (Selberg)), but there are many open problems
(below, VH V is used since V is rectangular in general):
I How can³we nd the
¡ H ¢k ´ moments of the Vandermonde matrices
(i.e. trL V V
) (not the determinant itself)?
I
I
I
Deconvolution problem: How to estimate the moments of D
from mixed moments DVH V?
Mixed moments of independent Vandermonde matrices?
Asymptotic results? If X is an N × N standard, complex,
Gaussian matrix, then
¡
¡1
¢¢
H
limN →∞ N1 log
det
I
+
ρ
XX
= ´
2
N
³
¢2
¢2
¡
¡√
1 √
2 log2 1 + ρ − 4
4ρ + 1 − 1
− log4ρ2 e
4ρ + 1 − 1 .
(which is the expression for the capacity). We are not aware of
similar asymptotic expressions for the determinant/capacity of
Vandermonde matrices.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Random Vandermonde matrices
We will consider Vandermonde matrices V of dimension N × L of
the form


1
··· 1
−j ω

· · · e −j ωL
1  e 1

V= √ 
(1)
 ..

.
.
.
.

. .
N .
e −j (N −1)ω1 · · · e −j (N −1)ωL
(i.e. we assume that the xi lie on the unit circle). The ωi are called
phase distributions. We will limit the study of Vandermonde
matrices to cases where
I
I
The phase distributions are i.i.d.
The asymptotic case N , L → ∞ with limN →∞ NL = c. The
normalizing factor √1 is included to ensure limiting
N
asymptotic behaviour.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Where can such Vandermonde matrices appear?
Consider a multi-path channel of the form:
h(τ ) =
L
X
αi g (τ − τi )
i =1
αi are i.d. Gaussian random variables with power Pi ,
I τi are uniformly distributed delays over [0, T ],
I g is the low pass transmit lter.
I L is the number of paths
In the frequency domain, the channel is given by:
I
c (f ) =
L
X
αi G (f )e −j2πf τi
i =1
We suppose the transmit lter to be ideal (G (f ) = 1).
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Sampling the continuous frequency signal at fi = i W
N (N is the
number of frequency samples) where W is the bandwidth, our
model becomes

 

α1
n1
1
.   . 
r = VP 2 
(2)
 ..  +  ..  ,
αL
nN
where V is a random Vandermonde matrix of the type (1), and
I
P is the L × L diagonal power matrix,
I
ni is independent, additive, white, zero mean Gaussian noise of
variance √σ .
N
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Main result
Denition
Dene
Kρ,ω,N =
RN
1
n+1−|ρ| ×
(0,2π)|ρ|
Qn
jN (ω
−ω
)
b(k −1)
b(k )
1−e
k =1 1−e j (ωb(k −1) −ωb(k ) )
,
(3)
d ω1 · · · d ω|ρ| ,
where ωρ1 , ..., ωρ|ρ| are i.i.d. (indexed by the blocks of ρ), all with
the same distribution as ω , and where b(k ) is the block of ρ which
contains k (where notation is cyclic, i.e. b(−1) = b(n)). If the limit
Kρ,ω = lim Kρ,ω,N
N →∞
exists, then Kρ,ω is called a Vandermonde mixed moment expansion
coecient.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Main result 2
Assume that
I
{Dr (N )}1≤r ≤n are diagonal L × L matrices which have a joint
limit distribution as N → ∞,
I L
N
→ c.
We would like to express the limits
Mn = lim E [trL (D1 (N )VH VD2 (N )VH V · · · × Dn (N )VH V)]. (4)
N →∞
It turns out that this is feasible when all Vandermonde mixed
moment expansion coecients Kρ,ω exist.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
For convenience, dene
£ ¡¡
¢n ¢¤
mn = (cM )n = c limN →∞ E trL D(N )VH V
,
dn = (cD )n = c limN →∞ trL (Dn (N )) ,
(5)
Theorem
Assume D1 (N ) = D2 (N ) = · · · = Dn (N ). When ω = u,
m1 = d1
m2 = d2 + d12
m3 = d3 + 3d2 d1 + d13
m4 = d4 + 4d3 d1 + 8/3d22 + 6d2 d12 + d14
m5 = d5 + 5d4 d1 + 25/3d3 d2 + 10d3 d12 + 40/3d22 d1 + 10d2 d13 + d15
m6 = d6 + 6d5 d1 + 12d4 d2 + 15d4 d12 + 151/20d32 + 50d3 d2 d1
+20d3 d13 + 11d23 + 40d22 d12 + 15d2 d14 + d16
m7 = d7 + 7d6 d1 + 49/3d5 d2 + 21d5 d12 + 497/20d4 d3 + 84d4 d2 d1
+35d4 d13 + 1057/20d32 d1 + 693/10d3 d22 + 175d3 d2 d12
+35d3 d14 + 77d23 d1 + 280/3d22 d13 + 21d2 d15 + d17 .
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Comparison
The Gaussian equivalent of this is
m1
m2
m3
m4
m5
m6
d1
d2 + d12
d3 + 3d2 d1 + d13
d4 + 4d3 d1 + 3d22 + 6d2 d12 + d14
d5 + 5d4 d1 + 5d3 d2 + 10d3 d12 + 10d22 d1 + 10d2 d13 + d15
d6 + 6d5 d1 + 6d4 d2 + 15d4 d12 + 3d32 + 30d3 d2 d1
+20d3 d13 + 5d23 + 10d22 d12 + 15d2 d14 + d16
m7 = d7 + 7d6 d1 + 7d5 d2 + 21d5 d12 + 7d4 d3 + 42d4 d2 d1
+35d4 d13 + 21d32 d1 + 21d3 d22 + 105d3 d2 d12
+35d3 d14 + 35d23 d1 + 70d22 d13 + 21d2 d15 + d17 ,
(6)
1
H
H
when we replace V V with N XX , with X an L × N complex,
standard, Gaussian matrix.
=
=
=
=
=
=
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Sketch of proof
We can write
h ³
´i
E trL D1 (N )VH VD2 (N )VH V · · · Dn (N )VH V
as
L−1
P
i1 ,...,in
j1 ,...,jn
(7)
E ( D1 (N )(j1 , j1 )VH (j1 , i2 )V(i2 , j2 )
D2 (N )(j2 , j2 )VH (j2 , i3 )V(i3 , j3 )
..
.
(8)
Dn (N )(jn , jn )VH (jn , i1 )V(i1 , j1 ))
The (j1 , ..., jn ) give rise to a partition ρ of {1, ..., n}, where each
block ρj consists of equal values, i.e.
ρj = {k |jk = j }.
This ρ will actually represent the ρ used in the denition of Kρ,ω,n .
The rest of the proof goes by carefully computing this limit
quantity using much combinatorics.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Comparisons
Denote by
µµ
¶
¶∗n
λ
λ
ν(λ, α) = lim
1−
δ − 0 + δα
n→∞
n
n
the (classical) Poisson distribution of rate λ and jump size α.
Denote also by
¶
¶¢n
µµ
λ
λ
δ − 0 + δα
µ(λ, α) = lim
1−
n→∞
n
n
the free Poisson distribution of rate λ and jump size α (the
Marchenko Pastur law). Denote also µc = µ( c1 , c ), νc = ν(c , 1).
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Comparisons 2
Corollary
Assume that V has uniformly distributed phases. Then the limit
moment
h ³³
´n ´i
Mn = lim E trL VH V
N →∞
satsies the inequality
φ(a1n ) ≤ Mn ≤
1
E (a2n ),
c
where a1 ∼ µc , a2 ∼ νc . In particular, equality occurs for
m = 1, 2, 3 and c = 1.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Comparisons 3
1
1
0.9
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
0
1
2
3
4
5
6
7
8
9
10
0
0
1
2
3
4
5
6
7
8
9
10
H
1
N XX ,
H
with X an 800 × 1600 com(a) V V, with V a 1600 × 800 Van- (b)
dermonde matrix with uniformly dis- plex, standard, Gaussian matrix.
tributed phases.
Figure: Histogram of mean eigenvalue distributions.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Other results
I
Mixed moments of (more than one) independent Vandermonde
matrices.
I
Generalized
¡
¢Vandermonde matrices: These have the form
V = e j αk βl 1≤k ≤N ,1≤l ≤L . It is known that V is nonsingular i
all αk are dierent, and all βl are dierent. [1, 2] also contain
results on the asymptotics of generalized Vandermonde
matrices.
I
Exact moments of lower order Vandermonde matrices. Reveals
slower convergence (O ( n1 , contrary to O ( n12 ) for Gaussian
counterpart).
I
Computation of the asymptotic moments when the phase
distribution is not uniform. Phase distributions with continous
density, and phase distributions with singularities.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Application 1: Estimation of the number of paths
Return to the multi-path channel model (2). For simplicity, set
W = T = 1, so that the phase distribution of the Vandermonde
matrix is uniform. We take K observations of (2) and form the
observation matrix
Y = [r1 · · · rK ]

(1)
(K )
α1
1  .
= VP 2  ..
···
..
.
α1
..
.
αL
···
αL
(1)
(K )


(1)
(K )
n1
  ..
+ .
···
..
.
n1
..
.
nN
···
nN
(1)


,
(K )
(9)
It is now possible to combine the deconvolution result for
Vandermonde matrices with known deconvolution results for
Gaussian matrices to estimate L from a number of observations
(assuming P is known). All values of L are tried, and the one which
"best matches" the observed values is chosen:
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Estimation of the number of paths 2
Proposition
Assume that V has uniformly distributed phases, and let mPi be the
^i ) the moments of the sample
moments of P, and mR̂i = trN (R
covariance matrix
^ = 1 YYH .
R
K
N
, c2 = NL , and c3 = KL . Then
Dene also c1 = K
£ ¤
E mR̂ = c2 mP1 + σ 2
µ
¶
h i
1
2
E mR̂ = c2 1 −
mP2 + c2 (c2 + c3 )(mP1 )2
N
+2σ 2 (c2 + c3 )mP1 + σ 4 (1 + c1 )
h i
E mR̂3
= ···
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Estimation of the number of paths 3
70
70
Estimate of L
Actual value of L
50
50
40
40
Estimate of L
Actual value of L
L
60
L
60
30
30
20
20
10
10
0
0
10
20
30
40
50
60
Number of observations
70
80
90
(a) K = 1
100
0
0
10
20
30
40
50
60
Number of observations
70
80
90
100
(b) K = 10
Figure:
√ Estimate for the number of paths. Actual value of L is 36. Also,
σ = 0.1, N = 100.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Application 2: Wireless capacity Analysis
For a general matrix W, the mean capacity is dened as
¡
¡
¢¢
CN = N1 E log2 det IN + σ12 WWH
¡
¡
¡
¢¢¢ R
¡
P
1
H
= N1 N
= log2 1 +
k =1 E log2 1 + σ 2 λk WW
¢
t µ(dt )
(10)
H
where µ is the mean empirical eigenvalue distribution of WW .
P
k +1 t k ,
Substituting the Taylor series log2 (1 + t ) = ln12 ∞
k =1 (−1)
k
we obtain
P
(−1)k +1 mk (µ)ρk
(11)
,
CN = ln12 ∞
k =1
k
where ρ is SNR, and where
mk (µ) =
1
σ2
Z
t k d µ(t ) for k ∈ Z+
However, if W is a Vandermonde matrix, many more moments are
required for precise estimation of capacity than we can provide with
the formulas for the rst 7 moments.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Wireless capacity Analysis 2
3
Asymptotic capacity
sample capacity
2.5
Capacity
2
1.5
1
0.5
0
0
1
2
3
4
5
ρ
6
7
8
9
10
¡
¢
Figure: Several realizations of the capacity N1 log2 det I + ρ N1 XXH
when X is standard, complex, Gaussian. Matrices of size 36 × 36 were
used. The known expression for the asymptotic capacity is also shown.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
3
3
2.5
2.5
2
2
Capacity
Capacity
Wireless capacity Analysis 3
1.5
1.5
1
1
0.5
0.5
0
0
1
2
3
4
5
ρ
6
7
8
(a)
¡Realizations
H¢
1
when
N log2 det I + ρVV
has uniform phase distribution.
9
10
0
0
1
2
3
4
5
ρ
6
7
8
9
10
of (b)
of
¡Realizations
¢
ω N1 log2 det I + ρVVH
when
ω
has a certain non-uniform phase
distribution.
Figure: Several realizations of the capacity for Vandermonde matrices for
two dierent phase distributions. Matrices of size 36 × 36 were used.
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Open questions
I
I
I
I
I
Do Vandermonde matrices display almost sure convergence?
How can one establish an analytical machinery for
Vandermonde matrices, such as a transform with similar
properties as the R-transform in free probability?
Have Vandermonde matrices compactly supported limiting
eigenvalue distributions?
Central limit theorem
Initial calculations suggest that the limit moments of a sum
VωH1 ,c Vω1 ,c + VωH2 ,c Vω2 ,c
(where Vω1 ,c , Vω2 ,c are independent, ω1 , ω2 are phase
distributions, c is the limit aspect ratio) equals the limit
moments of
VωH3 ,d Vω3 ,d
(for certain ω3 , d). What are the properties of such "additive
Vandermonde convolution".
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
I
This talk is available at
http://heim.i.uio.no/∼oyvindry/talks.shtml.
I
My publications are listed at
http://heim.i.uio.no/∼oyvindry/publications.shtml
THANK YOU!
Øyvind Ryan
Applications and fundamental results on random Vandermon
Boulder 2008
Applications and fundamental results on random Vandermonde
Ø. Ryan and M. Debbah, Random Vandermonde matrices-part
I: Fundamental results, Submitted to IEEE Trans. on
Information Theory, 2008.
, Random Vandermonde matrices-part II: Applications,
Submitted to IEEE Trans. on Information Theory, 2008.
Øyvind Ryan
Applications and fundamental results on random Vandermon