CMA 2009
The statistics of eigenvalue distributions in large
random matrices
Øyvind Ryan
October 2009
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
Random matrices
I
Have numerous applications: wireless communications,
nance, nuclear physics.
I
In many applications, large matrices are assumed.
I
It turns out that asymptotic results on the eigenvalue
distributions (i.e. results which hold in the limit when the
matrices grow large) can be found more easily than in the
nite matrix regime.
I
These asymptotic results can also be put into a matrix-based
probability framework.
I
The asymptotic results can also be applied successfully as
approximations to the nite matrix cases.
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
Denition
With the empirical eigenvalue distribution of an L × L hermitian
random matrix T we mean the (random) function
FT (λ) =
#{i |λi ≤ λ}
,
L
(1)
where λi are the (random) eigenvalues of T.
Questions addressed in random matrix theory:
1. When Tn is a random matrix ensemble, does FTn converge to
a non-random function as n → ∞, and how can one express
this function?
2. How does the statistics of the largest and smallest eigenvalue
of Tn behave as n → ∞?
3. Given independent random matrix ensembles Sn , Tn , when do
FSn Tn and FSn +Tn converge to a non-random function as
n → ∞, and how can this be expressed in terms of FSn and
FTn ?
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
I
Remarkably, for many ensembles of random matrices, one has
that FTn becomes "less random" when dimensions increase,
meaning that more or less the same eigenvalue distribution is
observed for dierent realizations when n is large.
I
Also, for many ensembles of random matrices An , Bn , when
An and Bn are independent, combinations such as An Bn and
An + Bn exhibit eigenvalue distributions which, for large
dimensions and for all realizations, can be completely
determined from that of An and Bn .
I
This last statement is very much in contrast to the fact that
the eigenvalues of A + B or AB in general can't be determined
from that of A and B. The statement is thus something which
applies only for certain large random matrices.
In random matrix theory, there are two methods which are widely
used to infer on the empirical eigenvalue distribution: the Stieltjes
transform approach, and the moment approach.
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
The Stieltjes transform approach
Denition
The Stieltjes transform of FT is the function
Z
1
dFT (x ) = tr((T − z I)−1 )
mFT (z ) =
x −z
I
The distribution function FT can be recovered from the
Stieltjes transform mFT , using the so-called Stieltjes inversion
formula.
I
Many limit eigenvalue distributions of random matrices can be
characterized by the fact that their Stieltjes transforms are
solutions to certain equations.
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
The moment approach
Denition
The k'th moment of FT is the number
Z
Tk = x n dFT (x ) = tr(Tk ).
P 1/2k
One can show that, if
T2k = ∞, then FT is uniquely
determined by the sequence of moments {Tk }∞
k =0 .
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
Gaussian matrices
I
It turns out that when Xn is n × n and Gaussian, the
eigenvalue distribution of Dn Xn and Dn + Xn depends only on
that of Dn for large n.
I
Similarly, the eigenvalue distribution of Dn can be determined
from that of Dn Xn or Dn + Xn . This is important in
applications, since Xn often represents some sort of noise, so
that the result can be used for denoising [1].
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
I
Statistics of eigenvalue distributions
The way we compute the eigenvalue distribution of Dn Xn and
Dn + Xn from that of Dn and Xn can be put into a framework
for a new type of probability theory, where
I
I
I
I
matrices take the role as random variables,
the trace takes the role as expectation,
eigenvalue distributions take the role as the distributions of the
random variables,
the Gaussian law has a central role as a central limit in a new
type of central limit theorem.
This framework is called free probability [2].
I
Useful way to think about free probability: Two independent
random matrices, where the eigenvectors for one of them point
in each direction with equal probability, then the two matrices
are "free" (to be dened).
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
The full circle law
Let Xn = √1n Yn where Yn is n × n and has i.i.d. complex standard
Gaussian entries. When n grows large, the eigenvalue distribution
grows towards the central limit in free probability, and is called the
full circle law. Here for n = 500.
1
0.8
0.6
0.4
0.2
0
−0.2
−0.4
−0.6
−0.8
−1
−1
−0.5
0
0.5
1
plot(eig( (1/sqrt(1000)) * (randn(500,500) +
j*randn(500,500)) ),'kx')
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
The semicircle law
35
30
25
20
15
10
5
0
−3
−2
−1
0
1
2
3
A = (1/sqrt(2000)) * (randn(1000,1000) +
j*randn(1000,1000));
A = (sqrt(2)/2)*(A+A');
hist(eig(A),40)
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
The Marchenko Pastur law
What happens with the eigenvalues of N1 Xn XH
n when Xn is an
n × N random matrix with standard complex Gaussian entries?
I One can show that
· µµ
¶p ¶¸
X
X ak
1
H
Xn Xn
=
c l (π̂)−1 +
E tr
.
N
N 2k
π̂∈NC2p
I
k
Convergence is "almost sure", i.e. we have very accurate
eigenvalue prediction when the matrices are large.
When Nn → c, the eigenvalue distribution converges to the
Marchenko Pastur law with parameter c, denoted µ Nn , with
density
p
(x − a)+ (b − x )+
1 +
µc
f (x ) = (1 − ) δ(x ) +
,
(2)
c
2π cx
√
√
where (z )+ = max(0, z ), a = (1 − c )2 and b = (1 + c )2 .
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
Four dierent Marchenko Pastur laws µ Nn are drawn.
1.6
c=0.5
c=0.2
c=0.1
c=0.05
1.4
1.2
Density
1
0.8
0.6
0.4
0.2
0
0.5
1
Øyvind Ryan
1.5
2
2.5
3
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
The non-asymptotic case
I
Free probability is a tool for computing the asymptotic
eigenvalue distributions for large Gaussian matrices.
I
One also has important results in the nite matrix regime
when one replaces limit eigenvalue distributions with mean
eigenvalue distributions.
I
For instance, one can nd the mean eigenvalue distribution of
D + X/DX when D is deterministic and X is Gaussian.
I
The author is implementing a machinery for computing such
mean eigenvalue distributions [3].
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
Vandermonde matrices
Consider N × L Vandermonde matrices V

1
···
−j ω1

e
···
1
V= √ 
 ..
..
.
N .
e −j (N −1)ω1 · · ·
of the form
1
e −j ωL
..
.





(3)
e −j (N −1)ωL
I
The entries in V lie on the unit circle
I
The ωi are called phase distributions, and are assumed i.i.d. on
[0, 2π)
I
We assume N , L → ∞ at the same rate with limN →∞ NL = c.
The normalizing factor √1 is included to ensure limiting
N
asymptotic behaviour.
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
Various results exist on the distribution of the determinant of
Vandermonde matrices, but there are many open problems (below,
VH V is used since V is rectangular in general):
I
How can we nd the singular value distribution of V?
I
How can we infer on the moments of D from the mixed
moments DVH V?
I
How can we express the eigenvalue distribution of
combinations of independent Vandermonde matrices?
I
Asymptotic results? If X is an N × N standard, complex,
Gaussian matrix, then the asymptotic capacity is
¡
¡1
¢¢
limN →∞ N1 log
XXH = ´
N
³ 2 det I + ρ
¡√
¢2
¡√
¢2
2 log2 1 + ρ − 14
4ρ + 1 − 1
− log4ρ2 e
4ρ + 1 − 1 .
We are not aware of similar asymptotic expressions for the
determinant/capacity of Vandermonde matrices.
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
Main result [4]
Assume that
I
{Dr (N )}1≤r ≤n are diagonal L × L matrices which have a joint
limit distribution as N → ∞,
I L
N
→ c.
We would like to express the limits
Mn = lim E[tr(D1 (N )VH VD2 (N )VH V · · · × Dn (N )VH V)]. (4)
N →∞
It turns out that for a large family of Vandermonde matrice, the
limits (4) exist, and the limit depend only on the moments of V
and the moments of Di (N ).
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
Eigenvalue distributions of Vandermonde matrices
1
1
0.9
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
0
1
2
3
4
5
6
7
8
9
H
10
0
0
1
2
3
4
5
6
7
8
9
10
H
1
N XX ,
(a) V V, with V a 1600 × 800 Vander- (b)
with X an 800 × 1600 commonde matrix with uniform phase dis- plex, standard, Gaussian matrix.
tribution.
Figure: Histogram of mean eigenvalue distributions.
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
Main results
I
Vandermonde matrices display almost sure convergence.
I
Vandermonde matrices do not have compactly supported
limiting eigenvalue distributions (contrary to Gaussian
matrices).
I
Characterization of when a "mixed moment"
³
´
lim tr D1 (N )ViH1 Vi2 · · · Dn (N )ViH2n−1 Vi2n
N →∞
(5)
is computable from only knowledge of the spectra of V1 , V2 , ...
I
Characterization of all "convolution operations", such as
nding the spectrum of V1H V1 + V2H V2 from that of V1 and
V2 when they are independent and large.
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
Open questions
I
How can one establish an analytical machinery for
Vandermonde matrices, such as a transform with similar
properties as the R-transform in free probability?
I
Is there a general criterion for saying exactly when the
spectrum of A + B/AB can be determined from that of A and
B? Gaussian matrices are bi-unitarily invariant, which is a
sucient criterion. But it may not be necessary.
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
Software implementation
Software implementation [3] which computes
I
Any asymptotic moment of Vandermonde matrices
I
Any asymptotic moment of Gaussian matrices
I
Any moment of nite Gaussian matrices
I
Any combination of independent matrices of the above types
The software implementation is in Matlab and can generate
formulas for all moments in latex as well as compute the moments
numerically. The implementation
I
Iterates through partitions,
I
Computes many dierent equivalence relations on partitions,
I
performs Fourier-Motzkin elimination
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
I
This talk is available at
http://folk.uio.no/oyvindry/talks.shtml
I
My publications are listed at
http://folk.uio.no/oyvindry/publications.shtml
I
The recipe for the random matrix cake can be found at
http://folk.uio.no/oyvindry/kaker/kaloribombekake.pdf
THANK YOU!
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma
CMA 2009
Statistics of eigenvalue distributions
Ø. Ryan and M. Debbah, Channel capacity estimation using
free probability theory, IEEE Trans. Signal Process., vol. 56,
no. 11, pp. 56545667, November 2008.
F. Hiai and D. Petz, The Semicircle Law, Free Random
Variables and Entropy. American Mathematical Society, 2000.
Ø. Ryan, Documentation for the Random Matrix Library, 2009,
http://i.uio.no/~oyvindry/rmt/doc.pdf.
Ø. Ryan and M. Debbah, Asymptotic behaviour of random
vandermonde matrices with entries on the unit circle, IEEE
Trans. on Information Theory, vol. 55, no. 7, pp. 31153148,
2009.
Øyvind Ryan
The statistics of eigenvalue distributions in large random ma