The CMA Seminar, 25. November 2010
Some particular types of random matrices and
their applictions
Øyvind Ryan
October 2010
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Setting
I
I
I
I
I
If A and B are matrices, the spectra of A + B and AB in
general depends on more than just the spectra of A and B.
The dependence is also on the eigenvector structures of A and
B: A and PAP−1 have the same eigenvalues for any invertible
P, but may not have the same eigenvectors.
If A and B are random, and independent, are there cases
where we still can say something about the spectrum of A + B
and AB, from those of A and B? We refer to this as the
problem of spectral separation.
The cases of spectral separation we will discuss are limited:
Matrices are large, have very specific forms. We will show that
random Gaussian matrices and Vandermonde matrices are
specific cases.
Spectral behaviour of such matrices find applications in various
fields: physics, finance, wireless communication. We will
discuss some of these.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Moments and mixed moments
Many probability
distributions are uniquely determined by their
R n
moments t d µ(t) (Carlemans theorem), and can thus be used to
characterize the spectrum of a random matrix.
I
Let tr be the normalized trace, and E[·] the expectation.
I
The quantities Ak = E[tr(Ak )] are called the (individual)
moments of A.
I
More generally, if Ai are random matrices,
E[tr(Ai1 Ai2 · · · Aik )]
is called a mixed moment in the Ai , when i1 6= i2 , i2 6= i3 , . . ..
I
More generally, we can define a mixed moment in terms of
algebras: if Ai are algebras, Ai ∈ Aki with ki 6= ki+1 for all i.
Spectral separation boils down to finding a computational rule for
computing mixed moments from individual moments.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Vandermonde matrices
An N × L Vandermonde matrix with entries on the unit circle [1] is
on the form


1
··· 1
−jω1

· · · e −jωL
1 
 e

V = √  ..
(1)

..
..


.
N
.
.
e −j(N−1)ω1 · · · e −j(N−1)ωL
I
I
I
ω1 ,...,ωL , also called phases, are assumed i.i.d., taking values
in [0, 2π).
N and L go to infinity at the same rate, c = limN→∞ NL (the
aspect ratio).
In general, the entries of a Vandermonde matrix need not to lit
on the unit circle, or be random. So this is a specialization of
Vandermonde matrices, to fit certain applications.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Questions
I
What can we say about the eigenvalue distributions in such
matrices?
I
Can we say something in particular when the matrices grow
large? Motivated by results for other matrices, where the
results are simpler in the large dimensional limit.
I
How does the sampling distribution/phase distribution
influence the eigenvalue distributions?
I
What can we say if we have a combination of independent
matrices on the form (1)?
I
Do we have spectral separation?
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Application 1: Signal reconstruction from irregular samples
I
I
I
I
Assume that we have a network composed of m wireless
sensors, which measure a number of observables (such as
pressure, temperature, coordinates, time).
The sensors are randomly distributed over the geographical
region of interest. This distribution may be uniform if the
terrain is regular, but may not be due to obstacles or terrain
asperities.
Sensors may enter low-operational mode to save energy, in
which their sampling rates are reduced. Sampling may not be
uniform.
The samples are corrupted by noise, such as round-off errors in
the sensing devices.
If there is only one observable, the
we sample is
Pfunction
int
approximated by a Fourier series
an e , where the an are the
Fourier coefficients.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
If there are L samples, their values are approximated by the vector
(we truncate the Fourier series to N terms)
!
X
X
X
int1
int2
intL
an e ,
an e , · · · ,
an e
n
n



= (a0 , a1 , . . . , aN−1 ) 

1

e it2
..
.
···
···
..
.
e itL
..
.
e i(N−1)t2
···
e i(N−1)tL


,

1
1
e it1
..
.
e i(N−1)t1
I
We recognize the Vandermonde matrix.
I
Knowledge about the spectrum of the Vandermonde matrix
will enable us to infer on the spectral characteristics of a
collection of irregular samples.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Application 2: A wireless network
L mobiles interacting with a basestation equipped with N receiving
antennas. The received signal at the basestation can be
approximated as above as
VP1/2 s + n,
where
I
V is an N × L Vandermonde matrix,
I
s is the L × 1 transmitted vector,
I
n is additive noise,
I
P is a diagonal matrix where the diagonal entries are the
powers which the different users use to transmit (higher power
for higher distance to the basestation).
Knowledge about the spectrum of the Vandermonde matrix enables
us to estimate the number of users and the corresponding powers.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Algebraic result for Vandermonde matrices [2]
Theorem
Let {Vi }i∈I , {Vj }j∈J be independent Vandermonde matrices, with
arbitrary phase distributions {ωi }i∈I and {ωj }j∈J , respectively, with
continuous density.
I
Let AI be the algebra generated by {(Vi1 )H Vi2 }i1 ,i2 ∈I .
I
Let AJ be the algebra generated by {(Vj1 )H Vj2 }j1 ,j2 ∈J .
We have that any mixed moment
lim E [tr (ai1 aj1 ai2 aj2 · · · ain ajn )] with aik ∈ AI , ajk ∈ AJ ,
N→∞
(2)
depends only on individual moments of the form
lim E [tr(a)] with a ∈ AI ,
N→∞
lim E [tr(a)] with a ∈ AJ .
N→∞
Øyvind Ryan
(3)
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Freeness: another computational rule for mixed moments
The previous theorem states that there exists a computational rule
for computing mixed moments of Vandermonde matrices. A similar,
but different rule exists for Gaussian matrices, and is governed by:
Definition
A family of unital ∗-subalgebras {Ai }i∈I is called a free family if


aj ∈ Aij


⇒ φ(a1 · · · an ) = 0. (4)
i1 6= i2 , i2 6= i3 , · · · , in−1 6= in


φ(a1 ) = φ(a2 ) = · · · = φ(an ) = 0
I
I
Spectral separation. E[tr(·)] replaced with general φ.
Defining σ as the partition where k ∼ l if and only if ik = il ,
the same formula for the mixed moment applies for any
a1 · · · an giving rise to σ. Is in this way a particularly nice type
of spectral separation.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
The general case
I
Cases of spectral separation seem to be easier to find when we
let the matrix sizes grow to infinity. For instance, it seems
unlikely that the result for Vandermonde matrices also holds in
the finite regime.
I
The only known intersting cases of spectral separation in the
finite regime are Gaussian matrices [3].
I
In the large n-limit, the question of spectral separation is
coupled with finding what modes of convergence apply for the
random matrices. Do we have almost sure convergence?
I
When is the computational rule for computing mixed moments
as simple as that of freeness.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Sketch of proof for Theorem 1
We need to compute
i
h lim E tr VkH1 Vk2 · · · VkH2n−1 Vk2n .
N→∞
I
I
I
I
I
Define σ ∈ P(2n) defined by r ∼σ s if and only if ωkr = ωks ,
let σj be the block of σ where ωki = ωj for i ∈ σj .
For π ∈ P(n), define ρ(π) ∈ P(2n) as the partition in P(2n)
generated by the relations:
bk/2c + 1 ∼π bl /2c + 1 and
k ∼ρ(π) l if
k ∼σ1 l
where σ1 defined by r ∼σ1 s if and only if Vkr = Vks .
B(n) ⊂ P(n) be defined as in [2],
write ρ(π) ∨ [0, 1]n = {ρ1 , ..., ρr (π) }, with each ρi ≥ [0, 1]kρi k/2
(r (π) the number of blocks). Can be written so in a unique
way.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
By carefully collecting terms we obtain in the limit
P
Qr (π) R Q
|ρi ∩σj | dx,
|ρ|−1
j pωj (x)
π∈B(n) Kρ,u (2π)
i=1
(5)
I
Here pω is the density of the phase distribution ω.
I
The Kρ,u are called Vandermonde mixed moment expansion
coefficients
I
When each VkH2j−1 Vk2j is in either AI or AJ , in each integral
RQ
|ρi ∩σj | dx, all ω are either contained in {ω }
j
i i∈I , or
j pωj (x)
in {ωj }j∈J ,
I
Each such integral can be written in terms of moments from
either AI or AJ , showing that we have spectral separation.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Due to (5), the moments of Vandermonde matrices are in the large
n-limit essentially determined from
Z 2π
k−1
Ik,ω = (2π)
pω (x)k dx.
(6)
0
I
Reduces the dimensionality of the problem.
I
In the finite regime, the moments are probably not uniquely
determined from such simple quantities.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Vandermonde mixed moment expansion coefficients
I
I
I
Write ρ(π) = {W1 , ..., W|ρ(π)| },
write Wj = Wj· ∪ WjH , with Wj· the even elements of Wj (the
V-terms), WjH the odd elements of Wj (the VH -terms).
Form the |ρ(π)| equations
X
X
x(k+1)/2+1 =
xk/2+1
(7)
k∈WrH
I
I
I
k∈Wr·
in n variables x1 , . . . , xn .
Kρ,u is the volume of the solution set to (7), when all xi are
constrained to [0, 1].
Kρ,u can be found with Fourier-Motzkin elmimination, and
always computes to a rational number in [0, 1].
Matrices such as Hankel and Toeplitz matrices also have
asymptotic eigenvalue distributions which can be determined
from such quantities.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Random Matrix Library [4]
1. Implementation with support for many types of matrices:
Vandermonde matrices, Gaussian matrices, Toeplitz matrices,
Hankel matrices e.t.c.
2. Can generate symbolic formulas for several convolution
operations, in cases where spectral separation applies,
3. Can compute the convolution with a given set of moments
numerically, as would be needed in real-time applications.
4. Can perform deconvolution, where this is possible, to infer on
the parameters in an underlying model.
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Further work
I
Find as general criteria as possible for when spectral separation
is possible.
I
Support for more matrices in the Random Matrix Library.
I
Optimization of the methods in the Random Matrix Library.
I
This talk is available at
http://folk.uio.no/oyvindry/talks.shtml
I
My publications are listed at
http://folk.uio.no/oyvindry/publications.shtml
THANK YOU!
Øyvind Ryan
Some particular types of random matrices and their applictio
The CMA Seminar, 25. November 2010
On some particular types of random matrices and their applicti
Ø. Ryan and M. Debbah, “Asymptotic behaviour of random
Vandermonde matrices with entries on the unit circle,” IEEE
Trans. on Information Theory, vol. 55, no. 7, pp. 3115–3148,
2009.
——, “Convolution operations arising from Vandermonde
matrices,” Submitted to IEEE Trans. on Information Theory,
2009.
Ø. Ryan, A. Masucci, S. Yang, and M. Debbah, “Finite
dimensional statistical inference,” To appear in IEEE Trans. on
Information Theory, 2009.
Ø. Ryan, Documentation for the Random Matrix Library, 2009,
http://folk.uio.no/oyvindry/rmt/doc.pdf.
Øyvind Ryan
Some particular types of random matrices and their applictio