Advances in Random Matrix Theory: Let there be tools Barcelona, Spain
advertisement
Advances in Random Matrix Theory:
Let there be tools
Alan Edelman
Brian Sutton, Plamen Koev,
Ioana Dumitriu,
Raj Rao and others
MIT: Dept of Mathematics,
Computer Science AI Laboratories
World Congress, Bernoulli Society
Barcelona, Spain
2/1/2005
Wednesday July 28, 2004
1
Message
Ingredient: Take Any important mathematics
Then Randomize!
This will have many applications!
We can’t keep this in the hands of specialists
anymore: Need tools!
2/1/2005
2
Tools
So
many applications …
Random matrix theory: catalyst for 21st century
special functions, analytical techniques, statistical
techniques
In addition to mathematics and papers
Need
tools for the novice!
Need tools for the engineers!
Need tools for the specialists!
2/1/2005
3
Themes of this talk
Tools
for general beta
What
is beta? Think of it as a measure of (inverse)
volatility in “classical” random matrices.
Tools
for complicated derived random matrices
Tools for numerical computation and simulation
2/1/2005
4
The
Wigner’s Semi-Circle
classical & most famous rand eig theorem
Let S = random symmetric Gaussian
MATLAB:
A=randn(n); S=(A+A’)/2;
S
known as the Gaussian Orthogonal Ensemble
Normalized eigenvalue histogram is a semi-circle
n=20;
%matrix
size, samples,
sample
s=30000;
Precised=.05;
statements
require
n→∞
etc.dist
e=[];
%gather up eigenvalues
im=1; %imaginary(1) or real(0)
for i=1:s,
a=randn(n)+im*sqrt(-1)*randn(n);a=(a+a')/(2*sqrt(2*n*(im+1)));
v=eig(a)'; e=[e v];
end
hold off; [m x]=hist(e,-1.5:d:1.5); bar(x,m*pi/(2*d*n*s));
axis('square'); axis([-1.5 1.5 -1 2]); hold on;
t=-1:.01:1; plot(t,sqrt(1-t.^2),'r');
5
sym matrix to tridiagonal form
G χ6
χ6 G χ5
χ5 G χ4
χ4 G χ3
Same eigenvalue distribution
as
GOE:
G
χ3
χ2
2
O(n) storage !! O(n ) compute
χ2 G χ1
χ1 G
2/1/2005
6
General beta
G χ6β beta:
1: reals 2: complexes 4: quaternions
χ6β G χ5β
χ5β GBidiagonal
χ4β Version corresponds
matrices of Statistics
G χ3β
χ4βTo Wishart
χ3β G χ2β
χ2β G χβ
χβ G
2/1/2005
7
Tools
Motivation: A condition number problem
Jack & Hypergeometric of Matrix Argument
MOPS: Ioana Dumitriu’s talk
The Polynomial Method
The tridiagonal numerical 109 trick
2/1/2005
8
Tools
Motivation: A condition number problem
Jack & Hypergeometric of Matrix Argument
MOPS: Ioana Dumitriu’s talk
The Polynomial Method
The tridiagonal numerical 109 trick
2/1/2005
9
κ(A)
Numerical Analysis:
Condition Numbers
= “condition number of A”
If A=UΣV’ is the svd, then κ(A) = σmax/σmin .
Alternatively, κ(A) = √λ max (A’A)/√λ min (A’A)
One number that measures digits lost in finite
precision and general matrix “badness”
Small=good
Large=bad
The
2/1/2005
condition of a random matrix???
10
Von Neumann & co.
Solve Ax=b via x= (A’A) -1A’ b
M ≈A-1
Matrix Residual: ||AM-I||2
||AM-I||2< 200κ2 n ε
≈
How should we estimate κ?
Assume, as a model, that the elements of A are
independent standard normals!
2/1/2005
11
Von Neumann & co. estimates
(1947-1951)
“For a ‘random matrix’ of order n the expectation value
has been shown to be about X
n” ∞
Goldstine, von Neumann
“… we choose two different values of κ, namely n and
√10n” P(κ<n)
≈0.02
Bargmann, Montgomery, vN
P(κ< √10n)≈0.44
“With a probability ~1 … κ < 10n”
P(κ<10n) ≈0.80
2/1/2005
Goldstine, von Neumann
12
Random cond numbers, n→∞
Distribution of κ/n
2 x + 4 −2 / x −2 / x 2
y=
e
3
x
2/1/2005
Experiment with n=200
13
Finite n
n=10
n=25
n=50
n=100
2/1/2005
14
Condition Number Distributions
P(κ/n > x) ≈ 2/x
P(κ/n2 > x) ≈ 4/x2
Real n x n, nÆ∞
Generalizations:
•β: 1=real, 2=complex
Complex n x n, nÆ∞
•finite matrices
•rectangular: m x n
2/1/2005
15
Condition Number Distributions
P(κ/n > x) ≈ 2/x
P(κ/n2 > x) ≈ 4/x2
Square, nÆ∞: P(κ/nβ > x) ≈ (2ββ-1/Γ(β))/xβ (All Betas!!)
Real
n x n, nÆ∞
General
Formula:
P(κ>x) ~Cµ/x β(n-m+1),
where µ= β(n-m+1)/2th moment of the largest eigenvalue of
Wm-1,n+1 (β)
and C is a known geometrical constant.
Complex n x n, nÆ∞
Density for the largest eig of W is known in terms of
1F1((β/2)(n+1), ((β/2)(n+m-1); -(x/2)Im-1) from which µ is available
Tracy-Widom law applies probably all beta for large m,n.
2/1/2005shows at least beta=1,2.
Johnstone
16
Tools
Motivation: A condition number problem
Jack & Hypergeometric of Matrix Argument
MOPS: Ioana Dumitriu’s talk
The Polynomial Method
The tridiagonal numerical 109 trick
2/1/2005
17
Multivariate Orthogonal Polynomials
&
Hypergeometrics of Matrix Argument
Ioana
Dumitriu’s talk
The important special functions of the 21st century
Begin with w(x) on I
pκ(x)pλ(x) ∆(x)β ∏i w(xi)dxi = δκλ
Jack Polynomials orthogonal for w=1
on the unit circle. Analogs of xm
∫
2/1/2005
18
Multivariate Hypergeometric Functions
2/1/2005
19
Multivariate Hypergeometric Functions
2/1/2005
20
Wishart (Laguerre) Matrices!
2/1/2005
21
Plamen’s clever idea
2/1/2005
22
Tools
Motivation: A condition number problem
Jack & Hypergeometric of Matrix Argument
MOPS: Ioana Dumitriu’s talk
The Polynomial Method
The tridiagonal numerical 109 trick
2/1/2005
23
Mops (Dumitriu etc.) Symbolic
2/1/2005
24
Symbolic MOPS applications
A=randn(n); S=(A+A’)/2; trace(S^4)
det(S^3)
2/1/2005
25
Symbolic MOPS applications
β=3; hist(eig(S))
2/1/2005
26
Smallest eigenvalue statistics
A=randn(m,n); hist(min(svd(A).^2))
2/1/2005
27
Tools
Motivation: A condition number problem
Jack & Hypergeometric of Matrix Argument
MOPS: Ioana Dumitriu’s talk
The Polynomial Method -- Raj!
The tridiagonal numerical 109 trick
2/1/2005
28
RM Tool – Raj!
Courtesy of the Polynomial Method
2/1/2005
29
2/1/2005
30
The Riemann Zeta Function
1
ζ( s ) =
Γ (s)
∞
∫
0
s −1
u
du =
u
e −1
∞
∑
k =1
1
ks
On the real line with x>1, for example
2
1 1 1
π
ζ( 2 ) = 1 + 2 + 2 + 2 + … =
2 3 4
6
May be analytically extended to the complex plane,
with singularity only at x=1.
2/1/2005
31
The Riemann Hypothesis
∞
x −1
-1
0
∞
u
1
1
du
=
ζ( x ) =
∑
u
x
∫
Γ( x ) 0 e − 1
k =1 k
-3
-2
½ 1
2
3
All nontrivial roots of ζ(x) satisfy Re(x)=1/2.
(Trivial roots at negative even integers.)
2/1/2005
32
=.5+i*
|ζ(x)| along Re(x)=1/2 Zeros
14.134725142
21.022039639
The Riemann Hypothesis
25.010857580
∞
x −1
∞
30.424876126
u
1
1
32.935061588
du
=
ζ( x ) =
∑
u
x
∫
37.586178159
Γ( x ) 0 e − 1 40.918719012
k =1 k
-3
-2
-1
0
43.327073281
48.005150881
49.773832478
½ 152.970321478
2
3
56.446247697
59.347044003
All nontrivial roots of ζ(x) satisfy Re(x)=1/2.
(Trivial roots at negative even integers.)
2/1/2005
33
Computation of Zeros
Odlyzko’s
fantastic computation of 10^k+1
through 10^k+10,000 for k=12,21,22.
See http://www.research.att.com/~amo/zeta_tables/
Spacings behave like the eigenvalues of
A=randn(n)+i*randn(n); S=(A+A’)/2;
2/1/2005
34
Nearest Neighbor Spacings &
Pairwise Correlation Functions
2/1/2005
35
Painlevé Equations
2/1/2005
36
Spacings
Take a large collection of consecutive zeros/eigenvalues.
Normalize so that average spacing = 1.
Spacing Function = Histogram of consecutive differences
(the (k+1)st – the kth)
Pairwise Correlation Function = Histogram of all possible
differences (the kth – the jth)
Conjecture: These functions are the same for random
matrices and Riemann zeta
2/1/2005
37
Tools
Motivation: A condition number problem
Jack & Hypergeometric of Matrix Argument
MOPS: Ioana Dumitriu’s talk
The Polynomial Method
The tridiagonal numerical 109 trick
2/1/2005
38
Everyone’s Favorite Tridiagonal
-2
1
1
n2
1
-2
…
1
…
…
…
…
1
1
-2
d2
dx2
2/1/2005
39
Everyone’s Favorite Tridiagonal
-2
1
1
n2
1
-2
…
G
1
…
…
…
…
1
d2
dx2
2/1/2005
1
+(βn)1/2
G
1
-2
G
+
dW
β1/2
40
Stochastic Operator Limit
2
d
− x +
2
dx
2
dW ,
β
⎛ N(0,2) χ (n −1)β
⎜
⎜ χ (n −1)β N(0,2) χ (n − 2)β
1 ⎜
β
…
Hn ~
…
⎜
2 nβ
⎜
χ 2β
⎜
⎝
H
2/1/2005
β
n
≈ H
∞
n
+
⎞
⎟
⎟
⎟
…
⎟,
N(0,2)
χβ ⎟
⎟
χβ
N(0,2) ⎠
2
G
β
n
,
41
Largest Eigenvalue Plots
2/1/2005
42
MATLAB
beta=1; n=1e9; opts.disp=0;opts.issym=1;
alpha=10; k=round(alpha*n^(1/3)); % cutoff parameters
d=sqrt(chi2rnd( beta*(n:-1:(n-k-1))))';
H=spdiags( d,1,k,k)+spdiags(randn(k,1),0,k,k);
H=(H+H')/sqrt(4*n*beta);
eigs(H,1,1,opts)
2/1/2005
43
Tricks to get O(n9) speedup
• Sparse matrix storage (Only O(n) storage is used)
• Tridiagonal Ensemble Formulas (Any beta is available due to the
tridiagonal ensemble)
• The Lanczos Algorithm for Eigenvalue Computation ( This allows
the computation of the extreme eigenvalue faster than typical
general purpose eigensolvers.)
• The shift-and-invert accelerator to Lanczos and Arnoldi (Since we
know the eigenvalues are near 1, we can accelerate the
convergence of the largest eigenvalue)
• The ARPACK software package as made available seamlessly in
MATLAB (The Arnoldi package contains state of the art data
structures and numerical choices.)
• The observation that if k = 10n1/3 , then the largest eigenvalue is
determined numerically by the top k × k segment of n. (This is an
interesting mathematical statement related to the decay of the Airy
function.)
2/1/2005
44
Level Densities
2/1/2005
45
Open Problems
The distribution for general beta
Seems to be governed by a convection-diffusion equation
2/1/2005
46
Random matrix tools!
2/1/2005
47
