advertisement

Applied Geometrical Matrix Computations Alan Edelman Dept of Mathematics: MIT MIT Laboratory for Computer Science Householder Symposium XV June 21, 2002 Outline • Geometrical Matrix Computations • Illustration with 2x2 matrices: • Excursions into eigenland (or why tangency and curvature matter!!) • Where do matrix factorizations come from? • Application to Color Science • Matrix Animations Geometrical Matrix Computations Working definition: •Concerns geometry of matrix space (n2 dimensions rather than n) •Involves numerical computation (probably MATLAB) •Relates to an NLA problem Some Other GMC People Absil, Demmel, Elmroth, Huhtanen, Kagstrom, Kahan, Lippert, Ma, Mahony, Malyshev, Sepulchre, Tisseur, Trefethen, Van Dooren Vector Space Diagrams •Points are vectors (not matrices!) •Geometric relationships for vectors, subspaces, and linear transformations Outline • Geometrical Matrix Computations • Illustration with 2x2 matrices: • Excursions into eigenland (or why tangency and curvature matter!!) • Where do matrix factorizations come from? • Application to Color Science • Matrix Animations x M = z Eigenland (in 2d) Isoeig surfaces are hyperbolas 2 x z 2 z x -z -x The Eigenvalue Map x M = z -z -x 2 x z zz 2 cos z x Eigvec Angle xx /2 Zero Matrix 0 0 The Eigenvalue Map x M = z -z -x 2 x z zz 2 cos z x Eigvec Angle xx •M /2 Zero Matrix 0 0 •M The Eigenvalue Map x M = z -z -x zz 2 x z 2 cos z x Eigvec Angle xx Uniformly /2 •M Zero Matrix 0 0 ?•M Pseudospectra (Trefethen) ( A) {zz C ::zz eig( A E ), with ||E||2 } “z is an eigenvalue of a matrix near A” {z C : min (zI - A) } Pseudoportraits = pictures of contours of z Pseudoportraits 1 1 1 1 0 A= -1 0 1 -1 1 1 1 1 1 1 0 0 -1 1 1 0 0 0 -1 1 Random Points pseudospectra & geometry matrix space C eig (w/singularity) spectral portrait n2 A XX 1 C X n2 Outline • Geometrical Matrix Computations • Illustration with 2x2 matrices: • Excursions into eigenland (or why tangency and curvature matter!!) • Where do matrix factorizations come from? • Application to Color Science • Matrix Animations Circle/Hyperbola Tangency = High Density * * * * * * * * * eigenvalue have eigenvalue distributions with 3 2 4 spikes. eigenvalue frequency frequency frequency •Circles tangent to 2 4 3 hyperbolas… eigenvalue Radius of Curvature = Highest Density •Circles are tangent to 3 hyperbolas when two tangency points collide * •The circle also shares a radius of curvature with the hyperbola at this point * eigenvalue frequency frequency frequency •This is even better than tangency, which means a higher spike * eigenvalue eigenvalue Outline • Geometrical Matrix Computations • Illustration with 2x2 matrices: • Excursions into eigenland (or why tangency and curvature matter!!) • Where do matrix factorizations come from? • Application to Color Science • Matrix Animations Where do Matrix Factorizations Come From? Classical Answer: A=UV’ Representation Theory of Semisimple Groups Semisimple group recipe •Nicely links factorizations •Three Examples Nonsingulars Unitary SVD UeiV’ SPD Eigen UeiU’ One more example Hyperbolic Svd as in last talk Group = SO(p,q) ( XJ=JX) Orthogonal CS decomp Essentially Sym Orth Matrix Factorizations Where can we look for new factorizations? • The Mathematics Literature – Lie Algebra: Cartan, Iwasawa, Bruhat – Representation Theory: Quivers • Nearness Problems • Applications – Engineering: A factorization is useful if someone can use it – Mathematics: The useful factorizations are characterized by an abstract criterion Ideas to Generalize 1: Cartan Decomposition E = (antisymmetric) + (symmetric) expm M= Non-singular expm Q Orthogonal expm * S Pos Definite [polar] Ideas to Generalize 1: Cartan Decomposition E = (antisymmetric) + (symmetric) expm M= Non-singular expm Q Orthogonal expm * S Pos Definite 2:KAK Decomposition Positive Diagonals = Maximal Group M=UV’ Conjugates give S=QQ’ [polar] Ideas to Generalize 1: Cartan Decomposition E = (antisymmetric) + (symmetric) expm M= Non-singular expm Q Orthogonal expm * S [polar] Pos Definite 2:KAK Decomposition Positive Diagonals = Maximal Group M=UV’ Conjugates give S=QQ’ 3:Iwasawa, Bruhat Above not unique at I. Gives M=LU, other permutations, totally positive, etc Ideas to Generalize 1: Cartan Decomposition E = (antisymmetric) + (symmetric) expm M= Non-singular expm Q Orthogonal expm * S [polar] Pos Definite 2:KAK Decomposition Positive Diagonals = Maximal Group M=UV’ Conjugates give S=QQ’ 3:Iwasawa, Bruhat Above not unique at I. Gives M=LU, other permutations, totally positive, etc 4:Eigenvalue, Jordan Schur Step 1:Cartan Decomposition •Group: non-singular matrices •Involution: ((M))=M (M1M2)= (M1)(M2) (M)=M-T •Fixed Points (M)=M are a group K K = orthogonal matrices •Near I M = (antisymmetric) + (symmetric) •Cartan: expm M = QS (S>0) (polar) Step 1:Cartan Decomposition (U/O) •Group: unitary matrices •Near I M = (antisymmetric) + (i*symmetric) •Cartan: M= (real orth)(unitary symmetric) Step 2:KAK Decomposition P = sym pos def • A = biggest group inside P (abelian) e.g. diagonal > 0, or conjugates UU’ (fix U) •KAK M=UV’ •P = union of conjugates S=QQ’ Step 2:KAK Decomposition (U/Q) P = unitary symmetric • A = biggest group inside P (abelian) e.g. diagonals (ei) or conjugates •KAK M=UeiV’ (U, V real orthogonal) •P = union of conjugates S=QeiQ’ (Q real orthogonal) Step 2:KAK Decomposition (On/Op X Oq ) C P = matrices orthogonally similar to ( -S • A = biggest group inside P (abelian) C S e.g. =( -S C ) or conjugates •KAK The CS Decomposition S C ) Missing •The constructible decompositions Tridiagonalization, Bidiagonalization •The NNMF (Lee, Seung 1999) •V WH Input: Vij>0 Output: Wij>0 Hij>0 (low rank) Algorithm: H H .* (W’V)./(W’WH) W W .* (VH’)./(WHH’) Original Application: Eigenfaces Another Example: Color Science Outline • Geometrical Matrix Computations • Illustration with 2x2 matrices: • Excursions into eigenland (or why tangency and curvature matter!!) • Where do matrix factorizations come from? • Application to Color Science • Matrix Animations Color Science: Light Spectra from film wavelength vs density Reds Greens Grays Blues Film Recording and measurements • Solid colors sent to film recorder, e.g. reds • Negative is produced: film appears as cyans • Negative sent through projector to spectrometer • Energy data at each wavelength Reds • Log ratio with no film (only bulb) film density = log(no film / with film) The Data • • • • Inputs (r,g,b) for 1r,g,b 10 scaled (1000 frames) Output Space: Densities at 400:3:700 nm’s Data Structure: 101 x 1000 matrix “A” Compute SVD(A) •Project onto best 3 space svd Three significant singular values index SVD Basis = no physical meaning The NNMF Basis = primary colors Outline • Geometrical Matrix Computations • Illustration with 2x2 matrices: • Excursions into eigenland (or why tangency and curvature matter!!) • Where do matrix factorizations come from? • Application to Color Science • Matrix Animations Singular 2x2 Matrices (by svd) cos _ sin cos sin A 0 sin cos sin cos A= ( cos cos -sin cos cos sin -sin sin and scales cone ) Torus! Torus Cone? All isoeig surfaces are translates of the =0 surface! hyperpolas and hyperboloids are cross sections! Bohemian Dome Linear Algebra with movies Horizontal A=QQT Vertical A=QQ Villarceau A=QR Hopf Fibration Challenges Incorporate 3d graphics tools directly into Matrix computations. Include geometry of matrix space. How should this look? Generalize everything and incorporate into software