Transpose and Adjoint
Lecture 2
Orthogonal Vectors and Matrices, Norms
MIT 18.335J / 6.337J
Introduction to Numerical Methods
Per-Olof Persson (persson@mit.edu)
September 10, 2007
• For real A, the transpose of A is obtained by interchanging rows/columns




a11 a12


a
a
a
11
21
31
T



A=
a21 a22  =⇒ A =
a12 a22 a32
a31 a32
• The adjoint or hermitian conjugate also takes complex conjugate




a11 a12


a
a
a
11
21
31
∗



A=
a21 a22  =⇒ A =
a12 a22 a32
a31 a32
• If real A = AT , then A is symmetric. If A = A∗ , then A is hermitian.
1
2
Inner Product
In MATLAB
• Inner product of two column vectors x, y ∈ Cm
x∗ y =
m
X
Quantity
x i yi
kxk =
x∗ x =
i=1
• Angle α between x, y
cos α =
A.’
Transpose only
Adjoint A
A’
Transpose + complex conjugate
Inner product x∗ y
x’*y
dot(x,y)
Length kxk
sqrt(x’*x) or
norm(x)
∗
• Euclidean length of x
m
X
Comment
Transpose A
i=1
√
MATLAB Syntax
T
2
|xi |
!1/2
or
’* assumes column vectors
’* assumes column vector
x∗ y
kxkkyk
3
4
Orthogonal Vectors
Orthogonal and Unitary Matrices
• The vectors x, y ∈ Rm are orthogonal if
• A square matrix Q ∈ Cm×m is unitary (orthogonal in real case), if
x∗ y = 0
Q∗ = Q−1
• For unitary Q
• The sets of vectors X, Y are orthogonal if
every x
∈ X is orthogonal to every y ∈ Y
• Interpretation of unitary-times-vector product:
• A set of (nonzero) vectors S is orthogonal if
vectors pairwise orthogonal, i.e., for x, y
Q∗ Q = I , or qi∗ qj = δij
∈ S, x 6= y ⇒ x∗ y = 0
and orthonormal if, in addition,
x = Q∗ b = solution to Qx = b
= the vector of coefficients of the expansion of b
in the basis of columns of Q
every x
∈ S has kxk = 1
5
6
Vector Norms
Preservation of Geometry Structure
• A norm is a function k · k : Cm → R satisfying
• Inner product is preserved under multiplication by unitary Q
(Qx)∗ (Qy) = x∗ Q∗ Qy = x∗ y
(1) kxk ≥ 0, and kxk = 0 only if x = 0
(2) kx + yk ≤ kxk + kyk
• Therefore, lengths of vectors and angles between vectors are preserved
• A real orthogonal Q is either a rigid rotation or reflection
Rotation
• Example: The Euclidean length function
√
kxk2 = x∗ x
Reflection
Qu
Qv
v
Qv
v
Qu
u
(3) kαxk = |α| kxk
u
• k · k2 is a special case of the p-norms
!1/p
m
X
p
kxkp =
|xi |
(1 ≤ p < ∞)
i=1
7
8
Examples of Vector Norms
Verification of Norm Conditions
• Show that kxk1 =
kxk1 =
kxk2 =
m
X
i=1
(1)
|xi |
m
X
|xi |2
i=1
!1/2
i=1
|xi | is a norm
kxk1 = |x1 | + |x2 | + · · · + |xm | ≥ 0, equality only if x = 0
=
√
x∗ x
(2)
kx + yk1 =
kxk∞ = max |xi |
1≤i≤m
kxkW = kW xk2 =
Pm
m
X
i=1
|wi xi |2
!1/2
=
m
X
i=1
m
X
i=1
|xi + yi | ≤
|xi | +
m
X
i=1
m
X
i=1
|xi | + |yi |
|yi | = kxk1 + kyk1
(3)
m
X
kαxk1 =
i=1
|αxi | =
m
X
i=1
|α| |xi | = |α|
m
X
i=1
|xi | = |α| kxk1
9
10
Induced Matrix Norms
Examples of Matrix Norms
• For a matrix A ∈ Cm×n , the induced matrix norm is
kAk(m,n) = sup
x∈Cn
x6=0
kAxk(m)
=
kxk(n)
sup kAxk(m)
x∈Cn
kxk(n) =1
where k · k(n) and k · k(m) are given vector norms
kAk1 = max
1≤j≤n
kAk∞ = max
1≤i≤m
• The “maximum stretching” by A
kAkF =
kAk2 =
11
m
X
i=1
n
X
j=1
m X
n
X
i=1 j=1
p
|aij |
“maximum column sum”
|aij |
“maximum row sum”
2
|aij |
!1/2
λmax (A∗ A)
The Frobenius norm
More later
12
In MATLAB
Properties of Matrix Norms
• Bound on Matrix Product - Induced norms and Frobenius norm satisfy
kABk ≤ kAkkBk
but some matrix norms do not!
• Invariance under Unitary Multiplication - For A ∈ Cm×n and unitary
Q ∈ Cm×m , we have
kQAk2 = kAk2 ,
Proof. Since kQxk2
kQAkF = kAkF
= kxk2 (inner product is preserved), the first result
follows from the definition of induced norm. For the Frobenius norm,
p
tr((QA)∗ (QA)) =
p
= tr(A∗ A) = kAkF
kQAkF =
13
p
tr(A∗ Q∗ QA)
Quantity
MATLAB Syntax
kxk1
sum(abs(x)) or norm(x,1)
kxk2
sqrt(x’*x) or norm(x)
kxkp
sum(abs(x).ˆ p).ˆ (1/p) or norm(x,p)
kxk∞
max(abs(x)) or norm(x,inf)
kAk1
max(sum(abs(A),1)) or norm(A,1)
kAk2
norm(A)
kAk∞
max(sum(abs(A),2)) or norm(A,inf)
kAkF
sqrt(A(:)’*A(:)) or norm(A,’fro’)
14