MATH 423 Linear Algebra II Lecture 27: Norms and inner products. Euclidean structure In addition to the linear structure (addition and scaling), space R3 carries the Euclidean structure: • length of a vector: |x|, • angle between vectors: θ, • dot product: x · y = |x| |y| cos θ. C y θ A x B Euclidean structure Properties of vector length: |x| ≥ 0, |x| = 0 only if x = 0 (positivity) |r x| = |r | |x| (homogeneity) |x + y| ≤ |x| + |y| (triangle inequality) y x x+y Euclidean structure Properties of dot product: x · x ≥ 0, x · x = 0 only if x = 0 (positivity) x·y =y·x (symmetry) (x + y) · z = x · z + y · z (distributive law) (r x) · y = r (x · y) (homogeneity) Relations between lengths and dot products: √ • |x| = x · x • x · y = 12 |x|2 + |y|2 − |x − y|2 C x−y y θ A x B Law of cosines: |x − y|2 = |x|2 + |y|2 − 2|x| |y| cos θ |x − y|2 = |x|2 + |y|2 − 2 x·y Norm The notion of norm generalizes the notion of length of a vector in Rn . Definition. Let V be a vector space over F, where F = R or C. A function α : V → R is called a norm on V if it has the following properties: (i) α(x) ≥ 0, α(x) = 0 only for x = 0 (positivity) (ii) α(r x) = |r | α(x) for all r ∈ F (homogeneity) (iii) α(x + y) ≤ α(x) + α(y) (triangle inequality) Notation. The norm of a vector x ∈ V is usually denoted kxk. Different norms on V are distinguished by subscripts, e.g., kxk1 and kxk2 . Examples. V = Rn , x = (x1 , x2 , . . . , xn ) ∈ Rn . • kxk∞ = max(|x1 |, |x2 |, . . . , |xn |). Positivity and homogeneity are obvious. Let x = (x1 , . . . , xn ) and y = (y1 , . . . , yn ). Then x + y = (x1 + y1 , . . . , xn + yn ). |xi + yi | ≤ |xi | + |yi | ≤ maxj |xj | + maxj |yj | =⇒ maxj |xj + yj | ≤ maxj |xj | + maxj |yj | =⇒ kx + yk∞ ≤ kxk∞ + kyk∞ . • kxk1 = |x1 | + |x2 | + · · · + |xn |. Positivity and homogeneity are obvious. The triangle inequality: |xi + yi | ≤ |xi | + |yi | P P P |x | + |x + y | ≤ =⇒ j j j j |yj | j j Examples. V = Rn , x = (x1 , x2 , . . . , xn ) ∈ Rn . 1/p • kxkp = |x1 |p + |x2 |p + · · · + |xn |p , p > 0. Remark. kxk2 = Euclidean length of x. Theorem kxkp is a norm on Rn for any p ≥ 1. Positivity and homogeneity are still obvious (and hold for any p > 0). The triangle inequality for p ≥ 1 is known as the Minkowski inequality: 1/p |x1 + y1 |p + |x2 + y2 |p + · · · + |xn + yn |p ≤ 1/p 1/p . + |y1 |p + · · · + |yn |p ≤ |x1 |p + · · · + |xn |p Normed vector space Definition. A normed vector space is a vector space endowed with a norm. The norm defines a distance function on the normed vector space: dist(x, y) = kx − yk. Then we say that a sequence x1 , x2 , . . . converges to a vector x if dist(x, xn ) → 0 as n → ∞. Also, we say that a vector x is a good approximation of a vector x0 if dist(x, x0 ) is small. Unit circle: kxk = 1 kxk = (x12 + x22 )1/2 1/2 kxk = 21 x12 + x22 kxk = |x1 | + |x2 | kxk = max(|x1 |, |x2 |) black green blue red Theorem 1 Let k · k be an arbitrary norm on Rn . Then there exist positive constants c1 , c2 such that c1 kxk2 ≤ kxk ≤ c2 kxk2 for any x ∈ Rn . Idea of the proof: One shows that the function f (x) = kxk is continuous on Rn (in the usual sense). Since the Euclidean unit sphere S n−1 (given by kxk2 = 1) is a closed bounded set, the function f attains its minimum and maximum values on S n−1 . These are c1 and c2 in the above inequalities. Theorem 2 Let k · k1 and k · k2 be arbitrary norms on a finite-dimensional vector space V . Then there exist positive constants c1 , c2 such that c1 kxk2 ≤ kxk1 ≤ c2 kxk2 for any x ∈ V . Corollary A sequence x1 , x2 , . . . of vectors from V converges to a vector x ∈ V with respect to the distance induced by the norm k · k1 if and only if it converges to x with respect to the distance induced by k · k2 . Examples. V = C [a, b], f : [a, b] → R. • kf k∞ = max |f (x)|. a≤x≤b b • kf k1 = Z • kf kp = Z |f (x)| dx. a a b p |f (x)| dx 1/p , p > 0. Theorem kf kp is a norm on C [a, b] for any p ≥ 1. Inner product: real vector space The notion of inner product generalizes the notion of dot product of vectors in R3 . Definition. Let V be a real vector space. A function β : V × V → R, usually denoted β(x, y) = hx, yi, is called an inner product on V if it is positive, symmetric, and bilinear. That is, if (i) hx, xi ≥ 0, hx, xi = 0 only for x = 0 (positivity) (ii) hx, yi = hy, xi (symmetry) (iii) hr x, yi = r hx, yi (homogeneity) (iv) hx + y, zi = hx, zi + hy, zi (distributive law) An inner product space is a vector space endowed with an inner product. Examples. V = Rn . • hx, yi = x · y = x1 y1 + x2 y2 + · · · + xn yn . • hx, yi = d1 x1 y1 + d2 x2 y2 + · · · + dn xn yn , where d1 , d2 , . . . , dn > 0. • hx, yi = (Dx) · (Dy), where D is an invertible n×n matrix. Remarks. (a) Invertibility of D is necessary to show that hx, xi = 0 =⇒ x = 0. (b) The second example is a particular case of the 1/2 1/2 1/2 third one when D = diag(d1 , d2 , . . . , dn ). Example. V = Mm,n (R), space of m×n matrices. • hA, Bi = trace (AB t ). If A = (aij ) and B = (bij ), then hA, Bi = m P n P i=1 j=1 Examples. V = C [a, b]. Z b f (x)g (x) dx. • hf , g i = a • hf , g i = Z b f (x)g (x)w (x) dx, a where w is bounded, piecewise continuous, and w > 0 everywhere on [a, b]. w is called the weight function. aij bij .