Homogeneous equations, Linear independence 1. Homogeneous equations: Ex 1: Consider system: B" B" #B# œ! #B3 œ ! B$ œ ! B# Matrix equation: Ô " " Õ ! # ! " ! ×Ô B" × Ô ! × # B# œ ! œ 0Þ ØÕ " B$ Ø Õ ! Ø Ð3Ñ Homogeneous equation: Ex œ 0. At least one solution: x œ 0Þ Other solutions called nontrivial solutions. Theorem 1: A nontrivial solution of Ð$Ñ exists iff [if and only if] the system has at least one free variable in row echelon form. The same is true for any homogeneous system of equations. Proof: If there are no free variables, there is only one solution and that must be the trivial solution. Conversely, if there are free variables, then they can be non-zero, and there is a nontrivial solution. Ex 2: Reduce the system above: Ô " " Õ ! # ! " ! # " l !× l ! l !Ø as beforeÔ " ! Ä Õ! # ! " " ! ! l !× l ! l !Ø Ê B" #B# œ !à B# B$ œ !à ! œ !Þ Note that B$ œ free variable (non-pivot); hence general solution is B# œ B$ à B" œ #B# œ #B$ Þ Ô B" × Ô #B$ × Ô # × B$ x œ B# œ œ B$ " ß Õ B$ Ø Õ B$ Ø Õ " Ø Parametric vector form of solution. B$ arbitrary: straight line - Theorem 2: A homogeneous system always has a nontrivial solution if the number of equations is less than the number of unknowns. Pf: If we perform a Gaussian elimination on the system, then the reduced augmented matrix has the form: Ô" Ö! Ö Ö! Ö 0 Õ +"# ! ! 0 +"$ " ! 0 ã á +#% " 0 á +$& 0 á á l !× l !Ù Ù l ãÙ Ù l ã l !Ø with the remaining rows zeroes on the left side. If the number of equations is less than the number of unknowns, then not every column can have a 1 in it, so there are free variables. By previous theorem, there are nontrivial solutions. 1. Inhomogeneous equations: [we should briefly mention the relationship between homogeneous and inhomogeneous equations:] Consider general system: Ex œ bÞ (1)(1) Suppose p is a particular solution of (1), so Ep œ b. If x is any other solution of (1), we still have Ex œ b. Subtracting the two equations: Ex E p œ 0 Ê EÐx pÑ œ 0Þ So v2 œ x p satisfies the homogeneous equation. Generally: Theorem 1: M0 p is a particular solution of (1), then for any other solution x, we have that v2 œ x p solves the homogeneous equation (i.e., with b œ 0). Thus every solution x of (1) can be written x œ p v2, where v2 is a solution of the homogeneous equation. 2. Application: Network flows Traffic pattern at Drummond Square: Quantities in cars/min. What are the flows on the inside streets? One equation for each node: B" B$ B% œ %! B" B# œ #!! B# B$ B& œ "!! B % B& œ '! Ô " Ö " Ö ! Õ ! ! " " ! " ! 1 ! " ! ! " ! ! " " l l l l %! × #!! Ù Ù "!! '! Ø Ô" Ö! Ö ! Õ! ! " " ! " " " " 1 ! ! " ! l %! × ! l "'! Ù Ù " l "!! " l '! Ø Ô" Ö! Ö ! Õ! ! " " ! " " 1 ! " " ! " ! ! " " l l l l %! × "'! Ù Ù "!! '! Ø Ô" Ö! Ö ! Õ! ! " ! ! " " 0 ! " " " " ! ! " " l l l l %! × "'! Ù Ù '! '! Ø ! ! " " l l l l %! × "'! Ù Ù '! '! Ø Ô" Ö! Ö ! Õ! ! " ! ! " " ! ! " " " " Ô" Ö! Ö ! Õ! ! " ! ! " " ! ! ! ! " ! " " " ! l "!! × l "!! Ù Ù l '! l ! Ø So: B" œ "!! B$ B& B# œ "!! B$ B& B% œ '! B& ß where B$ ß B& are free. Constraint: if for example all flows have to be positive; then we require B3 ! for all 3. Therefore: B$ ß B& ! "!! Ÿ B$ B& Ÿ "!! B& Ÿ '! This corresponds to a region in the B$ ß B& plane - can be plotted if desired. If they closed off road B$ and B& ß then we have B$ œ B& œ !, so that B" œ 100ß B# œ "00, B% œ '! note that then traffic flow becomes uniquely determined. Definition 1: A collection of vectors v" ß v# ß á ß v8 is linearly independent if no vector in the collection is a linear combination of the others. Equivalently, Definition 2: A collection of vectors v" ß á ß v8 is linearly independent if the only way we can have -" v" -# v# á -8 v8 œ 0 is if all of the -3 œ !Þ Equivalence of the definitions: Def 1 Ê Def 2 If no vector is a linear combination of the others, then if -" v " - # v # á - 8 v 8 œ 0 we will show that -" ß á ß -8 have also to be 0. Proof: Suppose not (for contradiction). Without loss of generality, assume -" Á ! (proof works same way otherwise). Then we have: v" œ -# Î-" v# á -8 Î-" v8 ß contradicting that no vector is a combination of the others. Thus the -3 all have to be 0 as desired. Note: If W# is a collection of vectors and W" is a subcollection of W2 ß then If W# is linearly independent Ê no vector in W# is a linear combination of the others Ê no vector in W" is a linear combination of the others (since every vector in W" is also in W# ) Ê W" is linearly independent. Logically equivalent [contrapositive] If W" is linearly dependent (i.e., not independent) Ê W# is linearly dependent [These are stated more formally in the book as theorems.] Theorem 2: P/> W œ Öv" ß á ß v8 × be a collection of vectors in ‘. . Then W is linearly dependent if and only if one of the vectors v3 is a linear combination of the previous ones v" ß á ß v3" Þ Proof: ( Ê ) If W is linearly dependent, then there is a set of constnats -3 not all ! such that -" v" á -8 v8 œ 0. Let -5 be the last non-zero coefficient. Then the rest of the coefficients are zero, and -" v" -# v# á -5" v5" -5 v5 œ 0 Ä v5 œ -" Î-5 v" -# Î-5 v # á -5" Î-5 v5" i.e. one of the vectors is a linear combination of the previous ones. ( É ) Obvious. 3. Checking for linear independence: Example 2: Consider the vectors " " " ß v$ œ ” v " œ ” •ß v # œ ” • " " ! • Are they linearly independent? -" v " - # v # - $ v $ œ 0 Ê -" - # - $ œ ! -" - # œ ! Reduced matrix: " ”" " " Ä” " ! " # Ä” " ! " " " ! ! !• l l " " l l ! !• " "Î# l l ! !• Conclude: there are free variables. By theorem on homogeneous equations there is a nontrivial solution, so -3 need not be !. Thus not all -3 must be ! Ê not linearly independent. [note that if number of vectors is greater than the size of the vectors, this will always happen]. More generally thus: Theorem 3: In ‘8 , if we have more than 8 vectors, they cannot be linearly independent. From above we have: Algorithm: To check whether vectors are linearly independent, form a matrix with them as columns, and row reduce. (a) If reduced matrix has free variables (i.e., b a non-pivot column), then they are not independent. (b) If there are no free variables (i.e., there are no nonpivot columns), they are independent.