Linear Algebra Study Guide
To help you prepare for midterms and the final, this study guide is already updated for
the whole semester…the idea is that if you go through the study guide on a regular basis
and make sure you understand the concepts as we go along, and keep going over it to
refresh your memory of key ideas throughout the course, you’ll be better able to pinpoint
what topics you need help on, and studying for finals should be a lot easier.
NOTE: To help you study, you should also look over exercises assigned for quizzes,
honour homework, and assignments, as well as examples done in lectures and tutorials.
(This may sound like a lot, but don’t panic…if you regularly attend class and do your
homework, then you’ve got this all done already).  Also, don’t forget any
independent-study material from assignments, as that isn’t included in the study guide
below.
1.1: Introduction to Systems of Linear Equations
This section introduces a lot of terminology…to help you practice this, remember that we’ve posted a
Glossary and practice game in “Key Course Resources”. Obviously, know what a linear equation is, and
what it means to find a solution to a linear system (if you sub the supposed solution values in, ALL of
the equations should be satisfied)…remember that a linear system can have no solutions, 1 solution, or
infinitely many. Be comfortable writing a system in augmented matrix form (or going from matrix form
to the system of equations).
1.2: Gaussian Elimination
This section is about as important as it gets!! The concept of row reduction will be used throughout the
entire course! To do row reduction, there are 3 possible row operations that don’t change the solution of
a system: 1. Multiply a row by a non-zero constant. 2. Interchange (swap) two rows. 3. Add a
multiple of one row to another. Now, using these operations, our goal is to transform our augmented
matrix into one that is easier to solve, namely one that is REF or RREF (basically, REF means leading
1’s in each row with 0’s below it; leading 1’s are always to the right of the previous leading 1 as you
move downwards in the matrix, and any rows of 0’s are at the bottom. For RREF, any numbers above
leading 1’s have to be 0’s as well). If you transform to REF, that’s called Gaussian elimination, while if
you transform to RREF, that’s called Gauss-Jordan elimination. It is SUPER-IMPORTANT to be able
to do Gaussian/Gauss-Jordan elimination efficiently. Get lots of practice! You really can’t pass the
course without this concept. Once you get the basic concept, get some practice at the trickier questions,
where you’re asked to find values for constants that make a system have no solutions, 1 solution, etc. –
make sure you really understand what’s going on here, as each question is different, and it’s really about
having a deeper understanding of the concept.
1.3: Matrices and Matrix Operations
Again, this section has a lot of terminology you’ll need to get comfortable with (e.g. transpose, trace,
etc.). While some things in this section are super-easy and intuitive (e.g. adding matrices, or multiplying
by a scalar), others, like matrix multiplication, are quite tricky, counter-intuitive, and will require lots of
practice to make sure you get the concept (the idea is that to get the ij entry in the product, you’re
multiplying the entries in the ith row of the 1st matrix by those in the jth column of the 2nd matrix and
adding these up). Don’t panic though -- it’s actually quite easy once you get the hang of it, and don’t
forget that www.nool.ca has several videos to help!
1.4: Inverses and Algebraic Properties of Matrices
Know that for matrices, some of our usual logic that works for numbers will no longer apply…e.g. AB is
NOT necessarily the same as BA when you do multiplication; there are several other examples listed in
this section. Know what an identity matrix is, and understand that a matrix B is the inverse of A if
AB = BA = I . While we have an easy-to-use formula for the inverse of a 2× 2 matrix (so know how
to apply that), for anything larger, you’ll need to do more work (we learn a method in the next section).
Finally, this section involves being able to do proofs that something is the inverse of something else (e.g.
B-1A-1 is the inverse of AB) -- to do the proof, multiply the 2 expressions together and show that you get I
as a result. CAUTION: Remember that order of multiplication matters for matrices, so careful how you
complete the proof…in the above example, you have B −1 A −1 AB , so you can simplify A −1 A to get I to
start, and then work on the B’s, but you couldn’t start with the B’s since they’re not next to each other
(i.e. not multiplied directly). Really take the time to understand this rather than memorizing it, as each
proof is different, and it’s important to multiply things in the appropriate order (if it’s done out of order,
it’s pretty much an automatic grade of 0). Similarly, this section sometimes asks you to isolate for a
matrix in an equation…e.g. AXB = C …to isolate for X, you can just multiply by the inverse of A and B
on both sides, but again, remember that the order of multiplication is key!
1.5: Elementary Matrices and a Method for Finding A-1
If you can apply a series of row operations to a matrix to turn it into I, then the matrix is invertible. So,
to find the inverse of a matrix, we have to put A beside I (just put them next to each other in one
augmented matrix…careful with the notation…make sure it doesn’t look like you’re multiplying them
by mistake!) and then work to transform A into I; performing those same operations on I will transform
it into A-1. This works for any size matrix (incl. 2× 2 , but in that case, the formula from the previous
section is easier). Finally, the other things covered in this section are elementary matrices (just a matrix
that can be obtained from I by performing a single row operation), and the Equivalence Theorem is
introduced. The Equivalence Theorem is really important, and we’ll be adding to it throughout the
course as we learn new concepts…it helps us understand how various concepts for a square matrix are
related (e.g. if it’s invertible, then you can write it as a product of elementary matrices, etc.)…you
shouldn’t memorize this theorem word for word or anything (and it’s actually going to be quite long
once we’re done), but you need to fundamentally understand why and how the concepts are related, so
that if you were asked, ‘does this imply this?’, you’d be able to figure it out.
1.6: More on Linear Systems and Invertible Matrices
This section doesn’t have a lot of new content…we use the concept of invertibility we just learned to go
back to previous sections and better understand/prove some theorems. Using the concept of invertible
matrices, we also learn that if A is invertible, Ax = b will have the solution x = A −1b .
1.7: Diagonal, Triangular, and Symmetric Matrices
Well, as the name implies, there are 3 types of special matrices introduced in this section, so know the
terminology, and understand the properties associated with them (e.g. for diagonal matrices, powers of
the matrix are really easy to find because we can just take the powers of the diagonal entries). Really
nice, easy section!
1.10: Applications of Linear Systems
There’s no new math here…just doing some Gaussian/Gauss-Jordan elimination to help us solve
application problems. Of course, each application problem is a bit different in terms of how you’d set it
up (e.g. traffic problems where you’re considering how many cars come into and out of each
intersection) and how to interpret the resulting solution. Make sure you understand how to do the set-up
and the resulting interpretation for various types of applications (circuits, traffic, chemical equations,
polynomial interpolation, etc.) as that’s the new stuff here.
2.1: Determinants by Cofactor Expansion
There’s some terminology introduced here like ‘minor’ and ‘cofactor’, but of course, the ultimate
highlight is being able to find a number called a ‘determinant’ using cofactor expansion, just as the name
of this section implies.  While the determinant of a 2× 2 matrix is super-easy, and we have the
“ 3× 3 trick” available to use for 3× 3 matrices, once we get to larger sizes, there are no tricks to help us
unless the matrix happens to be triangular/diagonal. The cofactor expansion method involves picking
ANY row or column (choosing the one with the most 0’s will make life way easier) and multiplying
each of the entries in that row/column by (−1) i+ j and by the determinant of the resulting matrix when
the row and column containing that entry are removed, then adding the results. Kinda hard to explain in
words, but just practice and you’ll get the hang of it. This is an important concept as it will come up
over and over throughout the course, so make sure you’re able to find determinants quickly and easily.
Oh, and finally, just a quick word about notation…remember that vertical bars around your matrix
denote determinant, while regular or square brackets is what we use for just a matrix – make sure you
don’t mix these up!
2.2: Evaluating Determinants by Row Reduction
This section gives you another method for finding determinants, and it’s using the row operations you
already know and love. In fact, you can even do column operations here. But there are some
catches…each time you swap rows/columns, the sign of the determinant changes; also, if you multiply
your row/column by a number, the determinants goes up by that factor as well, so be careful. If you’re
feeling adventurous and confident, you could even mix the 2 techniques together by doing some row
reduction to generate some 0’s, and then using cofactor expansion (which would be a lot easier with all
those extra 0’s).
2.3: Properties of Determinants; Cramer’s Rule
This section covers 2 distinct things…1st topic is properties. For example, we learn that for an n × n
matrix A, det(kA) = k n det( A) , which makes sense based on what we learned in the previous section,
since you basically multiplied n rows of A by the number k. Also, det( AB) = det( A) det( B) but there
are no such rules for determinants of sums or differences. We even add to our equivalence theorem by
learning a key concept that a matrix is invertible iff (that’s shorthand for “if and only if”) det(A) is
NOT 0. Finally, the section also introduces Cramer’s rule, which gives us an alternate method for
solving a system that has an invertible coefficient matrix…namely,
det( An ) where A means we’ve taken A and replaced the ith column
det( A1 )
det( A2 )
x1 =
, x2 =
, , x n =
i
det( A)
det( A)
det( A)
with b. This can be very useful if we only need to solve for one variable, and particularly important in
2nd year courses where the matrix might contain functions rather than numbers, so row reduction won’t
be much fun.
3.1: Vectors in 2-Space, 3-Space, and n-Space
This is just an intro section with lots of terminology and high school review, like how to add vectors,
etc. We also prove a few basic properties. Overall, very easy!
3.2: Norm, Dot Product, and Distance in Rn
Again, a lot of high school review. Know how to find the norm (length of a vector); if it’s of length 1,
it’s called a unit vector. Similarly, know how to find the distance between 2 vectors (just subtract them,
then find the norm of the resulting vector). Know how to find the dot product (also called the Euclidean
Inner Product) of 2 vectors. Finally, be able to use the formula u ⋅ v = u v cos(θ ) (note it just
involves finding a dot product and some norms) to find the angle between 2 vectors. Again, a few
basic properties are proved, and we finish off with the connection between dot product and matrix
multiplication (amazing how all the concepts are related, eh? )
3.3: Orthogonality
Not much to this section either…we learn that 2 vectors are orthogonal (perpendicular) if their dot
product is 0, and we also learn how to find the projection of one vector onto another (as well as the
component orthogonal to another vector)…you’d be given the formulas for this, and they just involve
finding some dot products and norms, so super-easy!
3.4: The Geometry of Linear Systems
On the surface, this section is super-easy…we just learn the equation of a plane
(a ( x − x0 ) + b( y − y0 ) + c( z − z0 ) = 0) and the parametric equations of a line
( x = x0 + ta,
y = y0 + tb,
z = z 0 + tc) . Fine. But what can get a bit tricky is if you’re asked to
find the equation of a plane or line that’s parallel or perpendicular to some other plane or line, because
then you start having to think about what these things look like, and that e.g. if a line and a plane are
parallel, that actually means the line and the normal to the plane are orthogonal, etc.
3.5: Cross-Product
Again, high school review. Just know how to find the cross-product of 2 vectors in 3-space, and some
of the properties of cross-product. Also, we learned that the area of a parallelogram determined by 2
vectors is just the norm of the cross-product of those 2 vectors (i.e. || u × v || ).
4.1: Real Vector Spaces
This section is all proof, and that makes it quite challenging for most. Take a deep breath and don’t
panic. At the end of the day, we’re just hoping to extend our work with vectors to objects such as
matrices and polynomials, but we can only do that if those objects satisfy the key fundamental properties
of regular vectors, so that’s what we’re testing for here. There are 10 properties that need to be satisfied
for a collection of objects to be deemed a “vector space”, so you just go through and test each one
individually (well, if one fails, then you can stop testing, as it’s definitely not going to be a vector
space). You don’t need to memorize these…just be able to do the proofs/tests for each property. Take
the time to really understand what’s being done here, rather than memorizing…each vector space
property is tested differently (1, 4, 5, and 6 are about checking whether the resulting vector is in the
given set, while the others are based on checking that the left-hand-side of the equation and the righthand-side give you the same value). Also, some questions even give you weird rules of addition or
scalar multiplication to work with, so get lots of practice. Make sure you understand the why and how,
and everything you’ll need to write for a complete formal proof.
4.2: Subspaces
This is another tricky proof-y section, but the good news is, if you’ve taken the time to master 4.1,
there’s nothing new here!  When your collection of objects is already a subset of something that is
known to be a vector space (watch the terminology – subset vs subspace, and make sure you understand
the difference), turns out you don’t need to test all 10 properties…just #1 (closure under addition) and
#6 (closure under scalar multiplication). If both hold, then we’ve got a subspace! So a lot less work.
Know these 2 properties! (i.e. you could be asked to check if something is a subspace, so you need to
know what to do). Again, get lots of practice, and really understand the structure of the proof so you can
complete it correctly (each little detail is important!)
4.3: Spanning Sets
First, this section introduces the concept of linear combinations (a vector is said to be a linear
combination of other vectors if it can be written as the sum of those vectors multiplied by some
coefficients, i.e. w = k1 v1 + k 2 v 2 +  + k r v r ) The last concept introduced in this section is ‘span’, which
is the space that your collection of objects ‘covers’ (i.e. the set of all linear combinations of those
vectors). For a set of vectors to span a space (as an example, let’s use R3 here) there has to be a way to
write every vector (a, b, c) as a linear combo of the given vectors….i.e. the resulting system of
equations that you set up (based on equating coefficients) has to always have a solution (i.e. be
consistent) for any choice of a, b, c. There’s no new math (it’s just row reduction again), but make
sure you very clearly understand the concept (set-up and interpretation of results). Note that in Rn, if a
set has fewer than n vectors, there’s no way for it to span.
4.4: Linear Independence
A set of vectors is called linearly independent if no vector in the set can be expressed in terms of the
others…this means k1 v1 + k 2 v 2 +  + k r v r = 0 will only have the trivial solution (otherwise, you’d be
able to solve for one of them in terms of the others, and then they’d be dependent). That means, if we
want to test if a set of vectors is independent, we just set up the above equation, plug the vectors in,
equate coefficients and set up the resulting linear system, and check if this system has only the trivial
solution. Just like with the concept of span, there’s absolutely no new math (it’ll all be row reduction
again), but make sure you really understand the concept, and don’t confuse it with span. Note that in Rn,
if a set has more than n vectors, there’s no way for it to be independent.
4.5: Coordinates and Basis
Know that in order to have a basis (for defining a coordinate system), our vectors must be both
independent (to ensure a unique way to write vectors in terms of the basis) AND span (to ensure that any
vector can be written in terms of the basis). Also, know that a coordinate vector is just comprised of the
coefficients we get when we write our vector as a linear combination of the basis vectors. If you wanted
to figure out the coordinate vector, then you’d just have to solve for those coefficients in the linear
combo. Guess what? -- more row reduction. 
4.6: Dimension
This section is kind of a continuation of the previous one, giving the name ‘dimension’ to the number of
vectors that’s needed to form a basis for a space. For a given set, know how to figure out both a basis
for it, and its dimension. It’s really easy once you get the hang of it…for example, if you had vectors of
the form (a, 2b, a − b) , it’s really easy to see there are 2 degrees of freedom: a and b…so the
dimension of the set is 2. Done. As for the basis, if we write this as a (1, 0, 1) + b(0, 2, − 1) (just ask
yourself how many a’s there are in each component, and how many b’s), we also get the basis vectors
right away… (1, 0, 1) and (0, 2, − 1) . We also learn that if you have too many vectors in a set V (more
than you’d need for a basis), it can’t be indep, but if you have too few, they can’t span…so you’d have
to have exactly the right amount e.g. n in the case of Rn. So if we have a set of n vectors in Rn (any
other number couldn’t possibly be a basis), and we’re curious as to whether these form a basis, we’ll be
working with a square coefficient matrix when you set things up to either test for independence or span
(the same square matrix in both cases, so it’ll either have a 0 determinant or it won’t). Where I’m going
with this is that to test for basis, it’s enough to test for either independence OR span (since the
determinant of the resulting matrix would work out the same in both cases, so why re-do it twice?) If
it’s non-zero, then you know your vectors are independent, span, and form a basis. If it’s zero, then
they’re dependent, don’t span, and don’t form a basis. Of course basis means independent AND spans,
so on a test, you’d have to write a note explaining that both things need to hold, and why it’s ok to just
test for one!
4.8: Row Space, Column Space, and Null Space
Know that the row space is just the space spanned by the rows of your matrix (similarly column space is
space spanned by columns) and nullspace of a matrix A is just a fancy new name for the solution of the
homogeneous system Ax = 0 . We already know how to solve Ax = 0 , so once you do, assuming there
are non-trivial solutions, you’ll get parameters multiplying some vectors…those vectors are called the
basis of your nullspace. To find rowspace, elementary row operations don’t affect it, so just row reduce,
and whatever rows have leading 1’s in the REF matrix are a basis for your rowspace. Unfortunately,
row operations do change the column space, but they preserve the info about which columns form the
basis, so go to REF, look at the columns with leading 1’s, and the corresponding columns in the original
matrix will form a basis for your column space.
4.9: Rank, Nullity, and the Fundamental Matrix Spaces
This section is pretty easy…“rank” is the number of vectors in the basis for your row or column space
(it’ll be the same for both of these, since it’s just the number of leading 1’s in your REF or RREF
matrix). Nullity is the number of vectors in your nullspace basis (which is just the number of parameters
in the solution of your homogeneous system). Since each column in your coefficient matrix corresponds
ultimately to either a leading 1 or a parameter, it makes a lot of sense that rank ( A) + nullity ( A) = n
T
(where n is the # of columns in A). Oh, and this section also mentions that the row space of A is the
same as the column space of A, and vice versa (makes sense since you’re just flipping rows and
columns) and things like rank ( A) = rank ( AT ) . This section also introduces vocabulary like
“overdetermined system” (more equations than unknowns) and “underdetermined system” (fewer
equations than unknowns).
1.8: Introduction to Linear Transformations
In this section, we’re applying some sort of mapping to go from one vector to another; in the case of
linear transformations (know how to test for linearity by checking the axioms T (u + v) = T (u) + T ( v)
and T (ku) = kT (u) ), we can perform this transformation by multiplying our vector by the appropriate
transformation matrix. While this section is full of transformation matrices for rotations, reflections, etc,
please don’t memorize these!...the main point of the section is to know how to figure out the appropriate
matrix for a transformation by following the key steps:
Step 1: Find the images of the standard basis vectors e1 , e 2 ,, e n for R n in column form.
Step 2: Construct the matrix that has the images obtained in Step 1 as its successive columns. This
matrix is the standard matrix for the transformation.
1.9: Compositions of Matrix Transformations
To find the matrix denoting a transformation that’s a more complicated composition of transformation
(e.g. rotation, then reflection, then projection), you can proceed as in the previous section, figuring out
the resulting location of the standard unit basis vectors, or figure out the matrix for each individual
transformation, then multiply them in the appropriate order (right to left, in order of application).
Know what it means for a transformation to be one-to-one – basically, it can be undone – which means
we’d just need to find the inverse matrix to get the inverse transformation.
5.1: Eigenvalues and Eigenvectors
For an n × n matrix A, if Ax = λx , then λ is called an eigenvalue of A and x is called an eigenvector
of A. The concept of finding eigenvalues and eigenvectors (eigenvectors might also be referred to as a
basis for the eigenspace of A) is very important for upper year courses, so make sure you know how to
do this well. Luckily, it’s all just more determinants, and you guessed it, row reduction.  To find
eigenvalues, we’re just solving det( A − λI ) = 0 ; then for each eigenvalue, we solve ( A − λI )x = 0 to get
the corresponding eigenvectors. Keep in mind that if your matrix is triangular, the eigenvalues are just
the entries on the main diagonal, so that makes it way easier! Also, be comfortable working with
applications related to this topic, like Markov chains and differential equations. By the way, we finally
stop adding things to the Equivalence Theorem in this section, so now is a good time to review it again,
really making sure you understand each and every aspect.
5.2: Diagonalization
A matrix is called diagonalizable if it’s possible to find a matrix P so that P −1 AP = D (a diagonal
matrix), which can always be done as long as you have n linearly independent eigenvectors for A;
so….basically, you just find the eigenvalues and eigenvectors of A, and then form P by placing the
eigenvectors of A in columns; D will naturally work out to be a diagonal matrix with the eigenvalues of
A on the diagonal (the order of the eigenval in D will match the order of the eigenvectors in P) [and if
you couldn’t get enough eigenvectors to form a square matrix for P, then that means A isn’t
−1
diagonalizable]. One useful feature of all this diagonalization stuff is that we have A = PD P
which can help us compute powers of a matrix, which we’re able to do because we also learned that for
k
k
k
the matrix A , the eigenvalues are just those of A, but raised to the power k, and the eigenvectors are
identical to those of A. Now, we have a theorem that tells us that if you get distinct eigenvalues for
your matrix, you’ll end up with enough linearly independent eigenvectors, so the only way you can run
into problems diagonalizing A is if you have repeated eigenvalues, and they don’t lead to enough
linearly independent eigenvectors…know the terminology like ‘algebraic multiplicity” and “geometric
multiplicity” that we use to talk about how many eigenvectors we’re expecting and how many we end
up with.
6.1: Inner Products
Really, this is mostly the same as 3.1, so very easy. Now that we’ve generalized the concept of a vector
to apply to things like polynomials and matrices when we studied vector spaces, it makes sense to revisit
concepts like norm, distance, and inner product…basically, this section is just an extension of the
formulas we had before (know how to find these!). We also introduced weighted inner products in this
section, so know how to find these, as well as norms and distances when you’re working with these
weighted inner products. Finally, know how to find inner products generated by matrices, given by
u, v = vT AT Au , which is easy, since it’s just based on doing matrix multiplication. The section
actually ends with finding inner products of linear combinations, like 2 v − w, 3u + 2w , so you’ll need
to know how to apply inner product properties to evaluate these.
6.2: Angle and Orthogonality in Inner Product Spaces
Just like in 3.2, two objects are orthogonal if their inner product is 0. Easy. We’re also introduced to
the concept of W ⊥ in this section (the orthogonal complement of a space), which is the set of all vectors
orthogonal to W. We learn that the nullspace of A and the row space of A happen to be orthogonal
complements; similarly, the column space of A and the nullspace of AT are orthogonal complements too
– so next time you see a question asking for the orthogonal complement of a bunch of vectors, you can
just write them as rows in a matrix and find the nullspace (since that’s the orthogonal complement of the
row space).
6.3: Gram-Schmidt Process
This whole section (well, we skip the QR decomposition part, so just focus on Gram-Schmidt) is about
finding an orthonormal basis (a basis in which all the vectors are orthogonal and have length 1).
Orthonormal bases are cool for various reasons…for example, finding the coordinate vector relative to
the basis is much easier than usual…it’s just (u) S = ( u, v1 , u, v 2 ,, u, v n ) , and let’s face it, inner
products are much easier than the usual row reduction we’d normally have to do to find a coordinate
vector back in section 4.4 (although that method still works here too…just unnecessarily long). To find
an orthonormal basis, we start with a given set of vectors u1 , u 2 , etc. and do:
a) Set v1 = u1
b) v 2 = u 2 − projW1 u 2
c) v 3 = u 3 − projW2 u 3
Etc.
In other words, we keep the 1st vector, but then to make sure all the other ones will be orthogonal to it,
we subtract off the projection of the other vectors on those that are already in our basis.
Know/understand this process!! This is all done using the projection formula
, which is the only formula that’s provided here.
Applying the process correctly gets you an orthogonal basis, and to make it orthonormal, you just divide
each vector by its length (that’ll preserve its direction, while making it have length 1).