� Linear Algebra Study Guide
Table of Contents
1. Vectors and Vector Spaces
2. Matrices and Matrix Operations
3. Systems of Linear Equations
4. Determinants
5. Vector Spaces and Subspaces
6. Linear Independence, Basis, and Dimension
7. Linear Transformations
8. Eigenvalues and Eigenvectors
9. Diagonalization
10. Orthogonality and Inner Product Spaces
11. Gram-Schmidt Process and QR Decomposition
12. Applications of Linear Algebra
1. � Vectors and Vector Spaces
Definitions:
●
Vector: An element of a vector space (e.g., in ℝⁿ, it’s an n-tuple of real numbers).
●
Vector Space: A set closed under vector addition and scalar multiplication.
●
Zero Vector: The unique vector 0 such that v + 0 = v.
Properties:
●
Closure under addition and scalar multiplication
●
Existence of additive identity and inverse
●
Associativity, distributivity, etc.
Example:
ℝ² is a 2D vector space:
v⃗=[23],w⃗=[−15]\vec{v} = \begin{bmatrix} 2 \\ 3 \end{bmatrix}, \quad \vec{w} = \begin{bmatrix} 1 \\ 5 \end{bmatrix}v=[23],w=[−15]
2. � Matrices and Matrix Operations
Definitions:
●
Matrix: Rectangular array of numbers.
●
Transpose: Swap rows and columns.
●
Matrix Multiplication: Multiply rows by columns (dot product).
●
Identity Matrix (I�): Diagonal matrix with 1s.
Operations:
●
Addition/subtraction
●
Scalar multiplication
●
Matrix multiplication
●
Transpose
●
Inverse (if exists)
Properties:
●
(AB)^T = B^T A^T
●
Associative and distributive laws
●
Non-commutative multiplication
3. � Systems of Linear Equations
Forms:
●
Matrix form: Ax = b
●
Augmented matrix: [A∣b][A|b][A∣b]
Solution Methods:
●
Gaussian Elimination
●
Row Echelon Form (REF)
●
Reduced Row Echelon Form (RREF)
●
LU Decomposition
Solution Types:
●
Unique
●
Infinite (free variables)
●
No solution (inconsistent)
4. � Determinants
Definition:
A scalar that gives information about the matrix (area, invertibility).
Properties:
●
det(AB) = det(A)·det(B)
●
det(A^T) = det(A)
●
A is invertible ⇔ det(A) ≠ 0
Methods:
●
Expansion by minors
●
Row reduction
●
Special formulas for 2×2 and 3×3
5. � Vector Spaces and Subspaces
Definitions:
●
Subspace: A subset of a vector space that is itself a vector space.
●
Must contain zero vector, be closed under addition and scalar multiplication.
Examples:
●
Null space of a matrix
●
Column space
●
Row space
Theorems:
●
Intersection of subspaces is a subspace
●
Span(S) is the smallest subspace containing S
6. � Linear Independence, Basis, and Dimension
Definitions:
●
Linear Independence: No vector in the set can be written as a combination of others.
●
Basis: A linearly independent spanning set.
●
Dimension: Number of vectors in a basis.
Tests:
●
Use row reduction to check independence.
●
dim(Col A) = rank(A)
7. � Linear Transformations
Definitions:
●
Linear Transformation: T: V → W such that
T(u + v) = T(u) + T(v), T(c·v) = c·T(v)
●
Kernel: Set of vectors mapped to 0
●
Image: Set of all outputs
Matrix Representation:
If T: ℝⁿ → ℝᵐ, there exists matrix A such that
T(x) = Ax
Properties:
●
T is injective ⇔ ker(T) = {0}
●
T is surjective ⇔ im(T) = W
8. � Eigenvalues and Eigenvectors
Definitions:
●
Eigenvalue (λ): Scalar such that A·v = λ·v
●
Eigenvector (v): Nonzero vector satisfying A·v = λ·v
●
Characteristic Polynomial: det(A - λI)
How to Find:
1. Solve det(A - λI) = 0 to find eigenvalues
2. Solve (A - λI)v = 0 to find eigenvectors
9. � Diagonalization
Definition:
A matrix A is diagonalizable if
A = PDP⁻ ¹, where D is diagonal with eigenvalues
Conditions:
●
A has n linearly independent eigenvectors ⇔ A is diagonalizable
●
Symmetric matrices are always diagonalizable (and orthogonally)
10. � Orthogonality and Inner Product Spaces
Definitions:
●
Dot Product: a · b = a₁ b₁ + a₂ b₂ + ... + aₙbₙ
●
Orthogonal Vectors: a · b = 0
●
Norm: ‖v‖ = √(v·v)
Orthogonal Subspaces:
●
Row(A) ⊥ Null(A)
●
Col(A) ⊥Null(Aᵗ )
Orthogonal Matrix:
QᵀQ = I ⇒ Q⁻¹ = Qᵀ
11. � Gram-Schmidt and QR Decomposition
Gram-Schmidt:
Process to convert a basis into an orthonormal basis.
Steps:
1. v₁ = u₁
2. v₂ = u₂ - proj_{v₁ }(u₂ )
3. Normalize each vector
QR Decomposition:
A = QR
●
Q: orthonormal columns
●
R: upper triangular matrix
12. � Applications of Linear Algebra
●
Computer graphics: transformations using matrices
●
Machine Learning: data representation, PCA
●
Physics: state vectors, quantum mechanics
●
Economics: input-output models
●
Engineering: system modeling and controls
●
Cryptography, Networks, and more
✅ Tips for Studying
●
Memorize key definitions and theorems
●
Practice proofs, especially for vector spaces and transformations
●
Solve many problems, especially with row reduction, eigenvalues, and diagonalization
●
Use a calculator or software (like MATLAB, NumPy, or SageMath) to check work
●
Learn geometric interpretations (especially in ℝ² and ℝ³)
� Suggested Resources
●
●
Books:
○
Linear Algebra and Its Applications by David C. Lay
○
Introduction to Linear Algebra by Gilbert Strang
Online:
○
MIT OpenCourseWare (Gilbert Strang's lectures)
○
Khan Academy
○
3Blue1Brown's "Essence of Linear Algebra" (YouTube series)