- No category
Linear Algebra Textbook: Vectors, Matrices, Applications
advertisement
Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 What Is Linear Algebra and Why Learn It? About This Book Prerequisites Math Attitude Coding Mathematical Proofs Versus Intuition from Coding Code, Printed in the Book and Downloadable Online Code Exercises How to Use This Book (for Teachers and Self Learners) 1 2 2 3 3 3 4 5 5 6 2. Vectors, Part 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Creating and Visualizing Vectors in NumPy Geometry of Vectors Operations on Vectors Adding Two Vectors Geometry of Vector Addition and Subtraction Vector-Scalar Multiplication Scalar-Vector Addition Transpose Vector Broadcasting in Python Vector Magnitude and Unit Vectors The Vector Dot Product The Dot Product Is Distributive Geometry of the Dot Product 7 10 11 11 12 13 14 15 16 17 18 20 21 iii Other Vector Multiplications Hadamard Multiplication Outer Product Cross and Triple Products Orthogonal Vector Decomposition Summary Code Exercises 22 22 23 24 24 28 29 3. Vectors, Part 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Vector Sets Linear Weighted Combination Linear Independence The Math of Linear Independence Independence and the Zeros Vector Subspace and Span Basis Definition of Basis Summary Code Exercises 33 34 35 37 38 38 41 44 46 46 4. Vector Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Correlation and Cosine Similarity Time Series Filtering and Feature Detection k-Means Clustering Code Exercises Correlation Exercises Filtering and Feature Detection Exercises k-Means Exercises 49 52 53 57 57 58 60 5. Matrices, Part 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Creating and Visualizing Matrices in NumPy Visualizing, Indexing, and Slicing Matrices Special Matrices Matrix Math: Addition, Scalar Multiplication, Hadamard Multiplication Addition and Subtraction “Shifting” a Matrix Scalar and Hadamard Multiplications Standard Matrix Multiplication Rules for Matrix Multiplication Validity Matrix Multiplication Matrix-Vector Multiplication Matrix Operations: Transpose iv | Table of Contents 61 61 63 65 65 66 67 67 68 69 70 72 Dot and Outer Product Notation Matrix Operations: LIVE EVIL (Order of Operations) Symmetric Matrices Creating Symmetric Matrices from Nonsymmetric Matrices Summary Code Exercises 73 73 74 74 75 76 6. Matrices, Part 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Matrix Norms Matrix Trace and Frobenius Norm Matrix Spaces (Column, Row, Nulls) Column Space Row Space Null Spaces Rank Ranks of Special Matrices Rank of Added and Multiplied Matrices Rank of Shifted Matrices Theory and Practice Rank Applications In the Column Space? Linear Independence of a Vector Set Determinant Computing the Determinant Determinant with Linear Dependencies The Characteristic Polynomial Summary Code Exercises 82 83 84 84 88 88 91 94 96 97 98 99 99 100 101 102 103 104 106 107 7. Matrix Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Multivariate Data Covariance Matrices Geometric Transformations via Matrix-Vector Multiplication Image Feature Detection Summary Code Exercises Covariance and Correlation Matrices Exercises Geometric Transformations Exercises Image Feature Detection Exercises 113 116 120 124 124 124 126 127 8. Matrix Inverse. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 The Matrix Inverse Types of Inverses and Conditions for Invertibility 129 130 Table of Contents | v Computing the Inverse Inverse of a 2 × 2 Matrix Inverse of a Diagonal Matrix Inverting Any Square Full-Rank Matrix One-Sided Inverses The Inverse Is Unique Moore-Penrose Pseudoinverse Numerical Stability of the Inverse Geometric Interpretation of the Inverse Summary Code Exercises 131 131 133 134 136 138 138 139 141 142 143 9. Orthogonal Matrices and QR Decomposition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Orthogonal Matrices Gram-Schmidt QR Decomposition Sizes of Q and R QR and Inverses Summary Code Exercises 147 149 150 151 154 154 155 10. Row Reduction and LU Decomposition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Systems of Equations Converting Equations into Matrices Working with Matrix Equations Row Reduction Gaussian Elimination Gauss-Jordan Elimination Matrix Inverse via Gauss-Jordan Elimination LU Decomposition Row Swaps via Permutation Matrices Summary Code Exercises 159 160 161 163 165 166 167 169 170 171 172 11. General Linear Models and Least Squares. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 General Linear Models Terminology Setting Up a General Linear Model Solving GLMs Is the Solution Exact? A Geometric Perspective on Least Squares Why Does Least Squares Work? vi | Table of Contents 176 176 176 178 179 180 181 GLM in a Simple Example Least Squares via QR Summary Code Exercises 183 187 188 188 12. Least Squares Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Predicting Bike Rentals Based on Weather Regression Table Using statsmodels Multicollinearity Regularization Polynomial Regression Grid Search to Find Model Parameters Summary Code Exercises Bike Rental Exercises Multicollinearity Exercise Regularization Exercise Polynomial Regression Exercise Grid Search Exercises 193 198 199 199 200 204 206 206 206 207 208 210 210 13. Eigendecomposition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 Interpretations of Eigenvalues and Eigenvectors Geometry Statistics (Principal Components Analysis) Noise Reduction Dimension Reduction (Data Compression) Finding Eigenvalues Finding Eigenvectors Sign and Scale Indeterminacy of Eigenvectors Diagonalizing a Square Matrix The Special Awesomeness of Symmetric Matrices Orthogonal Eigenvectors Real-Valued Eigenvalues Eigendecomposition of Singular Matrices Quadratic Form, Definiteness, and Eigenvalues The Quadratic Form of a Matrix Definiteness ATA Is Positive (Semi)definite Generalized Eigendecomposition Summary Code Exercises 214 214 215 216 217 217 220 221 222 224 224 226 227 228 228 230 231 232 233 234 Table of Contents | vii 14. Singular Value Decomposition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 The Big Picture of the SVD Singular Values and Matrix Rank SVD in Python SVD and Rank-1 “Layers” of a Matrix SVD from EIG SVD of ATA Converting Singular Values to Variance, Explained Condition Number SVD and the MP Pseudoinverse Summary Code Exercises 241 243 243 244 246 247 247 248 249 250 251 15. Eigendecomposition and SVD Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 PCA Using Eigendecomposition and SVD The Math of PCA The Steps to Perform a PCA PCA via SVD Linear Discriminant Analysis Low-Rank Approximations via SVD SVD for Denoising Summary Exercises PCA Linear Discriminant Analyses SVD for Low-Rank Approximations SVD for Image Denoising 255 256 259 259 260 262 263 263 264 264 269 272 275 16. Python Tutorial. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Why Python, and What Are the Alternatives? IDEs (Interactive Development Environments) Using Python Locally and Online Working with Code Files in Google Colab Variables Data Types Indexing Functions Methods as Functions Writing Your Own Functions Libraries NumPy Indexing and Slicing in NumPy viii | Table of Contents 279 280 280 281 282 283 284 285 286 287 288 289 289 Visualization Translating Formulas to Code Print Formatting and F-Strings Control Flow Comparators If Statements For Loops Nested Control Statements Measuring Computation Time Getting Help and Learning More What to Do When Things Go Awry Summary 290 293 296 297 297 297 299 300 301 301 301 302 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 Table of Contents | ix
0
advertisement
Related documents
Download
advertisement
Add this document to collection(s)
You can add this document to your study collection(s)
Sign in Available only to authorized usersAdd this document to saved
You can add this document to your saved list
Sign in Available only to authorized users