Regression Analysis
• Jared Dean as quoted in Big Data, Data
Mining, and Machine Learning
– From my experience, regression is the most dominant
force in driving business decisions today. Regression
analysis has many useful characteristics; one is the easy
interpretation of results. Regression concepts are widely
understood, and the methodology is well developed
such that a well-tuned regression model by a skilled
practitioner can outperform many algorithms that are
gaining popularity from the machine learning discipline.
Simple Linear Regression
Simple Linear Regression
Khan Academy Videos
Formula Derivation (4 parts)
Examples (2 parts)
R-squared or coefficient of determination (2 parts)
Linear regression calculator
Compute the equation for the least-squares, best-fit line
through the 5 points {(2,2), (0,0), (-2,-2), (-1,1), (1,-1)}
General Regression – not just straight line
Least Squares Approximation
Introduction to Algorithms, Cormen, et al., MIT Press
Least Squares Approximation
• Pseudoinverse of Matrix A - Linear Example
• On Whiteboard
Compute the equation for the least-squares, best-fit line
through the 5 points {(2,2), (0,0), (-2,-2), (-1,1), (1,-1)}
Linear Regression versus
Principal Component Analysis
Reference
Linear Regression
Principal Component Analysis
Linear Regression versus
Reverse Linear Regression
Reference
Linear Regression
Reverse Linear Regression