Simple Linear Regression
Definition and Goals
Definition: A statistical method to model the relationship between a dependent variable and an
independent variable (x).
Equation: y = β0 + β1x + ε (ε = error term)
Goals: Predict y values, identify relationships, and estimate coefficients (β0, β1).
Assumptions
Linearity, independence, homoscedasticity, normality, and no multicollinearity.
Correlation
Definition: Measures the strength and direction of a linear relationship between two variables.
Types: Pearson's r (parametric), Spearman's ρ (non-parametric), and Kendall's τ.
Interpretation: Values range from -1 (perfect negative correlation) to 1 (perfect positive
correlation).
Correlation coefficient (r): Measures strength and direction.
Differences Between Regression and Correlation
Purpose: Regression predicts y values, while correlation measures relationship strength.
Direction: Regression implies causality, whereas correlation suggests association.
Equation: Regression provides a predictive model, whereas correlation provides a coefficient.
Relationship Between Regression and
Correlation
Key Concepts
Correlation coefficient (r): Square root of R-squared (coefficient of determination) in simple
linear regression.
R-squared: Measures variability explained by the regression model.
Regression slope (β1): Related to correlation coefficient (r).
Statistical Tests
t-test: Evaluates regression coefficients.
F-test: Assesses overall model significance.
p-value: Indicates probability of observing results by chance.
Common Metrics
Mean Squared Error (MSE): Measures regression model accuracy.
Coefficient of Determination (R-squared): Evaluates model fit.
Root Mean Squared Error (RMSE): Measures model accuracy.
Problem Example: Bakery Sales Prediction
Data
A bakery wants to predict the number of bread loaves sold based on the number of hours
advertised (x).
Data:
Hours
Bread
Advertised Loaves
(x)
Sold (y)
2
100
4
150
6
200
8
250
10
300
Step-by-Step Solution
Calculate means: x̄ = (2+4+6+8+10)/5 = 6, ȳ = (100+150+200+250+300)/5 = 200
Calculate deviations:
x deviation y deviation
x- x̄ y- ȳ
x
y
x-6
y-200
(x-6)(y200)
(x-6)^2
2
100
-4
-100
400
16
4
150
-2
-50
100
4
6
200
0
0
0
0
8
250
2
50
100
4
10
300
4
100
400
16
Calculate slopes (β1): β1 = Σ[(x-6)(y-200)] / Σ(x-6)2 = (400+100+0+100+400) / (16+4+0+4+16)
= 1000 / 40 = 25
Calculate intercept (β0): β0 = 200 - 25*6 = 200 - 150 = 50
Linear Regression Equation: y = 50 + 25x
Plot the graph using scatter plot
Interpret the result: Every additional hour advertised increases bread loaves sold by 25. The
bakery sells 50 loaves when no hours are advertised.