Section 3.3 ~ Least Squares Regression
Least-Squares regression:
Correlation measures the strength and direction of the linear relationship
Least-squares regression
Method for finding a line that summarizes that relationship between two variables in a specific setting.
Regression line
Describes how: a response variable y changes as an explanatory variable x changes
Used to: predict the value of y for a given value of x
Unlike correlation, requires an: explanatory and response variable
Least-Square Regression Line (LSRL):
If you believe the data show a linear trend, it would be appropriate to try to fit an LSRL to the data
We will use the line to predict y from x, so you want the LSRL to: be as close as possible to all the points in the
vertical direction
That’s because any prediction errors we make are errors in y, or the vertical direction of the
scatterplotError = actual – predicted
The least squares regression line of y on x is the line: that makes the sum of the squares of the vertical distances
of the data points from the line as small as possible
The equation for the LSRL is: 𝑦̂ = 𝑎 + 𝑏𝑥
𝑦̂ is used because: the equation is representing a prediction of y
To calculate the LSRL you need the: means and standard deviations of the two variables as well as the
correlation
The slope is b and the y-intercept is a
Every least-squares regression line passes through: the point (𝑥̅ , 𝑦̅)
Example 1 – Finding the LSRL
Using the data from example 1 (the number of student absences and their overall grade) in section 3.2, write the least
squares line.
Finding the LSRL and Overlaying it on your Scatterplot:
Press the STAT key
Scroll over to CALC
Use option 8
After the command is on your home screen:
Put the following L1, L2, Y1
To get Y1, press VARS, Y-VARS, Function
Press enter
The equation is now stored in Y1
Press zoom 9 to see the scatterplot with the LSRL
Use the LSRL to Predict:
With an equation stored on the calculator it makes it easy to calculate a value of y for any known x.
Using the trace button
2nd Trace, Value
x = 18
Using the table
2nd Graph
Go to 2nd window if you need change the tblstart
Example 2 Use the LSRL to predict the overall grade for a student who has had 18 absences. Also, interpret the slope and intercept
of the regression line.
A student who has had 18 absences is predicted to have an overall grade of about 14%
The slope is -4.81 which in terms of this scenario means that for each day that a student misses, their overall
grade decreases about 4.81 percentage points
Minitab Output:\The intercept is at 101.04 which means that a student who hasn’t missed any days is predicted to have
a grade
The role of r2 in regression:
Coefficient of determination
The proportion of the total sample variability that is explained by the least-squares regression of y on x
It is the square of the correlation coefficient (r), and is therefore referred to as r2
In the student absence vs. overall grade example, the correlation was r = -.946
The coefficient of determination would be: r2 = .8949
This means that: about 89% of the variation in y is explained by the LSRL
In other words,, 89% of the data values are accounted for by the LSRL
Facts about least square regression:
1. Distinction between explanatory and response variables is essential
a. If we reversed the roles of the two variables, we get a different LSRL
2. There is a close connection between correlation and the slope of the regression line
a. A change of one standard deviation in x corresponds to: r standard deviations in y
3. The LSRL always passes through the point
a.
We can describe regression entirely in terms of basic descriptive measures:
4. The coefficient of determination is the fraction of the variation in values of y that is explained by the leastsquares regression of y on x
Residuals:
Residuals are: Deviations from the overall pattern
Measured as vertical distances
They are the difference between: an observed value of the response variable and the value predicted by
the regression line
Residual = Observed y – predicted y
The mean of the least-squares residuals is always zero
If you round the residuals you will end up with a value very close to zero
Getting a different value due to rounding is known as roundoff error
Residual Plots:
A residual plot is a scatterplot of the regression residuals against the explanatory variable
Residual plots help us assess the fit of a regression line
Below is a residual plot that shows a linear model is a good fit to the original data
Reason
There is a uniform scatter of points
Below are two residual plots that show a linear model is not a good fit to the original data
Reasons
Curved pattern
Residuals get larger with larger values of x
An outlier is an observation that lies outside the overall pattern in the y direction of the other observations.
Influential Points:
An observation is influential if: removing it would markedly change the result of the LSRL
Outliers in the _______________ of a scatterplot are often times influential points
Have small residuals, because they pull the regression line toward themselves.
If you just look at residuals, you will miss influential points.
Can greatly change the interpretation of data
Location of influential vs outlier:
0
