Correlation, Linear and
Logistic Regression
Chapter 10
Quantitative Methods
 Causal Models or Associative Models
 Explores cause-and-effect relationships
 Uses leading indicators to predict the future
 Housing starts and appliance sales
 Time Series Models
 Assumes information needed to generate a forecast is
contained in a time series of data
 Assumes the future will follow same patterns as the past
Key Concept
• Linear correlation coefficient, r, is a number that
measures how well paired sample data fit a straightline pattern when graphed.
• Using paired sample data (sometimes called bivariate
data), we find the value of r, then we use that value to
conclude that there is (or is not) a linear correlation
between the two variables.
• There are statistical tests to determine whether a
correlation is significant or not. These tests will be
presented now but explained later in the course, in
order to introduce the concepts of correlation and
regression sooner.
Definition
A correlation exists between two variables when the
values of one are somehow associated (move together)
with the values of the other in some way.
A linear correlation exists between two variables when
there is a correlation and the plotted points of paired
data result in a pattern that can be approximated by a
straight line.
Correlation
 Measures the relative strength of the linear
relationship between two variables
 Unit-less
 Ranges between –1 and 1
 The closer to –1, the stronger the negative linear relationship
 The closer to 1, the stronger the positive linear relationship
 The closer to 0, the weaker any positive linear relationship
Scatter Plots of Data with Various
Correlation Coefficients
Y
Y
Y
X
X
r = -1
r = -.6
Y
r=0
Y
Y
r = +1
X
X
X
r = +.3
X
r=0
Linear Correlation
Linear relationships
Curvilinear relationships
Y
Y
X
Y
X
Y
X
X
Linear Correlation
Strong relationships
Weak relationships
Y
Y
X
Y
X
Y
X
X
Linear Correlation
No relationship
Y
X
Y
X
Notation for the
Linear Correlation Coefficient
n
number of pairs of sample data
 denotes the addition of the items indicated
 x sum of all x-values
 x indicates that each x-value should be squared and then those squares added
  x  indicates that each x-value should be added and the total then squared
 xy indicates each x-value is multiplied by its corresponding y-value. Then sum those up.
2
2
r
linear correlation coefficient for sample data

linear correlation coefficient for a population of paired data
Calculating by hand…
n
 ( x  x )( y  y )
i 1
cov ariance ( x, y )
rˆ 

var x var y
i
i
n 1
n
n
2
(
x

x
)
 i
2
(
y

y
)
 i
n 1
n 1
i 1
i 1
“Simpler” calculation formula…
Numerator of
covariance
n
 ( x  x )( y  y )
i
i 1
i
n 1
rˆ 
n

n
 (x  x)  ( y  y)
2
i
i 1
i 1
n 1
2
i
n 1
n
 ( x  x )( y  y )
i 1
i
i
n
 (x  x)  ( y  y)
2
i 1


n
i
i 1
2
SS xy
SS x SS y
i
You do not need to memorize any of these formulas!
rˆ 
SS xy
SS x SS y
Numerators of
variance
More r Formula Versions
The linear correlation coefficient r measures the strength
of a linear relationship between the paired values in a
sample. Here are two more formulas that you don’t need
to know:
r=
(
)
n Sxy - (Sx)(Sy)
n(Sx2 ) - (Sx)2 n(Sy2 ) - (Sy)2
z z 


r
x y
n 1
Technology can (and should) compute this.
Requirements for
Linear Correlation
1. The sample of paired (x, y) data is a simple random sample of
quantitative data.
2. Visual examination of the scatterplot must confirm that the points
approximate a straight-line pattern.
3. The outliers must be removed IF they are known to be errors. The
effects of any other outliers should be considered by calculating r with
and without the outliers included.
Interpreting whether r is “significant”
Ancient history approach: Manually look up Table A-6: If
the absolute value of the computed value of r, exceeds
the value in Table A-6, conclude that there is a linear
correlation. Otherwise, there is not sufficient evidence to
support the conclusion of a linear correlation.
Today’s approach using Software: If the computed
statistic “P-value” is less than or equal to the significance
level (usually 5%, meaning there’s at least a 95%
probability of a significant relationship), conclude that
there is a linear correlation. Otherwise, there is not
sufficient evidence to support the conclusion of a linear
correlation.
Caution
While our discussion is focusing on linear correlation, if
you conclude that there does not appear to be linear
correlation, it is still possible that there might be a nonlinear correlation.
For example,
Y = a + bx2
Y = aebx
Properties of the
Linear Correlation Coefficient r
1. – 1 ≤ r ≤ 1
2.
If all values of either variable are converted to a different scale, the value of
r does not change.
3.
Interchanging all x- and y-values will not change r .
4.
r measures strength of a linear relationship.
5.
r is very sensitive to outliers, which can dramatically affect the value of r.
Example
 The paired shoe / height data from five males (n=5)
are listed below. Use a computer to find the value of
the correlation coefficient r.
 Recommend not using a calculator for any statistical
calculations, even if it has these functions, because
it’s time consuming to enter data and easy to make
input mistakes with a calculator.
Example - Continued
Requirement Check: The data are a simple random
sample of quantitative data, the plotted points appear to
roughly (very roughly!) approximate a straight-line
pattern, and there are no outliers. (Some might say
there are outliers here!)
Example - Continued
A few technologies are displayed below, used to
calculate the value of r.
Is There a Linear Correlation?
So, for the shoe and height example, r = 0.591.
Just because we computed a value of r, we must still
decide whether or not this means that there is a linear
correlation between shoe print lengths and heights of
people.
In other words, is there a significant relationship?
We can base our interpretation on a P-value (or a critical
value from Table A-6).
Interpreting the Linear Correlation
Coefficient r
Using computer software:
If the P-value is less than the level of significance,
conclude there is a linear correlation.
Our example with technologies provided a P-value of
0.294.
Because that P-value is greater than the significance
level of 0.05, we conclude there is insufficient evidence
that there is a linear correlation between shoe print length
and heights of people.
Interpreting r:
Explained Variation
• The value of r2 is the proportion of the variation in y that
is explained by the linear relationship between x and y.
• We found previously for the shoe and height example
that r = 0.591.
• With r = 0.591, we get r2 = 0.349.
• We conclude that about 34.9% of the variation in height
can be explained by the linear relationship between
lengths of shoe prints and heights.
• Not exactly an inspiring correlation because there
is a lot of unexplained variation!
Common Errors
Involving Correlation
 Causation:
It is wrong to conclude that correlation implies
causality but that’s exactly what lots of people
erroneously do conclude.
 Averages:
Using averages, instead of the underlying raw
data, suppresses individual variation and may inflate the
correlation coefficient.
 Linearity:
There may be some non-linear relationship
between x and y even when there is no linear correlation.
Formal Hypothesis Test
We wish to determine whether there is a significant linear correlation between
two variables.
Notation:
n = number of pairs of sample data
r = linear correlation coefficient for a sample of paired data
ρ = linear correlation coefficient for a population of paired data
r is an estimate of ρ
Hypothesis Test for Correlation Hypotheses
H0 :   0
(There is no linear correlation.)
H1 :   0
(There is a linear correlation.)
Test Statistic: r
•
Another way of saying Hypothesis Test is that we are
testing for significance.
P-values: Obtain from technology.
Example
We found previously for the shoe and height example that
r = 0.591.
Conduct a formal hypothesis test of the claim that there is
a linear correlation between the two variables.
Use a 0.05 significance level.
P-Value Method for a Hypothesis
Test for Linear Correlation
• For the time being, consider this a “black box”
approach, meaning you accept the result without
knowing how it was derived.
• The test statistic t is below, use n – 2 degrees of
freedom, where n is the number of data points in our
sample.
• Huh, what’s that mean!? It’s a black box! 
• The t value is used with a probability table or technology to
compute the P-value.
t
r
1 r
n2
2
Example

Continuing the same black box example, we calculate the test
statistic:
t
r
1 r2
n2

0.591
1  0.5912
52
 1.269
Technology (or probability table) provides the P-value
(corresponding to t=1.269) as 0.2937.
 Because the P-value of 0.2937 is greater than the significance
level of 0.05, we fail to reject the null hypothesis that r=0.

 There’s nothing quite like statisticians using double negative
language “fail to reject r=0” to convey that there is no significant
correlation relationship.
We conclude there is not sufficient evidence to support the claim
of a linear correlation between shoe print length and heights.
 Hypothetically, if the P-value had been less than 0.05, we would
have rejected the null hypothesis of r=0, concluding that r≠0,
and that there is a significant correlation relationship.

Key Concept
We now find the equation of the straight line that best fits the paired
sample data. That equation algebraically describes the relationship
between two variables.
The best-fitting straight line is called a regression line and its equation is
called the regression equation.
This is one of the most important topics in statistics because it is the
most commonly used approach to make predictions.
While there is some overlap, there are two broad categories of
prediction analysis.
• Usually, when only historical time series data of a variable is used,
we are conducting time series analysis.
• When we analyze relationships among multiple variables, we are
conducting econometric analysis.
What is “Linear”?

Remember this:

Y=mX+B?

m is the slope

B is the Y intercept
m
B
Regression
The regression equation expresses a relationship
between x (called the explanatory variable, predictor
variable or independent variable), and ŷ (called the
response variable or dependent variable).
The typical equation of a straight line y = mx + b is
expressed in the form ŷ = b0 + b1x, where b0 is the yintercept and b1 is the slope.
Definitions
Regression Equation:
Given a collection of paired sample data, the regression line (or line of best
fit) is the straight line that “best” fits the scatterplot of data. The
mathematical method for deriving this line is called the least-squares
method.
The regression equation ŷ = b0 + b1x algebraically describes the regression
line.

Alternative equivalent expressions:

ŷ = a + bx

ŷ = α + βx

ŷ = a 0 + a1 x
Notation for
Regression Equation
Population
Parameter
Sample
Statistic
y-Intercept of
regression equation
β0
b0
Slope of regression
equation
β1
b1
Equation of the
regression line
y = β0 + β1 x
ŷ = b0 + b1x
Coefficient formulas…
Cov( x, y )
ˆ

Var ( x)
Slope (beta coefficient) =
Intercept=
Calculate : ˆ  y - ˆx
Regression line always goes through the point:
( x, y)
Relationship with correlation
SDx
rˆ  ̂
SDy
In correlation, the two variables are treated as equals. In regression, one variable is
considered independent (=predictor) variable (X) and the other the dependent
(=outcome) variable Y.
SD = standard deviation
Example
Let us return to the example from Section 10.2. We would like to use the
explanatory variable, x, shoe print length, to predict the response
variable, y, height.
The data are listed below:
Example - Continued
Requirement Check:
1.
The data are assumed to be a simple random sample.
2.
The scatterplot showed a roughly straight-line pattern.
3.
There are no outliers.
The use of technology is HIGHLY recommended for finding the equation of
a regression line.
Example – Continued
Example - Continued
All these technologies show that the regression equation can be
expressed as:
yˆ  125  1.73x
Now we use the formulas to determine the regression equation
(technology is recommended).
Example
Recall from the previous section that r = 0.591269.
Technology can be used to find the values of the sample means and sample
standard deviations used below.
sy
4.87391
b1  r  0.591269
 1.72745
sx
1.66823
(These are the same coefficients found using technology)
b0  y  b1 x  177.3  1.72745 30.04   125.40740
Example
Graph the regression equation on a scatterplot:
yˆ  125  1.73x
Using the Regression
Equation for Predictions

Use the regression equation for predictions only if the graph of the
regression line on the scatterplot confirms that the regression line fits the
points reasonably well.


We’ll see later what this means.
Use the regression equation for predictions only if the linear correlation
coefficient r indicates that there is a linear correlation between the two
variables (as described in Section 10-2).
Using the Regression
Equation for Predictions

Use the regression line for predictions only if the data do not go much
beyond the scope of the available sample data. (Predicting too far beyond
the scope of the available sample data is called extrapolation, and it could
result in bad predictions.)

If the regression equation does not appear to be useful for making
predictions, the best predicted value of a variable is its sample mean.
Strategy for Predicting Values of y
Using the Regression
Equation for Predictions
If the regression equation is not a good model, the best predicted value of
y is simply, the mean of the y values. y
Remember, this strategy applies to linear patterns of points in a
scatterplot.
Example
Use the 5 pairs of shoe print lengths and heights to predict the height of a
person with a shoe print length of 29 cm.
The regression line does not fit the points well. The correlation is r =
0.591, which suggests there is not a linear correlation (the P-value was
0.294).
The best predicted height is simply the mean of the sample heights:
y  177.3 cm
Example
Use the 40 pairs of shoe print lengths from Data Set 2 in Appendix B to
predict the height of a person with a shoe print length of 29 cm.
Now, the regression line does fit the points well, and the correlation of r
= 0.813 suggests that there is a linear correlation (the P-value is 0.000).
Example - Continued
Using technology we obtain the regression equation and scatterplot:
Example - Continued
The given shoe length of 29 cm is not beyond the scope
of the available data, so substitute in 29 cm into the
regression model:
yˆ  80.9  3.22 x
 80.9  3.22  29 
 174.3 cm
A person with a shoe length of 29 cm is predicted to be
174.3 cm tall.
Definition
In a scatterplot, an outlier is a point lying far away from the other data
points.
Paired sample data may include one or more influential points, which are
points that strongly affect the graph of the regression line.
Example
For the 40 pairs of shoe prints and heights, observe what happens if we
include this additional data point:
x = 35 cm and y = 25 cm
Example - Continued
The additional point is an influential point because the graph of the
regression line because the graph of the regression line changed
considerably.
The additional point is also an outlier because it is far from the other
points.
Jokingly, referred to as: Out Liar!
Definition
For a pair of sample x and y values, the residual is the difference between
the observed sample value of y and the y-value that is predicted by using
the regression equation.
That is:
residual  observed y  predicted y  y  yˆ
Residuals
Definition
A straight line satisfies the least-squares property if the sum of the squares
of the residuals is the smallest sum possible.
Definition
A residual plot is a scatterplot of the (x, y) values after each of the ycoordinate values has been replaced by the residual value y – ŷ (Actual
minus Predicted Value, where ŷ denotes the predicted value of y).
That is, a residual plot is a graph of the points (x, y – ŷ).
Residual Plot Analysis
When analyzing a residual plot, look for a pattern in the
way the points are configured, and use these criteria:
The residual plot should not have any obvious patterns
(not even a straight line pattern). This confirms that the
scatterplot of the sample data is a straight-line pattern.
The residual plot should not become thicker (or thinner)
when viewed from left to right. This confirms the
requirement that for different fixed values of x, the
distributions of the corresponding y values all have the
same standard deviation.
Example
The shoe print and height data are used to generate the following residual
plot:
Example - Continued
The residual plot becomes thicker/wider, which suggests that the
requirement of equal standard deviations is violated.
Example - Continued
On the following slides are three residual plots.
Observe what is good or bad about the individual regression models.
Example - Continued
Regression model is a good model:
Example - Continued
Distinct pattern: sample data may not follow a straight-line pattern.
Example - Continued
Residual plot becoming thicker: equal standard deviations violated.
Complete Regression Analysis
1. Construct a scatterplot and verify that the pattern of the points is
approximately a straight-line pattern without outliers. (If there are
outliers, consider their effects by comparing results that include
the outliers to results that exclude the outliers.)
2. Construct a residual plot and verify that there is no pattern (other
than a straight-line pattern) and also verify that the residual plot
does not become thicker (or thinner).
3. Use a histogram and/or normal quantile plot to confirm that the
values of the residuals have a distribution that is approximately
normal.
4. Consider any effects of a pattern over time.
Residual = observed - predicted
X=95
nmol/L
34
yi  48
yˆ i  34
yi  yˆ i  14
Residual Analysis for Linearity
Y
Y
x
x
Not Linear
residuals
residuals
x
x

Linear
Slide from: Statistics for Managers Using Microsoft® Excel 4th Edition, 2004 Prentice-Hall

Residual Analysis for
Homoscedasticity (constant variance)
Y
Y
x
x
Non-constant variance
residuals
residuals
x
x

Constant variance
Slide from: Statistics for Managers Using Microsoft® Excel 4th Edition, 2004 Prentice-Hall

Residual Analysis for Independence
(each point is unrelated to other points)
Not Independent
X
residuals
residuals
X
residuals
 Independent
X
Slide from: Statistics for Managers Using Microsoft® Excel 4th Edition, 2004 Prentice-Hall

Explained and Unexplained Variation
Assume the following:
•There is sufficient evidence of a
linear correlation.
•The equation of the line is
ŷ = 3 + 2x
•The mean of the y-values is 9.
•One of the pairs of sample data
is x = 5 and y = 19.
•The point (5,13) is on the
regression line.
Explained and Unexplained Variation
The figure shows (5,13) lies on the regression line, but (5,19) does
not.
We arrive at:
Total Deviation (from y  9) of the point (5, 19)  y  y  19  9  10.
Explained Deviation (from y  9) of the point (5, 19)  yˆ  y  13  9  4.
Unexplained Deviation (from y  9) of the point (5, 19)  y  yˆ  19  13  6.
Relationships
(total deviation) = (explained deviation) + (unexplained deviation)
( y  y)
=
( yˆ  y )
+
( y  yˆ )
(total variation) = (explained variation) + (unexplained variation)
( y  y )
2
=
2
ˆ
( y  y )
+
( y  yˆ ) 2
Definition
The coefficient of determination is the amount of the variation in
y that is explained by the regression line.
explained variation
r 
total variation
2
The value of r2 is the proportion of the variation in y that is
explained by the linear relationship between x and y.
Example
If we use the 40 paired shoe lengths and heights from the data in
Appendix B, we find the linear correlation coefficient to be r = 0.813.
Then, the coefficient of determination is
r2 = 0.8132 = 0.661
We conclude that 66.1% of the total variation in height can be explained
by shoe print length, and the other 33.9% cannot be explained by shoe
print length.
Prediction or Confidence Limits

The prediction values of a regression equation are uncertain and subject to estimation
error.

Many regression software programs (Statcrunch in MyLab or PredInt.xls) will compute a
statistical range (lower and upper values for each predicted value) around the
regression prediction values that indicate with high (95%) confidence that the predicted
value should fall within that range.
The level C prediction interval for a single observation on y when x takes
the value x* is:
ŷ ± t*n − 2 SEŷ
t* for t distribution with n – 2 df
95% prediction
interval for ŷ
The prediction interval represents
mainly the error from the normal
distribution of the residuals ei.
Graphically, a series of confidence
intervals for the whole range of x
values is shown as a continuous
interval on either side of ŷ.
Key Concept
This section presents a method for analyzing a linear
relationship involving more than two variables.
We focus on these key elements:
1.
Finding the multiple regression equation.
2.
The values of the adjusted R2, and the P-value as
measures of how well the multiple regression
equation fits the sample data.
Definition
A multiple regression equation expresses a linear
relationship between a response variable y and two or
more predictor variables
( x1 , x2 , x3 ..., xk )
The general form of the multiple regression equation
obtained from sample data is
yˆ  b0  b1 x1  b2 x2  ...  bk xk
Notation
yˆ  b0  b1 x1  b2 x2  ...  bk xk
(General form of the multiple regression equation)
n = sample size
k = number of predictor variables
ŷ = predicted value of y
( x1 , x2 , x3 ..., xk ) are the predictor variables
When trying to explain sales, plausible independent variables
include prices, economic factors, advertising intensity, and
consumers’ incomes in the area
Technology
Use a statistical software package that can do “stepwise”
multiple regression (software picks the best variables in the
regression) such as

SAS

SPSS

Excel, Statcrunch, STATDISK– NOT
recommended, NO stepwise feature.
Example
Table 10-4 includes a random sample
of heights of mothers, fathers, and their
daughters (based on data from the
National Health and Nutrition
Examination).
Find the multiple regression equation in
which the response (y) variable is the
height of a daughter and the predictor
(x) variables are the height of the
mother and height of the father.
Example
The Minitab results are shown here:
Definition

The multiple coefficient of determination, R2, is a measure of how well
the multiple regression equation fits the sample data. A perfect fit would
result in R2 = 1.

The adjusted coefficient of determination, R2, is the multiple coefficient
of determination modified to account for the number of variables and the
sample size.

More appropriate to use for multiple regression.
Adjusted Coefficient of Determination
Adjusted
where
(n  1)
2
R  1
(1  R )
[n  (k  1)]
2
n = sample size
k = number of predictor (x) variables
Example
The preceding technology display shows the adjusted coefficient of
determination as R-Sq(adj) = 63.7%.
When we compare this multiple regression equation to others, it is
better to use the adjusted R2 of 63.7%
P-Value
The P-value is a measure of the overall significance of the multiple regression
equation.
The displayed technology P-value of 0.000 is small, indicating that the multiple
regression equation has good overall significance and is usable for predictions.
That is, it makes sense to predict the heights of daughters based on heights of
mothers and fathers.
The value of 0.000 results from a test of the null hypothesis that β1 = β2 = 0, and
rejection of this hypothesis indicates the equation is effective in predicting the
heights of daughters.
Finding the Best Multiple
Regression Equation
1.
Use common sense and practical considerations to include or exclude variables.
2.
Consider the P-value. Select an equation having overall significance, as determined by
the P-value found in the computer display.
3.
Consider equations with high values of adjusted R2 and try to include only a few
variables.
•
If an additional predictor variable is included, the value of adjusted R2
does not increase by a substantial amount.
•
For a given number of predictor (x) variables, select the equation with the
largest value of adjusted R2.
•
In weeding out predictor (x) variables that don’t have much of an effect
on the response (y) variable, it might be helpful to find the linear
correlation coefficient r for each of the paired variables being
considered.
Example
Data Set 2 in Appendix B includes the age, foot length, shoe print
length, shoe size, and height for each of 40 different subjects.
Using those sample data, find the regression equation that is the best
for predicting height.
The table on the next slide includes key results from the
combinations of the five predictor variables.
Example - Continued
In practice, software packages like SPSS will use “stepwise” regression methods
to find the best set of independent variables in a multiple regression.
Example - Continued
Using critical thinking and statistical analysis:
1.
Delete the variable age.
2.
Delete the variable shoe size, because it is really a rounded form
of foot length.
3.
For the remaining variables of foot length and shoe print length,
select foot length because its adjusted R2 of 0.7014 is greater
than 0.6520 for shoe print length.
4.
Although it appears that only foot length is best, we note that
criminals usually wear shoes, so shoe print lengths are likely to be
found than foot lengths.
Multicollinearity

Multicollinearity (or intercorrelation) exists when at least some of the predictor
variables are correlated among themselves.

In observational studies, multicollinearity happens more often than not.

Essentially, perfect multicollinearity ruins our ability to estimate the coefficients
because the perfectly collinear variables cannot be distinguished from each
other:
•


You cannot “hold all the other independent variables in the equation constant” if every
time one variable changes, another changes in an identical manner!
Estimates will become very sensitive to changes in specification:
a.
The addition or deletion of an explanatory variable or of a few observations will
often cause major changes in the values of the coefficients when significant
multicollinearity exists
b.
For example, if you drop a variable, even one that appears to be statistically
insignificant, the coefficients of the remaining variables in the equation sometimes
will change dramatically
c.
This is again because with multicollinearity, it is much harder to distinguish the
effect of one variable from the effect of another
Solution: one of the collinear variables must be dropped (they are essentially
identical, anyway)
Severe Multicollinearity Increases the
Variances of the s
© 2011 Pearson Addison-Wesley. All rights reserved.
8-94
Detecting Multicollinearity
The easiest way to measure the extent of multicollinearity is simply to
look at the matrix of correlations between the individual variables.
 How high is high correlation?

 Some researchers pick an arbitrary number, such as 0.80

A better test for multicollinearity in SPSS:
 https://www.spsstests.com/2015/03/multicollinearity-test-example-
using.html
Example
We have quarterly data for
Imports (IMP)
Gross Domestic Product (GDP)
Consumer Price Index (CPI) and
Producer Price Index (PPI)
Examples
Correlation Matrix
IMP
GDP
CPI
PPI

IMP
1
0.979
0.916
0.883
GDP
0.979
1
0.910
0.8998
CPI
0.916
0.910
1
0.981
PPI
0.883
0.899
0.981
1
All the predictor variables have high correlation with each other.

Probably best to use just single variable rather than multiple regression.
Logarithmic transformations of variables

Logarithmically transforming variables in a regression model is a very common
way to handle situations where there is a non-linear relationship.


Logarithmic transformations are also a convenient means of transforming a
highly skewed variable (a lopsided probability distribution) into one that is
more approximately normal.


For example, exponential growth.
In fact, there is a probability distribution called the log-normal distribution
defined as a distribution whose logarithm is normally distributed – but whose
untransformed scale is skewed.
Prefer to use base 10 logarithms, log not ln!
Log transformation examples

Actually, only 3 transformations.

Consider exponential growth:


Sales = a*ebt transforms to log(Sales) = log a + bt = a’ + bt
Dummy Variable
Many applications involve a dichotomous variable which
has only two possible discrete values (such as
male/female, dead/alive, etc.).
A common procedure is to represent the two possible
discrete values by 0 and 1, where 0 represents “failure”
and 1 represents “success”.
A dichotomous variable with the two values 0 and 1 is
called a dummy variable.
Example
The data in the table also includes the dummy variable of
sex (coded as 0 = female and 1 = male).
Given that a mother is 63 inches tall and a father is 69
inches tall, find the regression equation and use it to
predict the height of a daughter and a son.
Example
Example - Continued
Using technology, we get the regression equation:
We substitute in 0 for the sex, 63 for the mother, and 69 for the
father, and predict the daughter will be 62.8 inches tall.
We substitute in 1 for the sex, 63 for the mother, and 69 for the
father, and predict the son will be 67 inches tall.
Note: In log transformation of independent dummy variable,
one can use very small value instead of zero, so dummy
equals either 10^-10 or 1.
Seasonal Dummy Variables SKIP
 Often when performing a time series regression, we want
to consider seasonal effects, typically indicated by the
month or quarter of the year.
 See 10 CODING SEASONAL DUMMY.xlsx
 In SPSS, you may need to convert dates or time periods
into the proper month or quarter variable name and then
use the Create Dummy Variable tool, as shown in a video
later.
Seasonal Dummy Variables
Alternatively, we can use commands in SPSS Syntax:
For months
compute Jan = 0.
compute Feb = 0.
compute Mar = 0.
compute Apr = 0.
compute May = 0.
compute Jun = 0.
compute Jul = 0.
compute Aug = 0.
compute Sep = 0.
compute Oct = 0.
compute Nov = 0.
compute Dec = 0.
IF (MonthX = 1) Jan = 1.
IF (MonthX = 2) Feb = 1.
IF (MonthX = 3) Mar = 1.
IF (MonthX = 4) Apr = 1.
IF (MonthX = 5) May = 1.
IF (MonthX = 6) Jun = 1.
IF (MonthX = 7) Jul = 1.
IF (MonthX = 8) Aug = 1.
IF (MonthX = 9) Sep = 1.
IF (MonthX = 10) Oct = 1.
IF (MonthX = 11) Nov = 1.
IF (MonthX = 12) Dec = 1.
Execute.

 For quarters:
compute Q1 = 0.
compute Q2 = 0.
compute Q3 = 0.
compute Q4 = 0.
IF (quarter = 1) Q1 = 1.
IF (quarter = 2) Q2 = 1.
IF (quarter = 3) Q3 = 1.
IF (quarter = 4) Q4 = 1.
Execute.
Logistic Regression
We can use the methods of this section if the dummy
variable is the predicted variable.
If the dummy variable is the response/dependent
variable we need to use a method known as logistic
regression.
As the name implies logistic regression involves the use
of natural logarithms.
This textbook does not include coverage of this
advanced topic nor detailed procedures for using logistic
regression (there is a brief example in Chapter 10 of the
text).
We will learn a very basic introduction to the concept and
not how to use or interpret software.
Introduction
 Logistic regression is a form of regression
analysis in which the outcome variable is binary
or dichotomous
 Used when the focus is on whether or not an
event occurred.
What is Logistic Regression?
100 day Mortality
CHD (Died=1, Alive=0)
1.0
0.8
0.6
0.4
0.2
0.0
20
30
40
50
Age
Age
(yrs.)
60
70
The Logistic Curve
p (probability)
exp  z 
 p 
LOGIT ( p)  ln 

z

p


1  exp  z 
 (1  p) 
p
LOGIT ( p)  z  log
p
(1  p)
z (log odds)
exp( z )
1  exp( z )
Logistic Regression

Simple logistic regression = logistic regression with 1
predictor variable

Multiple logistic regression = logistic regression with multiple
predictor variables

Multiple logistic regression =
Multivariable logistic regression =
Multivariate logistic regression
The Logistic Regression Model
Logistic Regression:
 P Y 
ln 
  0  1 X 1   2 X 2 

 1-P  Y  


 K X K
Linear Regression:
Y   0  1 X 1   2 X 2 
 K X K  e
The Logistic Regression Model
predictor variables
 P Y 
ln 
  0  1 X 1   2 X 2 

 1-P  Y  


dichotomous outcome
 PY   is the log(odds) of the outcome.

ln 
 1  PY  
 K X K
The Logistic Regression Model
 P Y 
ln 
  0  1 X 1   2 X 2 

 1-P  Y  


intercept
model coefficients
 PY   is the log(odds) of the outcome.

ln 
 1  PY  
 K X K
Relationship between
Odds & Probability
Odds  event  =
Probability  event 
1-Probability  event 
Probability  event  
Odds  event 
1+Odds  event 
Form for Predicted Probabilities
 P Y 
ln 
  0  1 X 1   2 X 2 

 1-P  Y  


 K X K
exp   0  1 X 1   2 X 2 
 K X K 
P Y 
1  exp   0  1 X 1   2 X 2 
 K X K 
In this latter form, the logistic regression model directly relates the
probability of Y to the predictor variables.
Logistic Regression Results


Overall goodness

Chi Square P-value should be less than .05

Pseudo R-square: similar concept to regular regression R-square
Individual variables


Coefficient P-value should be less than .05
A more technical explanation based Stata software

https://stats.idre.ucla.edu/stata/output/logistic-regression-analysis/
Binary logistic regression
Dependent variable
Explanatory variables
It is possible to
opt for step-wise
selection of
explanatory
variables
Declare categorical
variables
Additional
statistics
Save predicted values or residuals
117
Additional statistics
The test compares
the expected
frequencies with
those actually
observed after
dividing the subject
in ten equal groups
according to their
predicted
probabilities
In logistic regression, the
exponential function of
the coefficients are odds
ratio – this option
provides confidence
intervals
118
Output
Model Summary
Step
1
-2 Log
Cox & Snell
likelihood
R Square
a
467.079
.157
These are goodness-of-fit measures
similar to the regression R square
Nagelkerke
R Square
.217
a. Es timation terminated at iteration number 5 because
parameter estimates changed by less than .001.
Hosmer and Lemeshow Test
Step
Chi-square
1
3.030
df
The hypothesis of equality between
observed and predicted frequencies is not
rejected
Sig.
8
.932
Classification Table(a)
Predicted
Butcher
Observed
Step 1
Butcher
no
yes
Percentage
Correct
No
243
34
87.7
Yes
89
54
37.8
Overall Percentage
70.7
The classification
table shows the
frequencies of
correctly predicted
observations
a The cut value is .500
119
Coefficient estimates
These are odds ratios and are interpreted as follows: A one-year
increase in age (q51) leads to a 2.2% increase in the odds of
purchasing chicken at the butcher shop (i.e. the ratio between the
probability of doing it and the probability of not doing it)
Variables in the Equation
Step
a
1
q51
q43b
q21d
q5
Constant
B
.022
-.269
.441
.085
-3.169
S.E.
.007
.074
.077
.028
.615
Wald
8.988
13.327
32.888
8.975
26.539
df
1
1
1
1
1
Sig.
.003
.000
.000
.003
.000
Exp(B)
1.022
.764
1.554
1.088
.042
a. Variable(s) entered on step 1: q51, q43b, q21d, q5.
As requested, 95% confidence
intervals for the odds ratio are shown
A unit increase in trust in supermarket (q43b)
decreases the same odds ratio by 23.6%.
The safety perception is the most relevant
covariate as a unitary increase improves the
odds ratio by 55.4%
All predictors
are significantly
different from
zero
120
95.0% C.I.for EXP(B)
Lower
Upper
1.008
1.037
.661
.883
1.337
1.807
1.030
1.150
Logistic Regression Software

Best: SAS or SPSS


Convenient:


But harder for beginners to learn
Statcrunch in MyLab, expires with MyLab
XLSTAT – an Excel add-in that can be used free for temporary trial period
https://www.xlstat.com/en/

Real Statistics Using Excel website
http://www.real-statistics.com/
Time Series Models

Forecaster looks for data patterns as


Data = historic pattern + random variation
Historic pattern to be forecasted:

Level (long-term average) – data fluctuates around a constant mean

Trend – data exhibits an increasing or decreasing pattern

Seasonality – any pattern that regularly repeats itself and is of a constant
length

Cycle – patterns created by economic fluctuations

Random Variation cannot be predicted

Not useful if circumstances change
 For example, your or competitor price change
Time Series Patterns
123
Time Series Models
 Naive:
 The forecast is equal to the actual value observed during
the last period – good for level patterns
 Simple Mean:
Ft 1  At
 The average of all available data - good for level patterns
 Moving Average:
Ft 1   At / n
 The average value over a set time period
(e.g.: the last four weeks)
Ft 1   At / n
 Each new forecast drops the oldest data point & adds a
new observation
 More responsive to a trend but still lags behind actual data
Time Series Models con’t SKIP
 Exponential Smoothing:
Most frequently used time series method because of ease of use and
minimal amount of data needed
 Need just three pieces of data to start:
 Last period’s forecast (Ft)
 Last periods actual value (At)
Ft 1  αA t  1  α Ft
 Select value of smoothing coefficient,
 , between 0 and 1.0
 If no last period forecast is available, average the last few periods
or use naive method
 Higher values (e.g., .7 or .8) may place too much weight on last
period’s random variation
Time Series Problem

Determine forecast for periods
7&8

2-period moving average

4-period moving average

2-period weighted moving
average with t-1 weighted 0.6
and t-2 weighted 0.4

Exponential smoothing with
alpha=0.2 and the period 6
forecast being 375
Period
1
2
Actual
300
315
3
4
5
290
345
320
6
7
8
360
375
Time Series Problem Solution
Period
Actual
1
300
2
315
3
290
4
345
5
320
6
360
7
375
8
2-Period
4-Period
2-Per.Wgted.
Expon. Smooth.
340.0
328.8
344.0
372.0
367.5
350.0
369.0
372.6
Linear Trend Line
A time series technique that computes a forecast with trend by drawing a straight
line through a set of data using this formula:
Y = a + bx where
Y = forecast for period X
X = the number of time periods from X = 0
A = value of y at X = 0 (Y intercept)
B = slope of the line
Simple Seasonality Forecasting SKIP
 Use regression trend line with seasonal
dummy variables.
Measures of Forecast Error
Actual – Forecast
( Forecast Error  Yt  Ft )


Bias - The arithmetic sum of the errors or the
average error shown here. Large positive or
negative indicates under or over forecasting.
Bias   (forecast error) /T
t 1
T
  (Yt  Ft ) / T
Mean Square Error (MSE) - Similar to simple
sample variance

MAD - Mean Absolute Deviation, popular metric

MAPE – Mean Absolute Percentage Error, more
understandable percentage terms

T
t 1
T
MSE   | forecast error |2 /T
t 1
T
  (Yt  Ft ) 2 / T
Tracking Signal rule of thumb: larger than +/- 4
indicates forecasts are exhibiting bias.
t 1
T
T
t 1
t 1
MAD   | forecast error | /T   |Yt  Ft | / T
T
MAPE  100 [|Yt  Ft | / Yt ] / T
t 1
Tracking Signal = Accumulated Forecast Errors / Mean Absolute Deviation
How does this apply to marketing?

The core approach to making marketing predictions about markets, products
and customer behavior. For example,

Forecasting market size

Forecasting sales
View videos and spreadsheets

SPSS

Using Create Dummy Variable tool
https://www.youtube.com/watch?v=0y0q-lKiPYk starting around 3:20 minutes in video

Multiple Stepwise Regression
https://www.youtube.com/watch?v=eme0ErU7GJA

Scatterplots and trend line
https://www.youtube.com/watch?v=gQ1_gxzYhZ8

Logistic regression
https://www.youtube.com/watch?v=-2bol1PoqKg

Time Series Analysis
https://www.youtube.com/watch?v=sFH6tgylE2k

Statdisk Regression
https://www.youtube.com/watch?v=1zuzSHzKwNU

Statcrunch Prediction Interval
https://www.youtube.com/watch?v=XfsH_LzpnPo

10 statspearman.xlsx

10 PredInt.xls

http://www.danielsoper.com/statcalc/default.aspx

5 Statistical Functions and Demos.xlsx
Group Exercise
1.
Form a group of maximum 4 students to each side of you
and/or in front/back of you. Appoint one person to do the
work while everyone else supervises and gives
(contradictory) instructions. 

Using the height/sex data from Triola dataset # 10 –
Family Heights.xlsx, develop a regression using SPSS
predicting daughter.

HURRY!!

Describe your findings.

What did you find difficult to do?
Extra Homework SKIP





1.
2.
3.
4.
5.
6.
Use 10 Mrs Smyths Pies.xlsx and post results in Forums.
Create summary statistics for each variable.
Prepare complete multiple regression analysis of sales, preferably using stepwise method.
 This analysis requires both seasonal and region dummy variables.
Show scatterplot for residuals and dependent variable vs. each independent variable and
time. Scatterplots vs. dummy variables are usually not informative.
Use the regression results to forecast unit sales in the last region for the next quarter under
the following assumptions:
 P = $7.20
 A = $30,000
 PX = $6.00
 Y = $43,000
 Pop: Same as in prior period.
According to your output, if Mrs. Smyth’s lowered its price by $0.75 what would be the
impact on sales? Show your work.
According to your output, if Mrs. Smyth’s increased its advertising budget by $15,000 what
would be the impact on sales? Show your work.
According to your output, if a competitor lowered its price by $1.25 what would be the
impact on Mrs. Smyth’s frozen pie sales. Show your work.
Using the median values for your variables, predict sales for the first quarter of 2005 and for
the third quarter of 2005. Show your work. Do you think this estimate is reliable? Briefly
explain.
Determine the price elasticity of demand for frozen pies at the median price and quantity.
Are frozen pies elastic or inelastic according to your estimate? Show your work.
Determine the income elasticity of frozen pies at the median income and quantity. What
information does this provide about how consumers view frozen pies? Show your work.
Extra Homework

Chapter 10 HW
Extra Homework

Using the height/sex data from Triola dataset # 10 – Family Heights.xlsx,
develop a regression using SPSS predicting daughter.

Create summary statistics for each variable.

Prepare complete regression analysis.

Show scatterplots for dependent variable vs. each independent one.

Perform scatterplot residual analysis.
Extra Homework

Run time series regression on Audi monthly car sales including the final missing months.

10 AUDI SALES 2022.xlsx

Create summary statistics for each variable.

Prepare complete regression analysis.

Show data scatterplot and time trend.

Perform scatterplot residual analysis.

Use regression equation to:


predict the final months’ sales

compute prediction limits and

compare to actual sales.
5 volunteers for different regressions
1.
Sales = a + bt
2.
Sales = a + bt + bt2 + bt3
3.
Log-linear
4.
Linear-log
5.
Log-log
Extra Homework

Download and use the All Greens franchise dataset at:

http://college.cengage.com/mathematics/brase/understandable_statistics/7
e/students/datasets/mlr/frames/frame.html

Create summary statistics for each variable.

Prepare complete regression analysis.

Show scatterplots for dependent variable vs. each independent one.

Perform scatterplot residual analysis.
Extra Homework

Use 10 Sample Regressions 1-3.xlsx

Do regressions on tabs 1 to 3

Create summary statistics for each variable.

Prepare complete regression analysis of sales.

Show scatterplots for dependent variable vs. each independent one.

Perform scatterplot residual analysis.
Extra Homework

Use 10 cranberrydata.xlsx

Create summary statistics for each variable.

Prepare complete regression analysis of sales using the first 20 data points.

Show scatterplots for dependent variable vs. each independent one and time.

Perform scatterplot residual analysis.

Predict sales for the remaining data points together with prediction
confidence limits.
Extra Homework

Use 10 Promotiondata.xlsx

Create summary statistics for each variable.

Determine whether seasonality or promotion have a significant effect on
sales.

Show scatterplots for dependent variable vs. each independent one and time.

Perform scatterplot residual analysis.
Extra Homework

Use 10 Sports Info Service.xlsx

Create summary statistics for each variable.

Prepare complete logistic regression analysis to explain renewals.

Show scatterplot for dependent variable vs. each independent one.

Explain the output results and show relevant charts.
Discuss Team Project