Daniel S. Yates
The Practice of Statistics
Third Edition
Chapter 4:
More about Relationships
between Two Variables
Copyright © 2008 by W. H. Freeman & Company
Section 4.1 Modeling Nonlinear Data
Linear relationship
x
y
Dy/Dx
0
2.0
5.1
1
7.1
2
11.8
3
17.1
4
22.2
4.7
5.3
5.1
y 2  y1
x 2  x1

Dy
Dx

7 .1  2 .0
1 0
Constant value
Dy/Dx indicate
linear relationship
y = a + bx
 5 .1
Section 4.1 Modeling Nonlinear Data
Exponential relationship
x
y
0
1.0
1
2.1
2
4.3
3
7.9
4
16.0
yn/yn-1
2.10
2.05
1.84
2.03
y2
y1

2 .1
1 .0
Constant value yn/yn-1
called common ratio
indicates exponential
relationship
y = abx
Section 4.1 Modeling Nonlinear Data
Power relationship
x
y
0
0.0
1
2.1
2
16.2
3
53.5
4
127.4
yn/yn-1 Dy/Dx
7.7
2.1
14.1
Neither yn/yn-1 or
Dy/Dx are constant
indicates possible
power relationship
3.3
37.3
y = axb
2.38
73.9
Section 4.1 Modeling Nonlinear Data
•Many important real world situations
exhibit exponential or power relationships.
•Exponential and power relationships can
be transformed into linear forms so linear
regression analysis can be utilized.
• Linear regression only works for linear
models. (That sounds obvious, but when
you fit a regression, you can’t take it for
granted.)
• A curved relationship between two variables
might not be apparent when looking at a
scatterplot alone, but will be more obvious
in a plot of the residuals.
– Remember, we want to see “nothing” in a plot
of the residuals.
• No regression analysis is complete without
a display of the residuals to check that the
linear model is reasonable.
• Residuals often reveal subtleties that were
not clear from a plot of the original data.
Section 4.1 Modeling Nonlinear Data
• For exponential relationship - logy is linear with
respect to x
• For power relationship - logy is linear with respect to logx
Transforming Exponential Data
Steps
Year
Cell Phone Users
1986
503
1)
Graph data
1987
890
2)
1988
1545
Check common ratio if you suspect
exponential relationship
1989
2701
3)
Create new list with log of the y values
1990
4734
4)
Graph data. X vs log Y
1991
8345
5)
1992
14356
Perform linear regression on the
transformed data. Store equ. in Y1
1993
25019
6)
1994
45673
Transform back by taking raising 10 to
both sides of the equation.
7)
Graph data to check
 480 . 727179
0 . 2434186
ˆ
Y  10
* 10
X
• Linear models give a predicted value for
each case in the data.
• We cannot assume that a linear relationship
in the data exists beyond the range of the
data.
• Once we venture into new x territory, such a
prediction is called an extrapolation.
Section 4.2 Interpreting Correlation and Regression
• r and LSRL describe only linear relationships
• r and LSRL are strongly influenced by a few extreme
observations – influential points
• Always plot your data
• The use of a regression line to predict outside the domain
of values of the explanatory variable x is called
extrapolation and cannot be trusted.
Lurking Variables and Causation
• No matter how strong the association, no matter how large the
R2 value, no matter how straight the line, there is no way to
conclude from a regression alone that one variable causes the
other.
– There’s always the possibility that some third variable is
driving both of the variables you have observed.
• With observational data, as opposed to data from a designed
experiment, there is no way to be sure that a lurking variable is
not the cause of any apparent association.
Section 4.2 Interpreting Correlation and Regression
• Lurking variables are variables that can influence the
relationship of two variables.
• Lurking variables are not measured or even considered.
• Lurking variables can falsely suggest a strong relationship
between two variables or even hide a relationship.
Lurking Variables and Causation (cont.)
• The following scatterplot shows that the average life
expectancy for a country is related to the number of
doctors per person in that country:
Lurking Variables and Causation (cont.)
• This new scatterplot shows that the average life expectancy
for a country is related to the number of televisions per
person in that country:
Lurking Variables and Causation (cont.)
• Since televisions are cheaper than doctors, send TVs to
countries with low life expectancies in order to extend
lifetimes. Right?
• How about considering a lurking variable? That makes more
sense…
– Countries with higher standards of living have both longer
life expectancies and more doctors (and TVs!).
– If higher living standards cause changes in these other
variables, improving living standards might be expected to
prolong lives and increase the numbers of doctors and TVs.
y = 0.9716x + 194.48
R2 = 0.3658
New England Patriots
400
350
Weight (lbs.)
300
250
200
150
100
50
0
0
20
40
60
Jersey Number
80
100
120
• Strong association of variables x and y can reflect
any of the following underlying relationships
– Causation - changes in x cause changes in y
– ex. Consuming more calories with no change in
physical activity causes weight gain.
– Common response – both x and y respond to some
unobserved variable or variables.
– ex. There may be perceived cause and effect between
SAT scores and undergrad GPA but both variables are
likely responding to student knowledge and ability
– Confounding – the effect on y of the
explanatory variable x is mixed up with the
effects on y of other lurking variables.
– ex. Minority students have lower ave. SAT
scores than whites; but minorities on average
grew up in poorer households and attended
poorer schools. These socioeconomic variables
make cause and effect suspect.
Strong Association
Strong Association
Strong Association
• A carefully designed experiment is the best
way to get evidence that x causes y.
Lurking variables must be kept under
control.
Section 4.3 Relations in Categorical Data
• Categorical data may be inherently
categorical such as; sex,race and
occupation.
• Categorical data may be created by
grouping quantitative data.
• Two way tables – hold categorical data
example
Income
019,999
20,00039,999
40,00049,999
Total
Age Group
25-34
4,506
35-54
2,738
55 +
3,400
Total
10,644
8,724
5,622
4,789
19,135
12,643
16,893
7,642
37,178
25,873
25,253
15,831
66,957
Row variable – Income Column variable - Age
• The totals of the rows and column are called
marginal distributions.
• The totals may be off from the table data
due to rounding error.
• The data may also be represented by
percents.
• Relationships between categorical data may
be calculated from the two way table.
• Data may be represented by a bar chart.
• Conditional distributions satisfy a certain
condition on the table.
– Ex. Distribution of income level for 25-34 year
olds.
– Ex. Distribution of age for people making
$20,000 - $39,999
Example
Outcome Hospital
A
Hospital
B
Total
Died
63
(3%)
16
(2%)
79
Survived
2037
(97%)
784
(98%)
2,821
Total
2,100
800
2,900
Lurking Variable
Good Condition
Poor Condition
Outcome
Hospital
A
Hospital
B
Hospital
A
Hospital
B
Died
6
(1%)
8
(1.3%)
57
(3.8%)
8
(4%)
Survived
594
(99%)
592
(98.7%)
1,443
(96.2%)
192
(96%)
Total
600
600
1,500
200