ISyE 3030
Basic Statistical Methods
Other topics about regression (chp 11, 12)
Instructor: Professor Jing Li
H. Milton Stewart School of
Industrial and Systems Engineering
Georgia Tech
1
Correlation (chp 11.8)
2
Correlation coefficient
3
Hypothesis Test
• Testing whether or not there is
• Test statistic
4
Example 11.8
5
Example 11.8
6
Example 11.8
7
Regression on transformed variables
(chp 11.9)
8
Regression on transformed variables
• Deal with non-linearity: Sometimes visual
inspections, or prior knowledge, tells us that
there are some non-linear factors in regression
model
• Examples:
9
Example 11.9: Wind-mill power
10
Try fitting a linear model?
• Result of fitting linear regression model
Residual
• Residual plot indicates the linear
relationship does not capture all
the information in the wind-speed
variable.
11
A second try
• As wind speed increases, output (y) approach to
an upper limit (consist with physics of windmill
operation)
Raw data
Transformed data
12
Residual: Linear model
Residual: Transformed data model
13
14
Logistic regression (chp 11.10)
15
Logistic regression
𝑌𝑌𝑖𝑖 = 𝛽𝛽0 + 𝛽𝛽1 𝑥𝑥𝑖𝑖 + 𝜀𝜀𝑖𝑖
𝐸𝐸 𝑌𝑌𝑖𝑖 =
1
1 + 𝑒𝑒𝑒𝑒𝑒𝑒 − 𝛽𝛽0 + 𝛽𝛽1 𝑥𝑥𝑖𝑖
Linear regression does not work because the assumptions are not met.
Logistic regression
To estimate the logistic regression coefficients, we can use maximum likelihood estimation
(details skipped).
16
Example
17
Example
18
Example
Fitted logistic regression
𝐸𝐸 𝑌𝑌𝑖𝑖 =
1
1 + 𝑒𝑒𝑒𝑒𝑒𝑒 − 10.875 − 0.17132𝑥𝑥𝑖𝑖
19
Polynomial regression (chp 12.6.1)
20
Polynomial regression
21
Example 12.12
22
Example 12.12
23
Example 12.12
24
Categorical regressors (chp 12.6.2)
25
Categorical regressors
26
Example 12.13
27
Example 12.13
28
Example 12.13
29