Introduction to interactions in
regression models:
Concepts and equations
Jane E. Miller, PhD
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Overview
• What is an interaction?
– Definitions
– Synonyms
• Model specifications: equations for
– Main-effects-only models
– Model with interactions
• Illustrative charts
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
What is an interaction?
• The association between one independent
variable (X1) and the dependent variable (Y)
differs depending on the value of a second
independent variable (X2).
• Can be thought of as an exception to a general
pattern:
– X1 is associated with Y in one way when X2 = 1, but in
a different way when X2 = 2.
• X1 is sometimes termed the “focal predictor”
• X2 is referred to as the “modifier” or “modifying variable.”
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Statistical interactions defined
• When X1 and X2 not only potentially have
separate effects on Y, but also have a joint effect
that is different from the simple sum of their
respective individual effects.
– The association between X1and Y is conditional on
X2.
– The specific combinations of values of X1 and X2
determine the value of Y.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Three general
shapes of interaction patterns
1. Size: The effect of X1 on Y is larger for some
values of X2 than for others;
2. Direction: the effect of X1 on Y is positive for
some values of X2 but negative for other values
of X2;
3. The effect of X1 on Y is non-zero (either positive
or negative) for some values of X2 but is not
statistically significantly different from zero for
other values of X2.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Example interaction topics:
Magnitude
• The interaction can occur in terms of magnitude.
– The size of the association between X1 and Y
depends on values of X2.
• Birth weight increases more rapidly with family
income for non-Hispanic white than for Latino
infants.
– The steepness of the income (X1)/birth weight (Y)
gradient depends on ethnicity (X2).
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Example interaction topics:
Direction
• The interaction can occur in
terms of direction.
– The direction of the association
between X1 and Y depends on
values of X2.
• Being married is associated
with higher earnings for men,
but lower earnings for women.
– The association between marital
status (X1) and earnings (Y)
works in opposite directions for
each of the two genders (X2).
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Example interaction topics:
Effect for some but not all groups
• The interaction can occur in terms of magnitude.
– The association between X1 and Y is statistically
significant only for some values of X2.
• The harmful effect of secondhand smoke (X1) on
childhood asthma (Y) is ameliorated if the child
was breast-fed (X2 =1) but remains for children
who were not breast-fed (X2 =2).
– Breast-feeding modifies the smoke/asthma
association.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Synonyms for “interaction”
• Terminology for interactions varies by discipline.
• Common synonyms include:
– Effects modification
– Moderating effect
– Modifying effect
– Joint effect
– Contingency effect
– Conditioning effect
– Heterogeneity of effects
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Recognizing when an interaction
specification should be tested
• Could be based on
– Theory of how X1, X2, and Y are related to one
another.
• E.g., different mechanisms linking X1 and Y for
different values of X2
– Previous studies of the same topic.
– Empirical evidence in your own data:
• Three-way association among X1, X2, and Y
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Specifying an interaction model
• Multivariate regression specifications to test for
interactions include a combination of “main
effects terms” and “interaction terms.”
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Main-effects-only specification
• A main-effects-only model implies that
controlling for other covariates (Xi),
– the effect of X1 on Y is the same for all values of X2,
– and the effect of X2 is the same for all values of X1.
• Its specification can be written:
Y = β0 + β1X1 + β2X2, where
X1 is the main effect term for the first independent
variable (IV)
X2 is the main effect term for a second IV
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Example: Main-effects-only model
• If Y is birth weight in grams
• X1 is family income in dollars
• X2 is a dummy variable for black race
– coded 1 for black infants
– 0 for white infants, the reference category
• Birth weight = β0 + β1Income + β2Black implies that the
slope of the income/birth weight curve is the same for
black as for white infants
– the income/birth weight (X1/Y) association is not modified by
race
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Birth weight (grams)
Main effects of race and income,
but no interaction with income
Income/birth weight curves for blacks and whites have
same slope (their curves are parallel)
But different intercepts
White
Black
Income ($)
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Interaction specification
• A model with interactions implies that
controlling for other covariates,
– the effect of X1 on Y is different for different values
of X2.
Y = β0 + β1X1 + β2X2 + β3X1 _ X2, where
X1 is the main effect term for the focal IV in the
interaction,
X2 is the main effect term for the modifying IV,
X1 _X2 is the interaction term between the focal and
modifying IVs.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Interaction term
• The value of the interaction term variable is
defined as the product of the two component
variables:
X1_ X2 = X1 × X2
• When naming an interaction term variable, I
often use an “_” to connect the names of the
two component variables.
– E.g., <HS_black would be the interaction between
the two variables “<HS” and “black.”
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Example calculation
of interaction term
• E.g., for case #1, if
– age (X1) = 27,
– income (X2) =$10,000,
– the interaction term age_income = 27 × 10,000 =
27,000.
• See podcast on creating variables to test for
interactions for additional detailed examples.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Contingency of coefficients
in an interaction model
Y = β0 + β1X1 + β2X2 + β3X1 _ X2,
• Inclusion of the interaction term X1_ X2 means
that the βis on the main effects terms X1 and X2
no longer apply to all values of X1 and X2.
– The main effects and interactions βis for X1 and X2
are contingent upon one another and cannot be
considered separately.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Implications for interpreting main
effects and interaction coefficients
Y = β0 + β1X1 + β2X2 + β3X1_X2
• In the interaction model:
– β1 estimates the effect of X1 on Y when X2 = 0,
– β2 estimates the effect of X2 on Y when X1 = 0,
– β3 must also be considered in order to calculate the
shape of the overall pattern among X1, X2, and Y.
• E.g., when X1 and X2 take on other values.
• See podcast on calculating the shape of an
interaction pattern.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Example: Interaction model
BW = β1Income + β2Black + β3Income_black,
• If β3 is statistically significantly different from zero, the
slope of the income/birth weight curve is different for
black than for white infants.
• β1 estimates the association between income and birth
weight among whites (e.g., when Black = 0)
• β2 estimates the difference in birth weight for blacks
compared to whites at income = 0.
• β3 estimates how predicted birth weight deviates from
the value implied by β1 and β2 alone, for different
combinations of race and income.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Birth weight (grams)
Main effects of race and income,
and interaction of race with income
Birth weight/income curves for blacks and whites have
Different slopes
and
Different intercepts
White
Black
Income ($)
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Summary
• Interactions occur when the association
between one IV and the DV differs depending on
the values of a second IV.
• Can occur in terms of
– Direction
– Magnitude
– Statistical significance
• Can be tested using multivariate regression
models involving main effects and interaction
terms (variables).
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Suggested resources
• Chapter 16, Miller, J. E. 2013. The Chicago Guide
to Writing about Multivariate Analysis, 2nd
Edition.
• Chapters 8 and 9 of Cohen et al. 2003. Applied
Multiple Regression/Correlation Analysis for the
Behavioral Sciences, 3rd Edition. Florence, KY:
Routledge.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Suggested online resources
• Podcasts on
– Visualizing shapes of interactions
– Creating variables and specifying models to test for
interactions
– Calculating overall shape of an interaction pattern
from regression coefficients
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Suggested practice exercises
• Study guide to The Chicago Guide to Writing
about Multivariate Analysis, 2nd Edition.
– Question #1 in the problem set for Chapter 16
– Reviewing” exercise #1 in the suggested course
extensions for Chapter 16
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Contact information
Jane E. Miller, PhD
jmiller@ifh.rutgers.edu
Online materials available at
http://press.uchicago.edu/books/miller/multivariate/index.html
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.