Implementing “generalization,
example, exception”:
Behind-the-scenes work for
summarizing a pattern
Jane E. Miller, PhD
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Overview
• Steps
–
–
–
–
–
–
Display the data in a chart
Indentify the dimensions of the comparison
Choose a representative example
Characterize the pattern
Identify exceptions
Write the description
• “Generalization, example, exception” useful for
– Three-way patterns
– Interaction patterns calculated from multivariate coefficients
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Objectives of a GEE
• GEE = “generalization, example, exception”
• Aims to describe the forest, not the trees, in a complex
pattern involving a lot of numbers
– Which aspects can be summarized?
– Which aspects deviate from that summary?
• In a three-way association, describe the pattern among
– Two independent variables (IVs) and a dependent variable
(DV)
– One IV and a series of DVs
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Display the data in a chart
• Create a chart of the association
• Charts are the easiest way to see patterns
– Which values are highest and by how much compared to
other values?
– Whether trends are up, down, or level
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Table and corresponding chart
Knowledge about AIDS transmission,
by language spoken at home and ability to speak English, New Jersey, 1998
English
Mode of transmission
Likely modes of transmission
Language spoken at home/
language used on
questionnaire
Spanish Spanish
/English /Spanish
English
ques.
ques.
Spanish /English ques.
Spanish /Spanish ques.
100
90
80
70
60
Sexual intercourse with an infected person
93.6
87.5
95.0
50
Shared needles for IV drug use
92.4
90.6
65.0
40
Pregnant mother to baby
89.5
75.0
80.0
30
Blood transfusion from infected person
87.5
81.3
60.0
20
10
0
Sexual
Shared
Pregnant
Blood
intercourse needles for mother to transfusion
with an IV drug use
baby
from
infected
infected
person
person
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Design issues for your chart
• Draw it to scale, so you can assess
– Directionality
– Size of differences and trends across values
• Organize variables in the order you will describe them
– Theoretical criteria
– Empirical order
– See podcast on organizing data in tables and charts
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Use a spreadsheet to create your chart
• A spreadsheet can
– Test different ways of organizing the variables in your
chart
– Calculate magnitude of differences across values for
use in your description
• Difference (subtraction)
• Ratios (e.g., relative risks)
• Percentage change
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Identify dimensions of the comparison
• One dimension for each variable or set of variables in
your table or chart
• In a table, one dimension for each
– Row
– Column
– Panel
• In a chart, one dimension for each
– Axis
– Legend item
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Dimensions in a three-way table
• Moving down the rows within one column
• Moving across the columns within one row
Knowledge about AIDS transmission, by language spoken at home and ability to
speak English, New Jersey, 1998
Language spoken at home/
language used on questionnaire
Mode of transmission
Likely modes of transmission
Sexual intercourse with an infected person
Shared needles for IV drug use
Pregnant mother to baby
Blood transfusion from infected person
English
93.6
92.4
89.5
87.5
Spanish
Spanish
/English ques. /Spanish ques.
87.5
90.6
75.0
81.3
95.0
65.0
80.0
60.0
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Dimensions in a clustered bar chart
• Moving across clusters within one bar color
• Moving across bar colors within one cluster
100
90
80
70
60
50
English
40
Spanish /English ques.
30
Spanish /Spanish ques.
20
10
0
Sexual intercourse Shared needles for Pregnant mother Blood transfusion
with an infected
IV drug use
to baby
from infected
person
person
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Dimensions in a multiple-line chart
• Moving left to right along one line
• Moving vertically across lines at one value of the x
variable
Median sales price of new one-family homes,
by region, US, 1980 to 2000
Median price ($)
$250,000
$200,000
Northeast
Midwest
South
West
$150,000
$100,000
$50,000
$1980
1985
1990
1995
2000
Year
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Choose a representative example
• Identify one representative example as the basis for
each generalization about the shape of the pattern
– Must characterize many if not most values in the pattern
– Not an arbitrary choice!
• Find one value of the first independent variable to use
to describe the general shape of the association
between the second IV and the DV. E.g.,
– Hold one IV constant while you vary the other
– Don’t vary values of both IVs at the same time
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
What does “representative” mean?
• A representative example of a general pattern will be
characteristic of the association between the second IV
and the DV for many if not all subgroups of that first IV.
E.g.,
– A region that shows a time trend in prices that is similar to
that observed in several other regions
– A topic for which the rank order of language groups in test
performance is the same
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Possible starting points for identifying
representative examples
• Theoretically based
– A value of particular interest for your research question
– A summary measure that combines results for the various
component variables
• Useful if comparing across a series of related outcomes
• Empirically based
– Start at one end of an empirically organized axis, column, or
row
– Overall sample, for a comparison across groups
– Modal (most common) group
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Checking for representativeness of
example in a three-way table
• Describe the pattern across columns within one row
• Test whether the direction and magnitude of that
pattern also applies to other rows
Knowledge about AIDS transmission, by language spoken at home and ability to
speak English, New Jersey, 1998
Language spoken at home/
language used on questionnaire
Mode of transmission
Likely modes of transmission
Sexual intercourse with an infected person
Shared needles for IV drug use
Pregnant mother to baby
Blood transfusion from infected person
Mean percentage of “likely” questions correct
English
93.6
92.4
89.5
87.5
91.7
Spanish
Spanish
/English ques. /Spanish ques.
87.5
90.6
75.0
81.3
83.6
95.0
65.0
80.0
60.0
75.0
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Checking for representativeness of
example in a three-way table
• Describe the pattern down the rows within one column
• Test whether the direction and magnitude of that
pattern also applies to other column
Knowledge about AIDS transmission, by language spoken at home and ability to
speak English, New Jersey, 1998
Language spoken at home/
language used on questionnaire
Mode of transmission
Likely modes of transmission
Sexual intercourse with an infected person
Shared needles for IV drug use
Pregnant mother to baby
Blood transfusion from infected person
English
93.6
92.4
89.5
87.5
Spanish
Spanish
/English ques. /Spanish ques.
87.5
90.6
75.0
81.3
95.0
65.0
80.0
60.0
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Checking representativeness of an example
in a multiple-line chart
• Describe the pattern cutting vertically across lines (regions) at
one value of the x variable (year)
• Check whether that rank order of regions also applies to the
other dates
Median sales price of new one-family homes,
by region, US, 1980 to 2000
Median price ($)
$250,000
$200,000
Northeast
Midwest
South
West
$150,000
$100,000
$50,000
$1980
1985
1990
1995
2000
Year
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Checking representativeness of an
example in a multiple-line chart
• Describe the pattern along one line moving left to right
• Test whether that pattern applies to the other lines
Median sales price of new one-family homes,
by region, US, 1980 to 2000
Median price ($)
$250,000
$200,000
Northeast
Midwest
South
West
$150,000
$100,000
$50,000
$1980
1985
1990
1995
2000
Year
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Trial and error in identifying a
representative example
• Might have to try several groups or values to find one
that has a pattern that is similar to that in other groups
• If your initial choice isn’t similar to any of the others in
terms of direction or magnitude, it might be an
exception
• See whether other values have a pattern that can be
generalized
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Characterize the pattern
• For your representative value of one independent
variable, characterize these aspects of the association
between the other independent variable and the
dependent variable:
– Direction
– Magnitude
– Statistical significance
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Characterizing direction in a
cross-sectional comparison
• Which categories of the independent variable (IV) have
the highest and lowest values of the dependent
variable (DV)?
• For multiple category IVs, describe the rank order of
categories according to values of the DV
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Characterizing direction for trends
• Describe whether each line slopes
–
–
–
–
Upward
Downward
Is approximately level
Follows a standard statistical distribution
• e.g., a normal curve
– Asymptotically approaches some value
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Characterizing magnitude
• Do some behind-the-scenes math to measure the size
of differences between values you are comparing. E.g.,
– Difference (subtraction or OLS coefficient)
– Ratio (division, or odds ratio from logit model)
– Percentage difference or change
• See chapters on types of quantitative comparisons to
help you choose the kind of calculation that best suits
your objectives
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Take notes about
direction and magnitude
• On a hardcopy chart, make notes about:
– Direction: Use “>,” “<,” “=,” or arrows
– Size: Jot results of calculations to compare values
• To help you later write a correct, complete description, mention
– Units
– Type of calculation
– Shape: Verbs, adjectives, adverbs, analogies, or metaphors
that capture the shape of the pattern
– Statistical significance: of specific contrasts, marked with
• Symbols
• Shading (highlighter)
• Don’t worry about full sentences at this point
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Identify exceptions
• If parts of your table or chart depart substantially from
the general pattern you have identified, they are
exceptions
• Exceptions can occur in terms of
– Direction
– Size
– Statistical significance
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Exceptions in direction of a trend
• The median sales prices dipped in the early 1990s in
the West, but continued upward or remained level
in each of the other three regions
– The West was the exception
Median sales price of new one-family homes,
by region, US, 1980 to 2000
Median price ($)
$250,000
$200,000
Northeast
$150,000
Midwest
$100,000
South
West
$50,000
$1980
1985
1990
1995
2000
Year
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Exceptions in magnitude
of a cross-sectional difference
– Generalize based on the two
income groups for which the
racial gap in ER use is similar
(poor and near poor)
– Non-poor are the exception
Relative odds of emergency room
visits for asthma, by race and income
Black
Non-black
8
Relative odds
• The difference between
odds ratios across racial
groups is much larger
among the non-poor than in
the other two income
groups (compare brackets 1
and 2 to bracket 3)
2
7
6
5
4
1
3
3
2
1
0
Poor
Near poor
Non-poor
Income group
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Exceptions in statistical significance
• Examples of this type of exception
– All but one of the coefficients in a regression model are
statistically significant at p < 0.05
– Only one of the coefficients in a regression model is
statistically significant
• The generalization is lack of statistical significance
– In stratified models of the same outcome for several groups,
the coefficient on one or more specific independent variables
are statistically significant only for one group
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Write the description
• Organize the material into paragraphs
– One paragraph for each major pattern to be described
• Write topic sentences for each paragraph
– Introduce the substantive concepts to be discussed in that
paragraph
• Write evidentiary sentences to describe the
– General pattern
– Any exceptions
• In those descriptions, mention
– Direction
– Magnitude
– Statistical significance
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Wording for a generalization
• If a pattern characterizes many of the numbers you are
summarizing, introduce the generalization with
– “In general, . . .”
– “Typically, . . .”
– “By and large . . .”
– If the pattern characterizes all groups, say so!
– “In every region studied, . . .”
– “Throughout the entire period of observation . . .”
– “Regardless of [characteristic], . . .”
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Wording to present the example
• To document a pattern
– Introduce numeric evidence
– Refer to the table or chart that reports all of the numbers
• E.g.,
– “For instance, . . . [numeric contrast, with the W’s specified].”
– “For example, as shown in Figure X . . . [numeric fact, with its
W’s attached].”
• Reminder: “the W’s” refer to when, where, who, and
what
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Wording to introduce exception(s)
• To introduce exceptions to a general pattern, write
– “An exception [to that pattern] is . . . ”
– “On the other hand, . . .”
– “However, . . .”
• E.g., “In seven out of 10 years studied, [general pattern
and example]. However, in the other three years,
[contrasting pattern = exception].”
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Wording for exceptions in direction
• Some exceptions are literally the opposite of the
pattern described in the generalization. E.g.,
– A falling rather than rising trend
– A deficit instead of excess for one group compared to another
• Having described the direction of the general pattern,
introduce such exceptions with phrases such as:
– “On the contrary, . . .”
– “Conversely, . . .”
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Summary
• Use a systematic, step-by-step approach to identify
which parts of a broad pattern are
– Similar to one another
– Different from one another
• In terms of direction, magnitude, and statistical significance
• Create a chart or table that you refer to in your prose
description of that pattern
• Organize material into substantively cohesive
paragraphs
• Use vocabulary and phrasing to differentiate general
patterns from exceptions
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Suggested resources
• Chapter 2 and appendix A in
– Miller, J.E., 2004. The Chicago Guide to Writing about Numbers.
(“WA#”)
OR
– Miller, J.E., 2013. The Chicago Guide to Writing about Multivariate
Analysis, 2nd edition. (“WAMA II”)
• Chapter 7 in WA# or chapter 6 in WAMA II on creating
effective charts
• Chapter 5 in WA# or chapters 8 and 9 in WAMA II on types
of quantitative comparisons
• Chapter 9 of WA#, section on three-way associations
• Chapter 16 of WAMA II, on interactions
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Suggested online resources
• Podcasts on
– Reporting one number
– Comparing two numbers or series of numbers
– Summarizing a pattern with many numbers
– Organizing data in tables and charts
– Visualizing shapes of interaction patterns
– Writing prose to present results of an interaction
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Suggested practice exercises
• Study guide to The Chicago Guide to Writing about
Multivariate Analysis, 2nd Edition.
– Question #9 in problem set for chapter 2
– Question #9 in problem set for chapter 6
– Suggested course extensions for
• chapter 2
– “Reviewing” exercise #7
– “Writing” and “revising” exercise #4
• chapter 14
– “Applying statistics and writing” exercises #3 and 4
– “Revising” exercise #3
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.
Contact information
Jane E. Miller, PhD
jmiller@ifh.rutgers.edu
Online materials available at
http://press.uchicago.edu/books/miller/multivariate/index.html
The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.