Presenting statistical results to
nonstatistical audiences
Jane E. Miller, PhD
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Overview
• Academic and nonstatistical audiences
– Defined
– Interests and background
• Adapting description of methods
• Adapting presentation of results
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Why adapt material?
• A survey by Sorian and Baugh of government
policy makers showed that they
– Want to know how the findings relate to issues
– Don’t want to wade through a formal research
paper
• Complaints about many research reports
– “too long, dense, or detailed”
– “too theoretical, technical, or jargony”
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Example: Policy analysts and consultants
• Policy analysts must explain results of their
models to experts in government or nonprofit
agencies.
• Economic consultants have to communicate
results of their models to corporations or
community development agencies.
• Those experts are principally interested in
– How to interpret and apply the findings.
– Reassurance that you know the correct statistical
methods.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
How to adapt material
• Familiarize yourself with your audience’s
interests and likely applications of your study
findings.
– Present your analyses to match issues of concern
to them.
– DON’T make them translate statistical results to fit
their interests.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
What statistics courses teach
• Statistics courses emphasize
– understanding statistical assumptions
– estimating models
– interpreting statistical tests
– assessing coefficients and model fit.
• Students expected to demonstrate mastery by
– working with equations written in statistical
notation
– identifying the numbers for formal hypothesis
testing
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
What academic papers look like
•
•
•
•
Detailed review of the literature
Comprehensive data and methods section
Statistical tables
Jargon and equations used as shorthand
• A real mismatch with many applied audiences
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Example: Study of family and county level
factors associated with SCHIP disenrollment
• SCHIP = State Children’s Health Insurance Program
– Health insurance for children in low- to moderate-income
families who lack other health insurance
• Collaborative effort of
– Rutgers University’s Center for State Health Policy
– New Jersey Department of Human Services
• Project applied discrete time hazards models in a
multilevel (hierarchical linear model [HLM])
framework
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Academic audiences
for SCHIP study
• Northwestern/University of Chicago Joint Center for Poverty
Research (JCPR)
– Funding agency
– Policy oriented
• Rutgers Institute for Health, Health Care Policy and Aging
Research and Bloustein School of Planning and Public Policy
– Both policy-oriented research units
• University of Pennsylvania
– Academic but not policy oriented
• Health Services Research
– Journal with research emphasis
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Applied audiences
for SCHIP study
• New Jersey Department of Human Services
– Raised policy question
– Provided data
– Client for deliverable
• US Department of Health and Human Services
– Funding source
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Which would you rather have?
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Which would your client rather have?
Table 1. Multilevel discrete-time hazards models of disenrollment from SCHIP, New Jersey, 1998–2000
County Fixed
Effects Model
Variable
Intercept
Log Rel. Haz
Random Effects Model
Family Factors Only
s.e.
Log Rel. Haz
s.e.
Random Effects Model
Family + County Factors
Log Rel. Haz
s.e.
–5.581
(.159)
–5.421
(.142)
–5.455
(.159)
Black Race
0.047
(.150)
0.038
(.149)
0.198
(.165)
Hispanic Race
0.121
(.064)
0.109
(.063)
0.124
(.064)
Plans C and D (ref = Plan B)
0.826
(.142)
0.823
(.142)
0.825
(.142)
Black * Plans C/D
0.449
(.154)
0.456
(.154)
0.451
(.154)
Plans C/D * Months
0.078
(.036)
0.078
(.036)
0.077
(.036)
Plans C/D * Months2
–0.0069
(.0019)
–0.0069
(.0019)
–0.0068
(.0019)
.007
(.012)
–0.039
(.019)
Family-Level Characteristics
Interactions
County-Level Characteristics
% Black Physicians
Cross-Level Interaction
Black * % Black Physicians
Random Effects
Between-County Variance
Scaled Deviance Statistic
0.012
30,824.5
(.007)
30,948.4
0.005
(0.006)
30,895.4
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Chances of disenrollment by race, SCHIP plan,
and county physician racial composition
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Adapting results for nonstatisticians
• Increase prominence of the substantive
question.
• Reduce emphasis on technical details of data
and methods.
– Rephrase jargon and statistical concepts into
colloquial language.
– Avoid equations or Greek symbols.
– Minimize use of formal citations.
• Translate results to show how they apply to
real-world issues of interest to that audience.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Writing style and organization
• Write a clear, well-organized narrative
– What questions did you address?
– What answers did you find?
– How can the findings be applied?
• Use standard expository writing guidelines
– Good introduction
– Present numbers as evidence
• Explain what question each is intended to answer
– Good summary of findings and what they mean
• See article in Chance and podcast on presenting numbers as
evidence.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
How to write about technical stuff
• Explaining why your methods are needed
– Especially if using multivariate models
•
•
•
•
Showing how key variables are measured
Interpreting numeric values (coefficients)
Reporting statistical significance
Adapting tables and charts
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Acronyms and statistical vocabulary
• Even with a quantitatively sophisticated audience,
don’t assume that people will know the statistical
vocabulary used in other fields.
– Define the term you use, then mention synonyms.
• If you use acronyms, spell them out at first usage.
– “HEDIS”(Health Plan Employer Data and Information Set)
– “HLM”(hierarchical linear model)
• Avoid acronyms if they are not familiar to the field or
are used only once or twice.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Why your methods are needed
• Explain what your model did that couldn’t have
been answered with simpler techniques.
• Incorporate the specific concepts you study.
Poor: “We use logistic regression and a discrete-time hazards
specification to assess relative hazards of SCHIP disenrollment,
with plan level as our key independent variable.”
Better: “Because chances of disenrollment from the State
Children’s Health Insurance Program (SCHIP) vary by the
amount of time enrolled, our analyses correct for differences in
duration of enrollment across families when estimating the
patterns for different income levels.”
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Application of a method to your topic
• Replace technical terms with familiar names.
• Show how that method applies to your
research question and data.
• Poor: “The data structure can be formulated
as a two-level hierarchical linear model, with
families (the level-1 unit of analysis) nested
within counties (the level-2 unit of analysis).”
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Better presentation of methods:
Tailored to the audience
Better [for a nonstatistical but academic audience]:
“The data have a hierarchical (or multilevel)
structure, with families clustered within counties.”
Better [for a lay audience]: “To disentangle the
contributions of families’ and counties’
characteristics to the problem of program
disenrollment, we used models that incorporated
information at both levels.”
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Measurement of key variables
• To report an unfamiliar type of statistic,
embed the definition in your explanation.
Poor: “The sensitivity of the new screening test
for diabetes is 0.90.”
Better: “The new screening test had a sensitivity
of 0.90, correctly identifying 90% of diabetics.”
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Adapting tables and charts
• Create small tables or charts
– Divide up large complex tables into smaller parts
– Focus each on one fact or pattern
– Use simple, familiar formats
• Replace standard errors and test statistics with
– p-values
– Symbols such as asterisks or daggers
– Formatting such as color, italics, or bold
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Birth weight and socioeconomic characteristics
by race/ethnicity, US, 1988–1994 NHANES III
Non-Hispanic
white
Non-Hispanic
black
Mexican
American
3,426.8
3,181.3
3,357.3
3,379.2
5.8
11.3
7.0
6.8
All
Birth weight
Mean (grams)
% Low birth weight
Socioeconomic characteristics
% Teen mother
9.4
22.9
18.4
12.5
% Mother <high school
14.7
30.1
58.4
21.6
% Poor
14.7
48.5
50.7
23.9
3,733
2,968
3,112
9,813
Unweighted N
Statistics are weighted to population level using weights provided with the NHANES III (US DHHS 1997).
Differences across racial/ethnic origin groups were statistically significant for all variables shown (p < 0.01).
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Low birthweight by race/ethnicity
From second row of preceding table.
p < 0.05
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Minority racial groups have lower SES
From bottom three rows of table.
All p < 0.05
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Interpreting OLS coefficients
• Emphasize direction and size of the
association
• Name the specific variables involved
• Incorporate units of measurement
• Use colloquial language
“OLS” = ordinary least squares regression
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Examples of interpreting βs
Poor: “Age and weight were correlated.”
Poor version number2: “Beta was 10.7.”
Better: “For each additional year of mother’s
age at the time of her child’s birth, birth
weight increased by 10.7 grams.”
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Coefficients from logit models
• Replace log-odds (logit coeffs) with odds
ratios.
– Can be described in terms of simple multiples.
– Don’t need to use the term “odds ratio” at all!
Poor: “The log-hazard of disenrollment for one-child
families was 0.316.”
Better: “Families with only one child enrolled in the
program were about 1.4 times as likely as larger
families to disenroll.”
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Wording for statistical significance
• State the conclusion of the statistical test, not
the raw numbers or calculations.
Poor: “The log-relative hazard for SCHIP plans C and D
was 0.826 with a standard error of 0.142. Because
the beta was more than 2.56 times the standard
error, we conclude that the effect is statistically
significant at p < 0.01.”
Better: “Families in SCHIP plans C and D were roughly
2.3 times as likely to disenroll as those in plan B. A
difference that large is unlikely to occur by random
chance alone.”
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Wording for LACK of statistical
significance
• “The difference between the disenrollment
rates for Plans C and D could easily have
occurred by chance alone.”
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Summary
• Get to know your audience before you write.
– What questions do they want answered?
– How familiar are they with statistics?
• Avoid statistical language.
– Report direction and size of associations in plain
English.
– Mention conclusions of inferential statistics, not the
raw numbers or calculations.
• Use charts or simple tables to convey shape and
size of numeric patterns visually.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Suggested resources
• Chapter 20 in Miller, J. E. 2013. The Chicago Guide to
Writing about Multivariate Analysis, 2nd Edition.
• Miller, J.E. 2006. “How to Communicate Statistical
Findings: An Expository Writing Approach.” Chance.
19(4):43-49.
• Nelson, D. E., R. C. Brownson, P. L. Remington, and C.
Parvanta, editors. 2002. Communicating Public Health
Information Effectively: A Guide for Practitioners.
Washington DC: American Public Health Association.
• Sorian, R., and T. Baugh. 2002. “Power of Information:
Closing the Gap between Research and Policy.” Health
Affairs 21 (2): 264–73.
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Suggested online resources
• Podcasts on
– Reporting one number
– Comparing two numbers or series of numbers
– Creating effective tables and charts
– Interpreting multivariate coefficients
– Designing slides for a speech
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Suggested practice exercises
• Study guide to The Chicago Guide to Writing
about Multivariate Analysis, 2nd Edition.
– Questions #1 through 3 in the problem set for
chapter 20
– Suggested course extensions for chapter 20
• “Reviewing” exercises #1 through 5
• “Writing” exercises #1, 2, 3, 6, 7 and 9
• “Revising” exercises #2 and 4
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.
Contact information
Jane E. Miller, PhD
jmiller@ifh.rutgers.edu
Online materials available at
http://press.uchicago.edu/books/miller/multivariate/index.html
The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.