Watch in slide show mode to observe (modest) animation.
comments questions: dan.kahan@yale.edu
papers, etc: www.culturalcognition.net
www.culturalcognition.net
The Science of Science Communication
“Disentanglement Principle”
Dan M. Kahan
Yale University
& many others
Research Supported by:
National Science Foundation, SES-0922714
Annenberg Center for Public Policy
Skoll Global Threats Fund
What am I talking about? ...
What am I talking about? ...
scaling up SENCER . . .
What am I talking about? ...
scaling up SENCER . . . by leveraging what it already does
I. Two science literacy “measurement problems”
II. Reason and identity protection
III. The “disentanglement principle”
IV. SENCER and the disentanglement agenda
What am I talking about? ...
scaling up SENCER . . . by leveraging what it already does
I. Two science literacy “measurement problems”
II. Reason and identity protection
III. The “disentanglement principle”
IV. SENCER and the disentanglement agenda
What am I talking about? ...
scaling up SENCER . . . by leveraging what it already does
I. Two science literacy “measurement problems”
II. Reason and identity protection
III. The “disentanglement principle”
IV. SENCER and the disentanglement agenda
“Belief” in evolution
“Ordinary Science Intelligence”
Assessment
OSI_1.0
OSI_2.0
“Ordinary Science Intelligence” Assessment
18 items
• 6 “Basic facts” (NSF Indicators, Pew)
• 3 “Theory of science” (NSF Indicators)
• 6 Numeracy (Peters et al. 2006)
• 3 Cognitive reflection (Frederick 2005)
“Ordinary Science Intelligence” Assessment
18 items
• 6 “Basic facts” (NSF Indicators, Pew)
• 3 “Theory of science” (NSF Indicators)
• 6 Numeracy (Peters et al. 2006)
• 3 Cognitive reflection (Frederick 2005)
0
1
2
3
4
5
6
7
8
Dimensionality (principal factor)
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18
Factor
“Ordinary Science Intelligence” Assessment
18 items
• 6 “Basic facts” (NSF Indicators, Pew)
• 3 “Theory of science” (NSF Indicators)
• 6 Numeracy (Peters et al. 2006)
• 3 Cognitive reflection (Frederick 2005)
Dimensionality (principal factor)
0
1
2
3
4
5
6
Reliability (1-[1/I])
7
8
2PL Item resonse theory scaling
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18
Factor
2nd percentile
14th percentile
50th percentile
86th percentile
Ordinary science intelligence
98th percentile
Science literacy: item response functions
“In a lake, there is a patch of lilypads. Every day, the patch
doubles in size. If it takes 48 days for the patch to cover the
entire lake, how long would it take for the patch to cover
half of the lake?” [47]
-2
2nd percentile
00
0
.1
.2
.3
.4
.5
.6
.7
.8
probability of correct answer
answer
correct
.1
.2
.6
.1 probability
.2 .3
.3 .4
.4 of.5
.5
.6 .7
.7 .8
.8 .9
.9
.9
11
1
“Electrons are smaller than atoms.” (True/false)
-1
14th percentile
0
50th percentile
1
86th percentile
Science literacy score
2
98th percentile
-2
-2
2nd percentile
-1
-1
14th percentile
00
50th percentile
11
86th percentile
22
98th percentile
Science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N’s 1011 & 1999. Nationally representative sample, April/May 2014 (YouGov). Predicted
probabilities derived via Monte Carlo Simulation based on logistic regression. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science
Communication Measurement Problem, Adv. in Pol. Psych. (in press).
Science literacy: item response functions
“In a lake, there is a patch of lilypads. Every day, the patch
doubles in size. If it takes 48 days for the patch to cover the
entire lake, how long would it take for the patch to cover
half of the lake?” [47]
-2
-2
2nd percentile
00
0
0
.1
.1
answer
correct
.1
.2
.6
.1 probability
.2 .3
.3 .4
.4 of.5
.5
.6 .7
.7 .8
.8 .9
.9
answer
correct
of.5
probability
.2 .3
.3 .4
.4 .5
.6 .7
.7 .8
.8 .9
.9
.2
.6
11
1
1
“Human beings, as we know them today, developed from
earlier species of animals.” (True/false)
-1
-1
14th percentile
00
50th percentile
11
86th percentile
Science literacy score
22
98th percentile
-2
-2
2nd percentile
-1
-1
14th percentile
00
50th percentile
11
86th percentile
22
98th percentile
Science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N’s 1011 & 1999. Nationally representative sample, April/May 2014 (YouGov). Predicted
probabilities derived via Monte Carlo Simulation based on logistic regression. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science
Communication Measurement Problem, Adv. in Pol. Psych. (in press).
Science literacy: item response functions
“In a lake, there is a patch of lilypads. Every day, the patch
doubles in size. If it takes 48 days for the patch to cover the
entire lake, how long would it take for the patch to cover
half of the lake?” [47]
11
answer
correct
.1
.2
.6
.1 probability
.2 .3
.3 .4
.4 of.5
.5
.6 .7
.7 .8
.8 .9
.9
Below avg.
religiosity
Above avg.
religiosity
-2-2
2nd percentile
00
0
0
answer .9
of correct
probability
.1
.1 .2
.2 .3
.3 .4
.4 .5
.5 .6
.6 .7
.7 .8
.8 .9
1
1
“Human beings, as we know them today, developed from
earlier species of animals.” (True/false)
-1-1
14th percentile
00
50th percentile
11
86th percentile
Science literacy score
22
98th percentile
-2
-2
2nd percentile
-1
-1
14th percentile
00
50th percentile
11
86th percentile
22
98th percentile
Science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N’s 1011 & 1999. Nationally representative sample, April/May 2014 (YouGov). Predicted
probabilities derived via Monte Carlo Simulation based on logistic regression. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science
Communication Measurement Problem, Adv. in Pol. Psych. (in press).
Science literacy: item response functions
“In a lake, there is a patch of lilypads. Every day, the patch
doubles in size. If it takes 48 days for the patch to cover the
entire lake, how long would it take for the patch to cover
half of the lake?” [47]
11
answer
correct
.1
.2
.6
.1 probability
.2 .3
.3 .4
.4 of.5
.5
.6 .7
.7 .8
.8 .9
.9
Below avg.
religiosity
Above avg.
religiosity
-2-2
2nd percentile
Above avg.
religiosity
Below avg.
religiosity
00
0
0
answer .9
of correct
probability
.1
.1 .2
.2 .3
.3 .4
.4 .5
.5 .6
.6 .7
.7 .8
.8 .9
1
1
“Human beings, as we know them today, developed from
earlier species of animals.” (True/false)
-1-1
14th percentile
00
50th percentile
11
86th percentile
Science literacy score
22
98th percentile
-2
2nd percentile
-1
14th percentile
0
50th percentile
1
86th percentile
2
98th percentile
Science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N’s 1011 & 1999. Nationally representative sample, April/May 2014 (YouGov). Predicted
probabilities derived via Monte Carlo Simulation based on logistic regression. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science
Communication Measurement Problem, Adv. in Pol. Psych. (in press).
“Religiosity” (Cronbach’s α = 0.80)
item
focus
wording
measure
pew_religimp Religious
importance
How important is religion in
your life?
1 = very important
2 = somewhat important
3 = Not too important
4 = Not at all important
pew_churatd
Church
attendance
Aside from weddings and
funerals, how often do you
attend religious services…
1 = More than once a week
2 = once a week
3 = once or twice a month,
4= a few times a year
5 = seldom
6= or never
pew_prayer
Frequency of
prayer
Outside of attending religious
services, do you pray . . .
1 =several times a day
2 = once a day
3 = a few times a week
4 = once a week
5 = a few times a month
6= seldom
7= or never?
OSI_2.0 items plus Evolution & Religiosity items
0
1
2
3
4
5
6
7
Dimensionality (principal factor)
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21
Factor
Variable
Factor1
RADIOACTIV~c
LASERS_c
ELECTRONS_c
PEWGAS2_c
COPERNICUS~c
ANTIBIOTIC~c
probabil~a_c
probabil~b_c
valid_c
die_c
BUCKS_c
SWEEP_c
DISEASE1_c
DISEASE2_c
WIDGET_c
brst_c
BATBALL_c
pew_churatd
EVOLUTIONa_c
pew_prayer
pew_religimp
0.6103
0.5631
0.4283
0.4883
0.5617
0.6520
0.6457
0.5114
0.5076
0.7463
0.6525
0.8060
0.6865
0.6247
0.6523
0.6865
0.7160
Loadings < 0.4 suppressed.
Factor2
Factor3
0.4629
-0.4312
0.7495
0.5686
0.7546
0.8504
Fact
Science literacy: item response functions
“In a lake, there is a patch of lilypads. Every day, the patch
doubles in size. If it takes 48 days for the patch to cover the
entire lake, how long would it take for the patch to cover
half of the lake?” [47]
11
answer
correct
.1
.2
.6
.1 probability
.2 .3
.3 .4
.4 of.5
.5
.6 .7
.7 .8
.8 .9
.9
Below avg.
religiosity
Above avg.
religiosity
-2-2
2nd percentile
Above avg.
religiosity
Below avg.
religiosity
00
0
0
answer .9
of correct
probability
.1
.1 .2
.2 .3
.3 .4
.4 .5
.5 .6
.6 .7
.7 .8
.8 .9
1
1
“Human beings, as we know them today, developed from
earlier species of animals.” (True/false)
-1-1
14th percentile
00
50th percentile
11
86th percentile
Science literacy score
22
98th percentile
-2
2nd percentile
-1
14th percentile
0
50th percentile
1
86th percentile
2
98th percentile
Science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N’s 1011 & 1999. Nationally representative sample, April/May 2014 (YouGov). Predicted
probabilities derived via Monte Carlo Simulation based on logistic regression. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science
Communication Measurement Problem, Adv. in Pol. Psych. (in press).
Science literacy: item response functions
“In a lake, there is a patch of lilypads. Every day, the patch
doubles in size. If it takes 48 days for the patch to cover the
entire lake, how long would it take for the patch to cover
half of the lake?” [47]
11
answer
correct
.1
.2
.6
.1 probability
.2 .3
.3 .4
.4 of.5
.5
.6 .7
.7 .8
.8 .9
.9
Below avg.
religiosity
Above avg.
religiosity
-2
2nd percentile
Above avg.
religiosity
Below avg.
religiosity
00
0
.1
.2
.3 .4
.5
.6 .7
.8
probability of correct answer
.9
1
“According to the theory of evolution, human beings, as
we know them today, developed from earlier species of
animals.” (True/false)
-1
14th percentile
0
50th percentile
11
86th percentile
Science literacy score
22
98th percentile
-2
2nd percentile
-1
14th percentile
0
50th percentile
1
86th percentile
2
98th percentile
Science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N’s 1011 & 1999. Nationally representative sample, April/May 2014 (YouGov). Predicted
probabilities derived via Monte Carlo Simulation based on logistic regression. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science
Communication Measurement Problem, Adv. in Pol. Psych. (in press).
Teaching evolution to “nonbelievers”
What am I talking about? ...
scaling up SENCER . . . by leveraging what it already does
I. Two science literacy “measurement problems”
II. Reason and identity protection
III. The “disentanglement principle”
IV. SENCER and the disentanglement agenda
“Belief” in global warming
“Belief” in global warming
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
“Belief” in global warming
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
67
76
7
56
675
6
56477
45
5
4
3
2
> avg. Left_right
33
0
00
0
000
66
3
-1
333
01 3 0120
6
1
0
1203453
1 0
01
01234
12342 5
23145646 23
3425675
12
global warming risk
global warming risk
6
666
>< avg. Left_right
99
9
999
12
12
15
15
18
18
21
21
12
12
12
15
15
15
18
18
18
21
21
21
0
1
2
3
> avg.
Left_right
Very high
Science
Comprehension
Very
high
Science
comprehension
6
9 Comprehension
12
15
18
21
Science
2
0
Kahan, D.M., Peters,
E.,3Wittlin, M.,
Braman,18
D. & Mandel,
0
12 L.L.,15
15
18
21
66 Slovic, 99P., Ouellette,
12
21
0 3 6 9 12 015 18 21 3
G. The polarizing impact of science literacy and numeracy on perceived climate
change risks. Nature Climate Change 2, 732-735 (2012).
0
3
6
9
12
15
18
21
3
6
9
0
3
6
9
Very low
12 015 18
Very high
21
3
6
9 Comprehension
12
15
Science
3
6
1
0
0
12
15
18
18
21
21
0
.7
.6
.5
6
5
5
4
7
.4
.2
7
6
3
2
.1
6
5
0
2
1
5
4
1
0
4
3
3
0
3
2
< avg. Left_right> avg. Left_right
0
-2
1
0
2
1
< avg. Left_right
Very
Very low
Verylowlow
9 12 015 18 21 3
None at all
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
< avg. Left_right
-3
4
3
.3
7
6
7
.8
.9
1
Extremely high
risk
453676
7
3
4
77
“Belief” in global warming
0
9
12
15
18
21
Moderate
Between low
and moderate
6
5
Very low
1
None at all
0
Low
4
Between moderate
and high
3
High
2
Extremely high
risk
7
“How much risk do you believe global warming poses to human
health, safety, or prosperity?”
-1.6
-1
Very
Liberal
Veryliberal
liberal
Liberal
Strong
StrongDemocrat
Democrat Democrat
Democrat
0
Moderate
Moderate
Independent
Independent
1
1.6
Conservative Very
Very
Conservative
Conservative
Conservative
Republican Strong
Strong
Republican
Republican
Republican
Left_right political orientation
N = 1,885. Nationally representative sample, June 2013 (YouGov). Subjects “color coded” based on response to riskperception outcome variable. X-axis reflects subject score on composite scale that aggregates responses to 7-point
party identification item and 5-point “liberal-conservative” ideology item (α = 0.82).
Moderate
Between low
and moderate
6
5
Very low
1
None at all
0
Low
r = - 0.65, p < 0.01
4
Between moderate
and high
3
High
2
Extremely high
risk
7
“How much risk do you believe global warming poses to human
health, safety, or prosperity?”
-1.6
-1
Very
Liberal
Veryliberal
liberal
Liberal
Strong
StrongDemocrat
Democrat Democrat
Democrat
0
Moderate
Moderate
Independent
Independent
1
1.6
Conservative Very
Very
Conservative
Conservative
Conservative
Republican Strong
Strong
Republican
Republican
Republican
Left_right political orientation
N = 1,885. Nationally representative sample, June 2013 (YouGov). Subjects “color coded” based on response to riskperception outcome variable. X-axis reflects subject score on composite scale that aggregates responses to 7-point
party identification item and 5-point “liberal-conservative” ideology item (α = 0.82).
6
5
4
7
6
5
4
7
< avg. Left_right
2
3
> avg. Left_right
1
5
1
2 6 3
global warming risk
Extremely high
risk
7
“Belief” in global warming
0
0
4
None at all
>< avg. Left_right
0
00
3
33
6
66
Very
low
Very
low
12
12
15
15
18
18
Science
ScienceComprehension
comprehension
21
21
1
2
3
Very high
0
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
9
99
0
3
6
9
12
15
18
21
6
5
4
7
6
5
4
7
< avg. Left_right
2
3
> avg. Left_right
1
5
1
2 6 3
global warming risk
Extremely high
risk
7
“Belief” in global warming
0
0
4
None at all
>< avg. Left_right
0
00
3
33
6
66
Very
low
Very
low
12
12
15
15
18
18
Science
ScienceComprehension
comprehension
21
21
1
2
3
Very high
0
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
9
99
0
3
6
9
12
15
18
21
67
76
7
56
675
6
45
564
5
4
3
> avg. Left_right
2
231 6 23
342
12
1
120
5
01
.2
00
33
0
0
04 0
1
-3
< avg. Left_right
> avg. Left_right
3
2
-2
0
000
66
3
-1
333
6
0
666
>< avg. Left_right
99
9
1
999
12
12
15
15
18
18
21
21
12
12
12
15
15
15
18
18
18
21
21
21
2
3
2
3
Very
Very low
Very high
Science Comprehension
Verylowlow
Very
high
0 3 6 9 12 015 18 21 3
6Science9 comprehension
12
15
18
21
Science
Comprehension
Kahan, D.M., Peters, E., Wittlin, M., Slovic, P., Ouellette, L.L., Braman, D. & Mandel,
0 The
3 6 polarizing
9 12 15 impact
18 21 of science literacy and numeracy on perceived climate
G.
change risks. Nature Climate Change 2, 732-735 (2012).
1
.1
0
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
0
1
0
2
1
None at all
< avg. Left_right
0
.7
.6
global warming risk
global warming risk
6
5
.5
.4
5
4
4
3
.3
7
6
7
.8
.9
1
Extremely high
risk
453
7
34
77
“Belief” in global warming
0
3
6
9
12
15
18
21
Science literacy: item response functions
1
“According to the theory of evolution, human beings, as we
know them today, developed from earlier species of
animals.” (True/false)
correct
.1 probability
.2 .3 .4 of.5
.6 answer
.7 .8 .9
Below avg.
religiosity
Above avg.
religiosity
-2-2
2nd percentile
Below avg.
religiosity
Above avg.
religiosity
0
0
0
answer .9
of correct
probability
.1
.1 .2
.2 .3
.3 .4
.4 .5
.5 .6
.6 .7
.7 .8
.8 .9
1
1
“Human beings, as we know them today, developed from
earlier species of animals.” (True/false)
-1-1
14th percentile
00
50th percentile
11
86th percentile
Science literacy score
22
98th percentile
-2
2nd percentile
-1
14th percentile
00
50th percentile
11
86th percentile
22
98th percentile
Science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N’s 1011 & 1999. Nationally representative sample, April/May 2014 (YouGov). Predicted
probabilities derived via Monte Carlo Simulation based on logistic regression. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science
Communication Measurement Problem, Adv. in Pol. Psych. (in press).
“Climate science literacy” battery
Kahan, D. The Science Communication Measurement Problem, Adv. in Pol. Psych. (in press)
“Ordinary climate science intelligence” item response curves
“Climate scientists believe that the
increase of atmospheric carbon dioxide
associated with the burning of fossil
fuels will reduce photosynthesis by
plants.” [True or False]
1
of correct answer
probability
.1 .2 .3 .4 .5 .6 .7 .8 .9
.8
.7
.6
.5
.4
.3
.2
-1
-.5
0
.5
1
1.5
0
2
-2
-1
-.5
0
.5
1
1.5
2
-2
-1
-.5
0
.5
1
1.5
2
-1
-.5
0
.5
1
1.5
-2
2
0
.5
1
1.5
2
2
-1.5
-1
-.5
0
.5
1
1.5
2
1
.1
.2
.3
.4
.5
.6
.7
.8
.9
probability of correct answer
0
0
-.5
1.5
“Climate scientists believe that globally
averaged surface air temperatures were
higher for the first decade of the twentyfirst century (2000-2009) than for the
last decade of the twentieth century
(1990-1999) [True or false]
1
of correct answer
probability
.1 .2 .3 .4 .5 .6 .7 .8 .9
1
.9
.8
.7
.6
.5
.4
.3
-1
Ordinary climate science intellience
1
Ordinary climate science intellience
“Climate scientists believe that here will
be positive as well as negative effects
from human-caused global warming.”
[True or false]
.2
-1.5
.5
1
-1.5
Ordinary climate science intellience
.1
-2
0
of correct answer
probability
.1 .2 .3 .4 .5 .6 .7 .8 .9
-2
Ordinary climate science intellience
“Climate scientists believe that
nuclear power generation
contributes to global warming”
[True or false]
-.5
0
of correct answer
probability
.1 .2 .3 .4 .5 .6 .7 .8 .9
0
0
-1.5
-1
“Climate scientists believe that
human-caused global warming has
increased the number and severity
of hurricanes around the world in
recent decades.” [True or false]
1
1
.9
.8
.7
.6
.5
.4
.3
.2
.1
-2
-1.5
Ordinary climate science intellience
“Climate scientists believe that if the
North Pole icecap melted as a result of
human-caused global warming, global
sea levels would rise.” [True or False]
“Climate scientists believe that
human-caused global warming will
result in flooding of many coastal
regions .” [True or False]
probability of correct answer
-1.5
Ordinary climate science intellience
Ordinary climate science intellience
-2
-1.5
-1
-.5
0
.5
1
1.5
Ordinary climate science intellience
2
0
-1.5
0
.1
-2
probability of correct answer
“Climate scientists believe that
human-caused global warming will
increase the risk of skin cancer in
human beings.” [True or False]
1
.9
probability of correct answer
.9
.8
.7
.6
.5
.4
.3
.2
.1
0
probability of correct answer
1
“What gas do most scientists believe
causes temperatures in the
atmosphere to rise? Is it [hydrogen,
helium, carbon dioxide, radon]?”
-2
-1.5
-1
-.5
0
.5
1
1.5
Ordinary climate science intellience
Figures plot the predicted probability of correctly responding to the item conditional on score on OCSI scale. Black bars
2
Climate science literacy: item response functions
“Climate scientists believe that the increase of
atmospheric carbon dioxide associated with the burning
of fossil fuels will reduce photosynthesis by plants.” [True
or False]
0
0
0
.1
.2
.3
.4
.5
.6
.7
.8
probability of correct answer
.1 probability
.2 .3 .4 .5correct
.6 .7
.8 .9
.1 .2 .3 .4 of .5
.6 answer
.7 .8 .9
.9
1
1
1
“What gas do most scientists believe causes
temperatures in the atmosphere to rise? Is it [hydrogen,
helium, carbon dioxide, radon]?”
-2
2nd percentile
-1
14th percentile
0
50th percentile
1
86th percentile
Climate science literacy score
2
98th percentile
-2-2
2nd percentile
-1
14th -1
percentile
00
50th percentile
11
86th percentile
22
98th percentile
Climate science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N = 1,769. Predicted probabilities derived via Monte Carlo Simulation based on logistic
regression. Nationally representative sample, April/May 2014 (YouGov). Political outlook predictor set at -1 SD & + 1 SD on “Left_right" scale for “liberal
democrat” and “conservative Republican,” respectively. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
Climate science literacy: item response functions
“Climate scientists believe that the increase of
atmospheric carbon dioxide associated with the burning
of fossil fuels will reduce photosynthesis by plants.” [True
or False]
1
1
“What gas do most scientists believe causes
temperatures in the atmosphere to rise? Is it [hydrogen,
helium, carbon dioxide, radon]?”
Liberal
Democrat
0
0
.1
.2
.3
.4
.5
.6
Conservative
Republican
Conservative
Republican
correct
.1 probability
.2 .3 .4 of.5
.6 answer
.7 .8 .9
.7
.8
probability of correct answer
.9
Liberal
Democrat
-2
-1
-1.514th percentile
-.5 50th percentile
0
.5 86th percentile
1
1.598th percentile
22
2nd percentile
Climate science literacy score
-2
-1
-1.514th percentile
-.5 50th percentile
0
.5 86th percentile
1
1.598th percentile
2
2nd percentile
Climate science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N = 1,769. Predicted probabilities derived via Monte Carlo Simulation based on logistic
regression. Nationally representative sample, April/May 2014 (YouGov). Political outlook predictor set at -1 SD & + 1 SD on “Left_right" scale for “liberal
democrat” and “conservative Republican,” respectively. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
9
Climate science literacy & positions on global warming
8
7
9
6
8
No. correct
5
7
6
4
59
3
48
2
37
26
1
15
0
04
3
2
1
Human caused
Naturally caused
No warming
Positions on global warming in “past few decades”
Annenberg Center for Public Policy & Cultural Cognition Project. N = 1957. Nationally representative sample, April/May
2014 (YouGov). X-axis is continuous “Ordinary Science Intelligence” scale formed by IRT-weighted responses to NSF & Pew
0
science literacy, Numeracy,
and Cognitive Reflection Test items (α=0.83). CIs reflect 095 level of confidence for estimated
population mean.
Climate science literacy & general science literacy
1
86th percentile
r = 0.32, p < 0.01
-1
0
50th percentile
14th percentile
2nd percentile
-2
Climate science literacy
2
98th percentile
-2
2nd percentile
-1
0
50th percentile
14th percentile
1
2
86th percentile
98th percentile
science comprehension
Science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N = 1957. Nationally representative sample, April/May
2014 (YouGov). X-axis is continuous “Ordinary Science Intelligence” scale formed by IRT-weighted responses to NSF & Pew
science literacy, Numeracy, and Cognitive Reflection Test items (α=0.83). CIs reflect 095 level of confidence for estimated
population mean. Source: Kahan, D. The Science Communication Measurement Problem, Adv. in Pol. Psych. (in press).
Climate science literacy & general science literacy
plus partisan identity . . .
< avg Left_Right
> avg Left_Right
1
86th percentile
r = 0.32, p < 0.01
-1
0
50th percentile
14th percentile
2nd percentile
-2
Climate science literacy
2
98th percentile
-2
2nd percentile
-1
0
50th percentile
14th percentile
1
2
86th percentile
98th percentile
science comprehension
Science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N = 1957. Nationally representative sample, April/May
2014 (YouGov). X-axis is continuous “Ordinary Science Intelligence” scale formed by IRT-weighted responses to NSF & Pew
science literacy, Numeracy, and Cognitive Reflection Test items (α=0.83). CIs reflect 095 level of confidence for estimated
population mean. Source: Kahan, D. The Science Communication Measurement Problem, Adv. in Pol. Psych. (in press).
“How much risk do you believe global warming poses to human health,
safety, or prosperity?”
11
1.6
1.6
Very Conservative
Strong Republican
7
6
5
4
7
6
5
> avg. Left_right
2
1
3
6
5
4
4
7
3
0
None at all
2
Very low
2 6 3
Low
1
1
5
and moderate
0
= 0.07, p < 0.01 Between low
< avg. Left_right
>< avg. Left_right
0
0.65, p < 0.01
Moderate
4
Between moderate
and high
0
00
Very low
3
33
-1.6
Very liberal
Strong Democrat
3
High
2
Extremely high
risk
7
Global warming
6
66
-1
9
99
12
12
0
15
15
1
18
18
Science Comprehension
Conservative
Moderate
Liberal
Democrat
Independent
Republican
21
21
1.6
Very high
Very Conservative
Strong Republican
Climate science literacy & general science literacy
plus partisan identity . . .
< avg Left_Right
> avg Left_Right
1
86th percentile
r = 0.32, p < 0.01
-1
0
50th percentile
14th percentile
2nd percentile
-2
Climate science literacy
2
98th percentile
-2
2nd percentile
-1
0
50th percentile
14th percentile
1
2
86th percentile
98th percentile
science comprehension
Science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N = 1957. Nationally representative sample, April/May
2014 (YouGov). X-axis is continuous “Ordinary Science Intelligence” scale formed by IRT-weighted responses to NSF & Pew
science literacy, Numeracy, and Cognitive Reflection Test items (α=0.83). CIs reflect 095 level of confidence for estimated
population mean. Source: Kahan, D. The Science Communication Measurement Problem, Adv. in Pol. Psych. (in press).
What do “climate scientists believe ...”?
0%
0%
0%
0%
Climate science literacy: item response functions
“Climate scientists believe that the increase of
atmospheric carbon dioxide associated with the burning
of fossil fuels will reduce photosynthesis by plants.” [True
or False]
.7
.6
1
Liberal
Democrat
0
0
.1
.2
.3
.4
.5
Conservative
Republican
Conservative
Republican
correct
.1 probability
.2 .3 .4 of.5
.6 answer
.7 .8 .9
Liberal
Democrat
.8
probability of correct answer
.9
1
“What gas do most scientists believe causes
temperatures in the atmosphere to rise? Is it [hydrogen,
helium, carbon dioxide, radon]?”
-2
-1.514th percentile
-1
-.5 50th percentile
00
.5 86th percentile
11
1.5 98th percentile
22
2nd percentile
Climate science literacy score
-2
-1
-1.514th percentile
-.5 50th percentile
0
.5 86th percentile
1
1.598th percentile
2
2nd percentile
Climate science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N = 1,769. Predicted probabilities derived via Monte Carlo Simulation based on logistic
regression. Nationally representative sample, April/May 2014 (YouGov). Political outlook predictor set at -1 SD & + 1 SD on “Left_right" scale for “liberal
democrat” and “conservative Republican,” respectively. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
Climate science literacy: item response functions
.7
11
Conservative
Republican
Liberal
Democrat
Conservative
Republican
00
0
.1
.2
.3
.4
.5
.6
“Climate scientists believe thathuman-caused global
warming will result in flooding of many coastal regions .”
[True or False]
answer
correct
.1 probability
.2 .3
.3 .4
.4 of.5
.5
.6 .7
.7 .8
.8 .9
.9
.1
.2
.6
Liberal
Democrat
.8
probability of correct answer
.9
1
“What gas do most scientists believe causes
temperatures in the atmosphere to rise? Is it [hydrogen,
helium, carbon dioxide, radon]?”
-2
-1.514th percentile
-1
-.5 50th percentile
00
.5 86th percentile
11
1.5 98th percentile
22
2nd percentile
-2
-1.514th percentile
-1
-.5 50th percentile
0
.5 86th percentile
1
1.598th percentile
2
2nd percentile
Climate science literacy score
Climate science literacy score
-2
-1
0
1
2
Annenberg Center for Public Policy & Cultural Cognition Project. N = 1,769. Predicted probabilities derived via Monte Carlo Simulation based on logistic
regression. Nationally representative sample, April/May 2014 (YouGov). Political outlook predictor set at -1 SD & + 1 SD on “Left_right" scale for “liberal
democrat” and “conservative Republican,” respectively. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
“97% consesnsus” social marketing campaign
Climate science literacy: item response functions
.7
11
Conservative
Republican
Liberal
Democrat
Conservative
Republican
00
0
.1
.2
.3
.4
.5
.6
“Climate scientists believe thathuman-caused global
warming will result in flooding of many coastal regions .”
[True or False]
answer
correct
.1 probability
.2 .3
.3 .4
.4 of.5
.5
.6 .7
.7 .8
.8 .9
.9
.1
.2
.6
Liberal
Democrat
.8
probability of correct answer
.9
1
“What gas do most scientists believe causes
temperatures in the atmosphere to rise? Is it [hydrogen,
helium, carbon dioxide, radon]?”
-2
-1.514th percentile
-1
-.5 50th percentile
00
.5 86th percentile
11
1.5 98th percentile
22
2nd percentile
-2
-1.514th percentile
-1
-.5 50th percentile
0
.5 86th percentile
1
1.598th percentile
2
2nd percentile
Climate science literacy score
Climate science literacy score
-2
-1
0
1
2
Annenberg Center for Public Policy & Cultural Cognition Project. N = 1,769. Predicted probabilities derived via Monte Carlo Simulation based on logistic
regression. Nationally representative sample, April/May 2014 (YouGov). Political outlook predictor set at -1 SD & + 1 SD on “Left_right" scale for “liberal
democrat” and “conservative Republican,” respectively. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
Climate science literacy: item response functions
.7
1
Conservative
Republican
0
0
.1
.2
.3
.4
.5
.6
There is “solid evidence” of global warming due to “human activity
such as burning fossil fuels” [agree, disagree]
correct
.1 probability
.2 .3 .4 of .5
.6 answer
.7 .8 .9
Liberal
Democrat
.8
probability of correct answer
.9
1
“What gas do most scientists believe causes
temperatures in the atmosphere to rise? Is it [hydrogen,
helium, carbon dioxide, radon]?”
-2
-1.514th percentile
-1
-.5 50th percentile
00
.5 86th percentile
11
1.5 98th percentile
22
2nd percentile
Climate science literacy score
-2
2nd percentile
-1
14th percentile
0
50th percentile
1
86th percentile
2
98th percentile
Climate science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N = 1,769. Predicted probabilities derived via Monte Carlo Simulation based on logistic
regression. Nationally representative sample, April/May 2014 (YouGov). Political outlook predictor set at -1 SD & + 1 SD on “Left_right" scale for “liberal
democrat” and “conservative Republican,” respectively. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
Climate science literacy: item response functions
.7
1
Conservative
Republican
Liberal
Democrat
Conservative
Republican
0
0
.1
.2
.3
.4
.5
.6
There is “solid evidence” of global warming due to “human activity
such as burning fossil fuels” [agree, disagree]
correct
.1 probability
.2 .3 .4 of.5
.6 answer
.7 .8 .9
Liberal
Democrat
.8
probability of correct answer
.9
1
“What gas do most scientists believe causes
temperatures in the atmosphere to rise? Is it [hydrogen,
helium, carbon dioxide, radon]?”
-2
-1.514th percentile
-1
-.5 50th percentile
00
.5 86th percentile
11
1.5 98th percentile
22
2nd percentile
-2
-1
0
.5 86th percentile
1
1.598th percentile
2
2nd percentile-1.514th percentile-.5 50th percentile
Climate science literacy score
Climate science literacy score
Annenberg Center for Public Policy & Cultural Cognition Project. N = 1,769. Predicted probabilities derived via Monte Carlo Simulation based on logistic
regression. Nationally representative sample, April/May 2014 (YouGov). Political outlook predictor set at -1 SD & + 1 SD on “Left_right" scale for “liberal
democrat” and “conservative Republican,” respectively. Colored bars reflect 0.95 confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
What am I talking about? ...
scaling up SENCER . . . by leveraging what it already does
I. Two science literacy “measurement problems”
II. Reason and identity protection
III. The “disentanglement principle”
IV. SENCER and the disentanglement agenda
What am I talking about? ...
scaling up SENCER . . . by leveraging what it already does
I. Two science literacy “measurement problems”
II. Reason and identity protection
III. The “disentanglement principle”
IV. SENCER and the disentanglement agenda
Measuring who we are rather than what we know . . .
Not too little rationality . . .
Not too little rationality . . . but too much
“Motivated system 2 reasoning”
National Science Foundation, SES-0922714
What am I talking about? ...
scaling up SENCER . . . by leveraging what it already does
I. Two science literacy “measurement problems”
II. Reason and identity protection
III. The “disentanglement principle”
IV. SENCER and the disentanglement agenda
What am I talking about? ...
scaling up SENCER . . . by leveraging what it already does
I. Two science literacy “measurement problems”
II. Reason and identity protection
III. The “disentanglement principle”
IV. SENCER and the disentanglement agenda
This measures who we are . . .
This measures who we are . . . so measure what we know
instead
Disentanglement in teaching evolution
Disentanglement principle:
“Don’t make reasoning, free people choose between
knowing what’s known & being who they are!”
Disentanglement principle:
“Don’t make reasoning, free people choose between
knowing what’s known & being who they are!”
Disentanglement principle:
“Don’t make reasoning, free people choose between
knowing what’s known & being who they are!”
Disentanglement principle:
“Don’t make reasoning, free people choose between
knowing what’s known & being who they are!”
Cultural Cognition Project
SE Fla. evidence-based science communication initiative
Soute
“How much risk do you believe global warming poses to human health,
safety, or prosperity?”
7
A polluted science communication environment . . .
7
67
5
6
< avg. Left_right
56
High
5
4
4
5
45
< avg. Left_right
3
7
34
Moderate
23
1
2
6
22
> avg. Left_right
2
3
Between low
and moderate
2 6 3
44
Egalitarian communitarian
r = 0.07, p < 0.01
Hierarch individualist
33
4
7
55
Between moderate
and high
r = - 0.65, p < 0.01
12
Low
01
0
0
0
4
4
0
None at all
no risk
> avg. Left_right
0
Very low
1
5
11
5
1
4 SE Fla. Counties
00
None at all
7
7
6
77
Extremely high
risk
66
Extremely high
risk
6
United States as
a whole
Global
warming (summer 2013)
0
0
3
33 0
00
3
> avg. Left_right
2 < 3
4
5
1
6
66 3
6
9
9
99 6
12
129
6
12
15
1512
15
7
18
15
18
8
18
9
21
18
21
21
21
10
6
6
5
5
< avg. Left_right
4
4
4
4
7
0
0
Moderate
123 15 6 18
219
12
15
21
<18avg. Left_right
2
2
Low
> avg. Left_right
3
6 09
3
3
3
and moderate
2
0
2 6 3
r = 0.07,
< 0.01p < 0.01Between low
r = -p0.60,
6
7
7
7
6
5
2
6
1
5
1
7
p < 0.01
Liberal
Democrat
1.6
1.61.6
-1.6
-1-1
00
11
1.6
1.6
-1.6
no risk Very-1.6
Left_right Moderate
Extremely
0
1
1.6Conservative
liberal -1
Very
at all Strong
Conservative Very
DemocratLiberal
high
risk
Strong
Republican
Independent
Very liberal
Conservative
Moderate
Strong Democrat Democrat
Independent
Republican
1
1
1
5
None at all
Very Conservative
Strong Republican
Strong Republican
0
0
11
Conservative
Republican
>< avg. Left_right
0
00
Moderate
Independent
0
-1
-1
1
5
> avg. Left_right
Very low
4
-1.6
Very liberal
Strong Democrat
r =p-< 0.01
0.65,
r = 0.07,
Between moderate
and high
0033
9
6
3
0
00
Very low
3
None at all
r = - 0.65, p < 0.01
4
11
None at all
Extremely high
risk
High
3
Very low
00
Low
0 00 1 11 2 22 3 33 4 44 5 55 6 66 7 77
4
4
Between low
and moderate
2
7
7
5
5
Moderate
22
Between moderate
and high
33
6
6
Extremely high
risk
High
7
Southeast Florida (Fall 2013)
or prosperity?”
Extremely high
risk
3
3
Extremely
-1.6
00
11
1.6
-1.6at all -1-1
1.6
noat risk
Extremely
all no risk
high risk
Very low
Comprehension Very high
“How much risk do you at
believe
fluoridated
water
poses
to
human
ScienceScience
Comprehension
Very
liberal
Very
Conservative
Moderate
all
high
risk “How much risk do you
believe
medical x-rays poses to human
Strong Democrat
Strong Republican
Independent
health, safety, or prosperity?”
“How muchone
risk. do
warming
to human health, safety,
health,
safety, poses
or prosperity?”
Global
warming
An unpolluted
. . you believe global
3366
6699
12
9912
15
15
12
12
18
18
15
15
Science
Comprehension
Science
Comprehension
21
21
18
21
18
21
Very high
11
4 SE Fla. Counties
“Landuse planners should identify assess and
revise existing laws to assure that they reflect the
risks posed by rising sea level and extreme
weather.”
“Local and state officials should be involved in
identifying steps that local communities can
take to reduce the risk posed by rising sea
levels.”
pct. agree
pct. agree
> avg. Left_right
78% agree
Kahan, D. The Science Communication Measurement Problem, Adv. in Pol. Psych. (in press)
Measuring what we know not who we are . . .
Disentanglement principle:
“Don’t make reasoning, free people choose between
knowing what’s known & being who they are!”
What should climate communicators communicate to citizens?
What should climate communicators communicate to citizens?
What should climate communicators communicate to citizens?
What should climate communicators communicate to citizens?
Katie’s “ ‘Compact connector’ scouting report” form
PB County Examples
•
•
•
•
•
Corporate Exec
HOA Leader
Architect
Community Organizer
COBWRA Leader
•
•
•
•
•
Construction Manager
Hotel President
Marina Director
Surf Club Leader
Investment Manager
Communicate normality
Communicate normality
Proselytizing the normality of climate science
Local businessman
Corp. exec.
Homeowner
climate scientist
Communicate normality
Not “us vs. them”
just us—using what we know
What am I talking about? ...
scaling up SENCER . . . by leveraging what it already does
I. Two science literacy “measurement problems”
II. Reason and identity protection
III. The “disentanglement principle”
IV. SENCER and the disentanglement agenda
What am I talking about? ...
scaling up SENCER . . . by leveraging what it already does
I. Two science literacy “measurement problems”
II. Reason and identity protection
III. The “disentanglement principle”
IV. Leveraging SENCER
Solving the “measurement problem”
Sites of civic science communication disentanglement
What am I talking about? ...
scaling up SENCER . . . by leveraging what it already does
I. Two science literacy “measurement problems”
II. Reason and identity protection
III. The “disentanglement principle”
IV. Leveraging SENCER
Dan M. Kahan
Yale Law School
Donald Braman
George Washington University
John Gastil
University of Washington
Geoffrey Cohen
Stanford University
Paul Slovic
University of Oregon
Ellen Peters
Ohio State University
Hank Jenkins-Smith
University of Oklahoma
David Hoffman
Temple Law School
Gregory Mandel
Temple Law School
Maggie Wittlin
Cultural Cognition Project Lab
Lisa Larrimore-Ouelette
Cultural Cognition Project Lab
Danieli Evans
Cultural Cognition Project Lab
June Carbone
Univ. Missouri-Kansas City
Michael Jones
Safra Ethics Center, Harv. Univ.
Naomi Cahn
George Washington University
Jeffrey Rachlinksi
Cornell Law School
John Byrnes
Cultural Cognition Project Lab
John Monahan
University of Virginia
www. culturalcognition.net
“I am you!”