Watch in slide show mode to observe (modest) animation.
comments questions: dan.kahan@yale.edu
papers, etc: www.culturalcognition.net
www.culturalcognition.net
The Science Communication Measurement
Problem
Dan M. Kahan
Yale University
& many x 103 others
Research Supported by:
National Science Foundation, SES-0922714
Annenberg Center for Public Policy
Skoll Global Threats Fund
What am I talking about? ...
What am I talking about? ...
Beliefs on global temperature “increase in recent decades”
100%
75%
50%
25%
0%
Human caused
Naturally caused
No warming
N = 1,885. Annenberg Public Policy Center & Cultural Cognition Project. Nationally
representative sample, June 2014 (YouGov). CIs are 0.95 confidence intervals for
estimated general population means.
What am I talking about? ...
I. The #scicomm measurement problem
II. The “disentanglement principle”
III. Cognitive dualism & communicative
pluralism
What am I talking about? ...
I. The #scicomm measurement problem
II. The “disentanglement principle”
III. SE Fla. climate political science
IV. Krista “disbelieves” evolution—but loves Your
Inner Fish!*
What am I talking about? ...
I. The #scicomm measurement problem
II. The “disentanglement principle”
III. SE Fla. climate political science
IV. Krista “disbelieves” evolution—but loves Your
Inner Fish!*
*time permitting!
What am I talking about? ...
I. The #scicomm measurement problem
II. The “disentanglement principle”
III. SE Fla. climate political science
IV. Krista “disbelieves” evolution—but loves Your
Inner Fish!*
*time permitting!
What am I talking about? ...
I. The #scicomm measurement problem
II. The “disentanglement principle”
III. SE Fla. climate political science
IV. Krista “disbelieves” evolution—but loves Your
Inner Fish!*
*time permitting!
What am I talking about? ...
I. The #scicomm measurement problem
II. The “disentanglement principle”
III. SE Fla. climate political science
IV. Krista “disbelieves” evolution—but loves Your
Inner Fish!*
*time permitting!
What am I talking about? ...
I. The #scicomm measurement problem
II. The “disentanglement principle”
III. SE Fla. climate political science
IV. Krista “disbelieves” evolution—but loves Your
Inner Fish!*
*time permitting!
What does “belief in” climate change meausure?
Beliefs on global temperature “increase in recent decades”
100%
75%
50%
25%
0%
Human caused
Naturally caused
No warming
N = 1,885. Annenberg Public Policy Center & Cultural Cognition Project. Nationally
representative sample, June 2014 (YouGov). CIs are 0.95 confidence intervals for
estimated general population means.
What does “belief in” evolution meausure?
“Ordinary Science Intelligence”
Assessment
OSI_1.0
OSI_2.0
“Ordinary Science Intelligence” Assessment
18 items
• 6 “Basic facts” (NSF Indicators, Pew)
• 3 “Theory of science” (NSF Indicators)
• 6 Numeracy (Peters et al. 2006)
• 3 Cognitive reflection (Frederick 2005)
0
1
2
3
4
5
6
7
8
Dimensionality (principal factor)
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18
Factor
“Ordinary Science Intelligence” Assessment
18 items
• 6 “Basic facts” (NSF Indicators, Pew)
• 3 “Theory of science” (NSF Indicators)
• 6 Numeracy (Peters et al. 2006)
• 3 Cognitive reflection (Frederick 2005)
0
1
2
3
4
5
6
7
8
Dimensionality (principal factor)
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18
Factor
Item response profiles
“Ordinary Science Intelligence” Assessment
18 items
• 6 “Basic facts” (NSF Indicators, Pew)
• 3 “Theory of science” (NSF Indicators)
• 6 Numeracy (Peters et al. 2006)
• 3 Cognitive reflection (Frederick 2005)
0
1
2
3
4
5
6
7
8
Dimensionality (principal factor)
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18
Factor
Ordinary science intelligence: item response functions
bars denote 0.95 CIs
Ordinary science intelligence: item response functions
bars denote 0.95 CIs
Ordinary science intelligence: item response functions
bars denote 0.95 CIs
Climate science-communication measurement problem:
Ordinary science intelligence: item response functions
bars denote 0.95 CIs
Ordinary science intelligence: item response functions
bars denote 0.95 CIs
Ordinary science intelligence: item response functions
bars denote 0.95 CIs
Ordinary science intelligence: item response functions
bars denote 0.95 CIs
“Religiosity” (Cronbach’s α = 0.80)
OSI_2.0 items plus Evolution & Religiosity items
Ordinary science intelligence: item response functions
bars denote 0.95 CIs
Ordinary science intelligence: item response functions
bars denote 0.95 CIs
Ordinary science intelligence: item response functions
bars denote 0.95 CIs
Teaching evolution to “nonbelievers”
Evolution science-communication measurement problem:
What we know
bars denote 0.95 CIs
vs.
Who we are
“Belief” in global warming
“Belief” in global warming
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
“Belief” in global warming
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
7
7
676
77
“Belief” in global warming
6
5675
Extremely high
risk
2
1
6
5
None at all
5
5
456747
4
76
34563
4
7
2
3
2 6 3
564
675
12341
23452
> avg. Left_right
00
33
0
0
3
6
Very
low
Very
low
Very
low
12 015 18 21 3
6
66 6
99
9
99 9
>< avg. Left_right
12
12
15
15
18
18
21
21
12
1212
15
1515
18
1818
21
2121
Very high
> avg.
Left_right
Science
Ordinary
Science Intelligence
ScienceComprehension
6
9comprehension
12
15
18Very high
21
Science
Comprehension
3
9
66
3
33 3
0120
1
0
5
4
000
0
4
3
> avg. Left_right
< avg. Left_right
1
0
0
2
2
1
01
3
2
Kahan,
M., Slovic, P.,
Ouellette,
D.
0 3 D.M.,
6 9 Peters,
12 015 18 E.,
21 Wittlin,
12 L.L., Braman,
15
18& Mandel,
21
0 impact
33 science66literacy and
99 numeracy
12
15
18
G. The polarizing
of
on perceived
climate 21
change risks. Nature Climate Change 2, 732-735 (2012).
0
3
6
9
12
15
18
21
0
3
6
0
3
6
9
Very low
12 015 18
Very high
21
1
0
9
12
15
18
3
6
9 Comprehension
12
15
Science
18
21
21
0
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
< avg. Left_right
1
4 0
5
1231
453
01230
0
012342
0
1
3
2
7
6
4
3
7
5
4
6
5
7
6
7
global warming risk
6
< avg. Left_right
0
3
6
9
12
15
18
21
r = - 0.65, p < 0.01
6
5
4
7
6
5
4
7
< avg. Left_right
2
3
> avg. Left_right
1
5
1
2 6 3
global warming risk
Extremely high
risk
7
“Belief” in global warming
0
0
4
None at all
>< avg. Left_right
0
00
3
33
6
66
Very
low
Very
low
12
12
15
15
18
18
Science
ScienceComprehension
comprehension
21
21
1
2
3
Very high
0
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
9
99
0
3
6
9
12
15
18
21
6
5
4
7
6
5
4
7
< avg. Left_right
2
3
> avg. Left_right
1
5
1
2 6 3
global warming risk
Extremely high
risk
7
“Belief” in global warming
0
0
4
None at all
>< avg. Left_right
0
00
3
33
6
66
Very
low
Very
low
12
12
15
15
18
18
Science
ScienceComprehension
comprehension
21
21
1
2
3
Very high
0
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
9
99
0
3
6
9
12
15
18
21
67
76
7
56
675
6
45
564
5
4
3
> avg. Left_right
2
231 6 23
342
12
1
120
5
01
.2
00
33
0
0
04 0
1
-3
< avg. Left_right
> avg. Left_right
3
2
-2
0
000
66
3
-1
333
6
0
666
>< avg. Left_right
99
9
1
999
12
12
15
15
18
18
21
21
12
12
12
15
15
15
18
18
18
21
21
21
2
3
2
3
Very
Very low
Very high
Science Comprehension
Verylowlow
Very
high
0 3 6 9 12 015 18 21 3
6Science9 comprehension
12
15
18
21
Science
Comprehension
Kahan, D.M., Peters, E., Wittlin, M., Slovic, P., Ouellette, L.L., Braman, D. & Mandel,
0 The
3 6 polarizing
9 12 15 impact
18 21 of science literacy and numeracy on perceived climate
G.
change risks. Nature Climate Change 2, 732-735 (2012).
1
.1
0
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
0
1
0
2
1
None at all
< avg. Left_right
0
.7
.6
global warming risk
global warming risk
6
5
.5
.4
5
4
4
3
.3
7
6
7
.8
.9
1
Extremely high
risk
453
7
34
77
“Belief” in global warming
0
3
6
9
12
15
18
21
“Belief” in global warming
77
There is “solid evidence” of recent global warming
due “mostly” to “human activity such as burning
fossil fuels.”
0
3
3
3
.6
< avg. Left_right
.3
Conservative
Republican
.2
> avg. Left_right
< avg. Left_right
.1
0
9
9
3
6
9
0
3
6
9
0
3
6
9
0
3
6
9
12
15
18
21
00
33 > avg.66 Left_right
99
12
15
18
21
1st percentile 16th-1
percentile 50th
percentile
-2.5
0 percentile 84th
1 percentile 99th2.5
0
3
6
9
12
15
18
21
12Intelligence
15
18
21
Science
avg.
00 low
33 >Ordinary
66 Left_right
99
12
15
18
21
Very
Very
high
12 015 18 21 3
6
9 Comprehension
12
15
18
21
Science
0
3
6
9
12
15
18
21
0
3
12
15
18
21
66
99
12
15
18
21
12 015 18 21 3
Very low
12 015 18
Very high
21
0
12 15 low18 21
Very
015 18 21
12 0
01
0
0
12
18
21
Very low
12 015 18
21
0
1
0
2
1
< avg. Left_right
.4
9
6
Liberal
Democrat
.5
9
6
6
< avg. Left_right
.7
1203
11 0
6
203
75532342 12316
7566
142533
4
14220 01 03
4421231 0120536
4
10120 0142536
43453 23427567754564 34536765675 456477676 5675 77 676
Probability of correct response
0
4
3
2
1
1
0
0
0
3
2
3
.8
01200
7
7
6
7
6
5
7
6
5
4
1
0
2
1
3
2
4
3
5
4
6
5
5
4
6
5
7
6
7
3
2 0
4
3 1
0
5
4 2
1
6
5 3
2
7
6 4
3
7
0
.9
15
0
12
15
18
3
3
6 Left_right
9 Comprehension
12
15
18
Science
> avg.
33
12
15
Science
66
99 Comprehension
12
15
6
9
12
15
18
21
21
Very high
3
6
9
12
15
18
18
21
21
18
21
Very high
0
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
1
21
3
6
9 Comprehension
12
15
Science
18
21
“Belief” in global warming
Probability of correct response
There is “solid evidence” of recent global warming
due “mostly” to “human activity such as burning
fossil fuels.”
Annenberg Center for Public Policy & Cultural Cognition Project. N’ = 1957.
Nationally representative sample, April/May 2014 (YouGov). CIs reflect 0.95
confidence intervals. Source: Kahan, D. The Science Communication
Measurement Problem, Adv. in Pol. Psych. (in press).
1
.9
.8
Liberal
Democrat
.7
.6
.5
.4
.3
Conservative
Republican
.2
.1
0
1st percentile
-2.5
16th-1
percentile 50th
percentile
0 percentile 84th
1 percentile 99th2.5
Ordinary Science Intelligence
“Belief” in global warming
There is “solid evidence” of global warming due to “human
activity such as burning fossil fuels” [agree, disagree]
Unconfounding measurement of identity & knowledge
“Human beings, as we know them today,
developed from earlier species of animals.”
(True/false)
probability of correct answer
11
.9
.9
.8
.8
Below avg.
religiosity
.7
.7
.6
.6
.5
.5
.4
.4
Above avg.
religiosity
.3
.3
.2
.2
.1
.1
00
-2.5 14th percentile
-1
2nd percentile
-2.5
-1
50th percentile
00
86th
11 percentile
Ordinary science intelligence
2.5percentile
98th
2.5
“Ordinary Climate Science Intelligence” battery
“Ordinary Climate Science Intelligence” battery
“What gas do most scientists believe causes temperatures
in the atmosphere to rise? Is it [hydrogen, helium, carbon
dioxide, radon]?”
“Ordinary Climate Science Intelligence” battery
“What gas do most scientists believe causes temperatures
in the atmosphere to rise? Is it [hydrogen, helium, carbon
dioxide, radon]?”
“Climate scientists believe that human-caused global
warming will result in flooding of many coastal regions.”
[True or False]
“Climate scientists believe that human-caused global
warming will increase the risk of skin cancer in human
beings.” [True or false]
“Ordinary Climate Science Intelligence” battery
“Ordinary climate science intelligence” item response curves
“Climate scientists believe that the
increase of atmospheric carbon dioxide
associated with the burning of fossil
fuels will reduce photosynthesis by
plants.” [True or False]
1
of correct answer
probability
.1 .2 .3 .4 .5 .6 .7 .8 .9
.8
.7
.6
.5
.4
.3
.2
-1
-.5
0
.5
1
1.5
0
2
-2
-1
-.5
0
.5
1
1.5
2
-2
-1
-.5
0
.5
1
1.5
2
-1
-.5
0
.5
1
1.5
-2
2
0
.5
1
1.5
2
2
-1.5
-1
-.5
0
.5
1
1.5
2
1
.1
.2
.3
.4
.5
.6
.7
.8
.9
probability of correct answer
0
-.5
1.5
“Climate scientists believe that globally
averaged surface air temperatures were
higher for the first decade of the twentyfirst century (2000-2009) than for the
last decade of the twentieth century
(1990-1999) [True or false]
1
of correct answer
probability
.1 .2 .3 .4 .5 .6 .7 .8 .9
1
.9
.8
.7
.6
.5
.4
.3
.2
-1
Ordinary climate science intellience
1
Ordinary climate science intellience
“Climate scientists believe that here will
be positive as well as negative effects
from human-caused global warming.”
[True or false]
.1
-1.5
.5
1
-1.5
Ordinary climate science intellience
0
-2
0
of correct answer
probability
.1 .2 .3 .4 .5 .6 .7 .8 .9
-2
Ordinary climate science intellience
“Climate scientists believe that
nuclear power generation
contributes to global warming”
[True or false]
-.5
0
of correct answer
probability
.1 .2 .3 .4 .5 .6 .7 .8 .9
0
0
-1.5
-1
“Climate scientists believe that
human-caused global warming has
increased the number and severity
of hurricanes around the world in
recent decades.” [True or false]
1
1
.9
.8
.7
.6
.5
.4
.3
.2
.1
-2
-1.5
Ordinary climate science intellience
“Climate scientists believe that if the
North Pole icecap melted as a result of
human-caused global warming, global
sea levels would rise.” [True or False]
“Climate scientists believe that
human-caused global warming will
result in flooding of many coastal
regions .” [True or False]
probability of correct answer
-1.5
Ordinary climate science intellience
Ordinary climate science intellience
-2
-1.5
-1
-.5
0
.5
1
1.5
Ordinary climate science intellience
2
0
-1.5
0
.1
-2
probability of correct answer
“Climate scientists believe that
human-caused global warming will
increase the risk of skin cancer in
human beings.” [True or False]
1
.9
probability of correct answer
.9
.8
.7
.6
.5
.4
.3
.2
.1
0
probability of correct answer
1
“What gas do most scientists believe
causes temperatures in the
atmosphere to rise? Is it [hydrogen,
helium, carbon dioxide, radon]?”
-2
-1.5
-1
-.5
0
.5
1
1.5
Ordinary climate science intellience
Figures plot the predicted probability of correctly responding to the item conditional on score on OCSI scale. Black bars
denote 0.95 CIs. The figures can thus be used to assess the relative difficulty of the items and the precision with which
they measure differences in comprehension.
2
0
0
.1
.1
.2
.2
.3
.3
.4
.4
.5
.5
.6
.6
.7
.7
.8
.8
.9
.9
1
1
Ordinary climate science intelligence: item response functions
-2
bars denote 0.95 CIs
-1
0
1
2
-2
-1
0
1
2
0
.1
.2
.3
.4
.5
.6
.7
.8
.9
1
Ordinary climate science intelligence: item response functions
-2
bars denote 0.95 CIs
-1
0
1
2
Ordinary climate science intelligence: item response functions
bars denote 0.95 CIs
9 climate science intelligence & positions on global warming
Ordinary
8
7
9
6
8
No. correct
5
7
6
4
59
3
48
2
37
26
1
15
0
04
3
2
1
0
Human caused Naturally caused No warming
Positions on global warming in “past few decades”
2
99th percentile
r = 0.32, p < 0.01
1
86th percentile
-1
0
50th percentile
14th percentile
1st percentile
-2
Ordinary climate science intelligence
Ordinary climate science intelligence & positions on global warming
2
1
0
-1
-2
1st percentile
14th percentile
50th percentile
99th percentile
86th percentile
science comprehension
shaded area denotes 0.95 CIs
Ordinary science intelligence
Ordinary climate science intelligence & general science
literacy & partisan identity . . .
> avg Left_Right
2
99th percentile
r = 0.32, p < 0.01
1
86th percentile
-1
0
50th percentile
14th percentile
1st percentile
-2
Ordinary climate science intelligence
< avg Left_Right
2
1
0
-1
-2
1st percentile
14th percentile
50th percentile
99th percentile
86th percentile
science comprehension
shaded area denotes 0.95 CIs
Ordinary science intelligence
shaded area denotes 0.95 CIs
Probability of correct response
There is “solid evidence” of recent global warming
due “mostly” to “human activity such as burning
fossil fuels.”
1
.9
.8
Liberal
Democrat
.7
.6
.5
.4
.3
Conservative
Republican
.2
.1
0
1st percentile
-2.5
16th-1
percentile 50th
percentile
0 percentile 84th
1 percentile 99th2.5
Ordinary Science Intelligence
Ordinary climate science intelligence & general science
literacy & partisan identity . . .
> avg Left_Right
22
99th percentile
r = 0.32, p < 0.01
11
86th percentile
-1
-1
00
50th percentile
14th percentile
1st percentile
-2
-2
Ordinary climate science intelligence
< avg Left_Right
1st percentile
14th percentile
50th percentile
99th percentile
86th percentile
-2-2
-1-1
00
11
22
comprehension
sciencecomprehension
science
shaded area denotes 0.95 CIs
Ordinary science intelligence
“Climate scientists believe that . . . ”—true or false?
< avg Left_Right
> avg Left_Right
Percent giving correct response
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
globally
averaged
surface air
temperatures
were higher for
the first decade
of the twentyfirst century
(2000-2009)
than for the last
decade of the
twentieth
century (19901999) True
human-caused
global warming
will result in
flooding of
many coastal
regions True
carbon dioxide
causes
temperatures in
the atmosphere
to rise True
nuclear power
generation
contributes to
global warming
False
there will be
positive as well
as negative
effects from
human-caused
global warming
True
human-caused
global warming
will increase the
risk of skin
cancer in
human beings
False
the increase of
atmospheric
carbon dioxide
associated with
the burning of
fossil fuels will
reduce
photosynthesis
by plants False
human-caused
global warming
has increased
the number and
severity of
hurricanes
around the
world in recent
decades False
if the north pole
icecap melted
as a result of
human-caused
global warming,
global sea
levels would
rise false
“Climate scientists believe that . . . ”—true or false?
< avg Left_Right
> avg Left_Right
Percent giving correct response
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
globally
averaged
surface air
temperatures
were higher for
the first decade
of the twentyfirst century
(2000-2009)
than for the last
decade of the
twentieth
century (19901999) True
human-caused
global warming
will result in
flooding of
many coastal
regions True
carbon dioxide
causes
temperatures in
the atmosphere
to rise True
nuclear power
generation
contributes to
global warming
False
there will be
positive as well
as negative
effects from
human-caused
global warming
True
human-caused
global warming
will increase the
risk of skin
cancer in
human beings
False
the increase of
atmospheric
carbon dioxide
associated with
the burning of
fossil fuels will
reduce
photosynthesis
by plants False
human-caused
global warming
has increased
the number and
severity of
hurricanes
around the
world in recent
decades False
if the north pole
icecap melted
as a result of
human-caused
global warming,
global sea
levels would
rise false
“Climate scientists believe that . . . ”—true or false?
< avg Left_Right
> avg Left_Right
Percent giving correct response
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
globally
averaged
surface air
temperatures
were higher for
the first decade
of the twentyfirst century
(2000-2009)
than for the last
decade of the
twentieth
century (19901999) True
human-caused
global warming
will result in
flooding of
many coastal
regions True
carbon dioxide
causes
temperatures in
the atmosphere
to rise True
nuclear power
generation
contributes to
global warming
False
there will be
positive as well
as negative
effects from
human-caused
global warming
True
human-caused
global warming
will increase the
risk of skin
cancer in
human beings
False
the increase of
atmospheric
carbon dioxide
associated with
the burning of
fossil fuels will
reduce
photosynthesis
by plants False
human-caused
global warming
has increased
the number and
severity of
hurricanes
around the
world in recent
decades False
if the north pole
icecap melted
as a result of
human-caused
global warming,
global sea
levels would
rise false
“Climate scientists believe that . . . ”—true or false?
< avg Left_Right
> avg Left_Right
Percent giving correct response
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
globally
averaged
surface air
temperatures
were higher for
the first decade
of the twentyfirst century
(2000-2009)
than for the last
decade of the
twentieth
century (19901999) True
human-caused
global warming
will result in
flooding of
many coastal
regions True
carbon dioxide
causes
temperatures in
the atmosphere
to rise True
nuclear power
generation
contributes to
global warming
False
there will be
positive as well
as negative
effects from
human-caused
global warming
True
human-caused
global warming
will increase the
risk of skin
cancer in
human beings
False
the increase of
atmospheric
carbon dioxide
associated with
the burning of
fossil fuels will
reduce
photosynthesis
by plants False
human-caused
global warming
has increased
the number and
severity of
hurricanes
around the
world in recent
decades False
if the north pole
icecap melted
as a result of
human-caused
global warming,
global sea
levels would
rise false
Ordinary climate science intelligence: item response functions
bars denote 0.95 CIs
Ordinary climate science intelligence: item response functions
bars denote 0.95 CIs
Ordinary climate science intelligence: item response functions
bars denote 0.95 CIs
0
.1
.2
.3
.4
.5
.6
.7
.8
.9
1
Ordinary climate science intelligence: item response functions
-2
bars denote 0.95 CIs
-1
0
1
2
Climate science-communication measurement problem:
What we know
bars denote 0.95 CIs
vs.
Who we are
Evolution science-communication measurement problem:
What we know
vs.
Who we are
Teaching evolution to “nonbelievers”
“97% consesnsus” social marketing campaign
“Climate scientists believe that . . . ”—true or false?
< avg Left_Right
> avg Left_Right
Percent giving correct response
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
globally averaged
surface air
temperatures
were higher for
the first decade
of the twentyfirst century
(2000-2009)
than for the last
decade of the
twentieth
century (19901999) True
human-caused
global warming
will result in
flooding of many
coastal regions
True
carbon dioxide
causes
temperatures in
the atmosphere
to rise True
nuclear power
generation
contributes to
global warming
False
there will be
positive as well
as negative
effects from
human-caused
global warming
True
human-caused
global warming
will increase the
risk of skin
cancer in human
beings False
the increase of
atmospheric
carbon dioxide
associated with
the burning of
fossil fuels will
reduce
photosynthesis
by plants False
human-caused
global warming
has increased the
number and
severity of
hurricanes
around the world
in recent decades
False
Annenberg Center for Public Policy & Cultural Cognition Project. N = 1,769 Nationally representative sample, April/May 2014 (YouGov). “Left_right” is
continuous political outlook scale formed by aggregating responses to 7-point party identification item and 5-point “liberal-conservative” ideology item
(α=0.78). CIs reflect 0.95 level of confidence for estimated population mean. CIs reflect 0.95 level of confidence for estimated population mean.
if the north pole
icecap melted as
a result of
human-caused
global warming,
global sea levels
would rise false
“97% consesnsus” social marketing campaign
Teaching evolution to “nonbelievers”
The science-communication measurement problem:
What we know
vs.
Who we are
What am I talking about? ...
 I. The #scicomm measurement problem
II. The “disentanglement principle”
III. SE Fla. climate political science
IV. Krista “disbelieves” evolution—but loves Your
Inner Fish!*
*time permitting!
This measures who we are . . .
This measures who we are . . . so measure what we know
instead
Uncounfinding identity & knowledge in
evolution science education
Disentanglement principle:
“Don’t make reasoning, free people choose between
knowing what’s known & being who they are!”
Disentanglement principle:
“Don’t make reasoning, free people choose between
knowing what’s known & being who they are!”
Disentanglement principle:
“Don’t make reasoning, free people choose between
knowing what’s known & being who they are!”
Disentanglement principle:
“Don’t make reasoning, free people choose between
knowing what’s known & being who they are!”
Climate science-communication measurement problem:
What we know
vs.
Who we are
Climate science-communication measurement problem:
How to disentangle
What we know
Who we are
from
Climate science-communication measurement problem:
How to disentangle
What we know
Who we are
from
Climate science-communication measurement problem:
How to disentangle
What we know
Who we are
from
Beliefs on global temperature “increase in recent decades”
100%
75%
50%
25%
0%
Human caused
Naturally caused
No warming
N = 1,885. Annenberg Public Policy Center & Cultural Cognition Project. Nationally
representative sample, June 2014 (YouGov). CIs are 0.95 confidence intervals for
estimated general population means.
What am I talking about? ...
 I. The #scicomm measurement problem
 II.
The “disentanglement principle”
III. SE Fla. climate political science
IV. Krista “disbelieves” evolution—but loves Your
Inner Fish!*
*time permitting!
Climate science-communication measurement problem:
How to disentangle
What we know
Who we are
from
Beliefs on global temperature “increase in recent decades”
100%
75%
50%
25%
0%
Human caused
Naturally caused
No warming
N = 1,885. Annenberg Public Policy Center & Cultural Cognition Project. Nationally
representative sample, June 2014 (YouGov). CIs are 0.95 confidence intervals for
estimated general population means.
Southeast Florida . . .
Southeast Florida . . .
Soute
Evidence-based Policy Science Communication Initiative
“How much risk do you believe global warming poses to human health,
safety, or prosperity?”
7
A polluted science communication environment . . .
7
67
5
6
< avg. Left_right
56
High
5
4
4
5
45
< avg. Left_right
3
7
34
Moderate
23
1
2
6
22
> avg. Left_right
2
3
Between low
and moderate
2 6 3
44
Egalitarian communitarian
r = 0.07, p < 0.01
Hierarch individualist
33
4
7
55
Between moderate
and high
r = - 0.65, p < 0.01
12
Low
01
0
0
0
4
4
0
None at all
no risk
> avg. Left_right
0
Very low
1
5
11
5
1
4 SE Fla. Counties
00
None at all
7
7
6
77
Extremely high
risk
66
Extremely high
risk
6
United States as
a whole
Global
warming (summer 2013)
0
0
3
33 0
00
3
> avg. Left_right
2 < 3
4
5
1
6
66 3
6
9
9
99 6
12
129
6
12
15
1512
15
7
18
15
18
8
18
9
21
18
21
21
21
10
6
6
5
5
< avg. Left_right
4
4
4
4
7
0
0
Moderate
123 15 6 18
219
12
15
21
<18avg. Left_right
2
2
Low
> avg. Left_right
3
6 09
3
3
3
and moderate
2
0
2 6 3
r = 0.07,
< 0.01p < 0.01Between low
r = -p0.60,
6
7
7
7
6
5
2
6
1
5
1
7
p < 0.01
Liberal
Democrat
1.6
1.61.6
-1.6
-1-1
00
11
1.6
1.6
-1.6
no risk Very-1.6
Left_right Moderate
Extremely
0
1
1.6Conservative
liberal -1
Very
at all Strong
Conservative Very
DemocratLiberal
high
risk
Strong
Republican
Independent
Very liberal
Conservative
Moderate
Strong Democrat Democrat
Independent
Republican
1
1
1
5
None at all
Very Conservative
Strong Republican
Strong Republican
0
0
11
Conservative
Republican
>< avg. Left_right
0
00
Moderate
Independent
0
-1
-1
1
5
> avg. Left_right
Very low
4
-1.6
Very liberal
Strong Democrat
r =p-< 0.01
0.65,
r = 0.07,
Between moderate
and high
0033
9
6
3
0
00
Very low
3
None at all
r = - 0.65, p < 0.01
4
11
None at all
Extremely high
risk
High
3
Very low
00
Low
0 00 1 11 2 22 3 33 4 44 5 55 6 66 7 77
4
4
Between low
and moderate
2
7
7
5
5
Moderate
22
Between moderate
and high
33
6
6
Extremely high
risk
High
7
Southeast Florida (Fall 2013)
or prosperity?”
Extremely high
risk
3
3
Extremely
-1.6
00
11
1.6
-1.6at all -1-1
1.6
noat risk
Extremely
all no risk
high risk
Very low
Comprehension Very high
“How much risk do you at
believe
fluoridated
water
poses
to
human
ScienceScience
Comprehension
Very
liberal
Very
Conservative
Moderate
all
high
risk “How much risk do you
believe
medical x-rays poses to human
Strong Democrat
Strong Republican
Independent
health, safety, or prosperity?”
“How muchone
risk. do
warming
to human health, safety,
health,
safety, poses
or prosperity?”
Global
warming
An unpolluted
. . you believe global
3366
6699
12
9912
15
15
12
12
18
18
15
15
Science
Comprehension
Science
Comprehension
21
21
18
21
18
21
Very high
11
4 SE Fla. Counties
Cultural Cognition Project
SE Fla. evidence-based science communication initiative
Soute
Disentanglement principle:
“Don’t make reasoning, free people choose between
knowing what’s known & being who they are!”
What is to be done? You tell me!
Communicate normality
Southeast Florida science communication
Not “us vs. them”
just us—using what we know
Katie’s “Compact connector scouting report” form
Proselytizing the normality of climate science
Local businessman
Corp. exec.
Homeowner
climate scientist
Proselytizing the normality of climate science
Southeast Florida Climate Change Political Science:
Southeast Florida Climate Change Political Science:
It’s not the words but the conversation that count
Disentanglement principle:
“Don’t make reasoning, free people choose between
knowing what’s known & being who they are!”
What am I talking about? ...
 I. The #scicomm measurement problem
 II.
The “disentanglement principle”
 III. SE Fla. climate political science
IV. Krista “disbelieves” evolution—but loves Your
Inner Fish!*
*time permitting!
New data: shame & critical reasoning!
www.culturalcognition.net
Evidence-based Science Communication Filmmaking Initiative
The “missing audience hypothesis . . . ”
Engagement
The “missing audience hypothesis . . . ”
Science Curiosity
Engagement
The “missing audience hypothesis . . . ”
Science Curiosity
The “missing audience hypothesis . . . ”
Engagement
“Natural audience hypothesis”
?
“Missing audience hypothesis”
Science Curiosity
The “missing audience hypothesis . . . ”
a cool show . . .
The “missing audience hypothesis . . . ”
Engagement
Existing audience
“Missing audience”
Science Curiosity
The “missing audience hypothesis . . . ”
Engagement
Existing audience
“Missing audience”
Science Curiosity
State of the art “Science Curiosity”/“curiosity”
Test of Science Related Attitudes (TOSRA)





I am curious about the world in which we live
I find it boring to hear about new ideas
I would enjoy visiting a science museum at the weekend
I would like to be given a science book as a present
I get bored when watching science programs on TV
“Science Curiosity Index”
1. Measures: mixed strategy
2. Psychometric properties
3. Validation . . .
Sample self-report item
Sample self-report item
Sample (self-report) behavior
Sample (self-report) behavior
Performance measure
“Science Curiosity Index” (SCI)
(item response theory 2PL)
BSCIENCE
NISCIENCE
Not at all
1
.8
closely
.75
.7
very
closely
A little
but not closely
.9
.6
Prob.
probability of pro-curiosity response
1
.5
.4
.5
.3
.25
.2
.1
0
0
-3
-2
-1
0
1
science curiosity
science curiosity
2
3
-3
-2.5
-2
-1.5
-1
SCI
-.5
0
SCI
.5
1
1.5
2
2.5
3
“Science Curiosity Index”
1. Measures: mixed strategy
2. Psychometric properties
3. Validation . . .
The “missing audience hypothesis . . . ”
Engagement
Existing audience
“Missing audience”
Science Curiosity
The “missing audience hypothesis . . . ”
Engagement
Existing audience
“Missing audience”
Science Curiosity
YIF Clip: Origins of color vision
Self-report engagement
Behavioral engagement
The “missing audience hypothesis . . . ”
Engagement
Existing audience
“Missing audience”
Science Curiosity
The “missing audience hypothesis . . . ”
Engagement
Existing audience
“Missing audience”
Science Curiosity
Engagement: Evolution “believers” vs. “disbelievers”
Engagement: Evolution “believers” vs. “disbelievers”
believers
disbelievers
M = 0.17 (0.02)
-2.5
-2
-1.5
-1
-.5
0
.5
1
Science Curiosity Index
1.5
2
2.5
M = -0.27 (0.03)
-2.5
-2
-1.5
-1
-.5
0
.5
1
Science Curiosity Index
1.5
2
2.5
SCI: group “differences” . . .
Engagement: Evolution “believers” vs. “disbelievers”
believers
disbelievers
M = 0.17 (0.02)
-2.5
-2
-1.5
-1
-.5
0
.5
1
Science Curiosity Index
1.5
2
2.5
M = -0.27 (0.03)
-2.5
-2
-1.5
-1
-.5
0
.5
1
Science Curiosity Index
1.5
2
2.5
YIF Clip: Origins of color vision
Engagement: Evolution “believers” vs. “disbelievers”
1
engagement
.5
0
-.5
-1
-2
-1
0
Science Curiosity
1
2
Engagement: Evolution “believers” vs. “disbelievers”
1
Evolution believer
engagement
.5
0
-.5
-1
-2
-1
0
Science Curiosity
1
2
Engagement: Evolution “believers” vs. “disbelievers”
1
Evolution believer
engagement
.5
0
Evolution disbeliever
-.5
-1
-2
-1
0
Science Curiosity
1
2
Request full show: Evolution “believers” vs. “disbelievers”
100%
probability of requesting full show
probability
1
75%
.75
50%
.5
.25
25%
0
0%
1st-2
-1
16th
Colored bars reflect 0.95 level of confidence
0
1 84th
50th
Science
Interest
(percentile)
science
interest
Science
Curiosity
(percentie)
Science
Curiosity
2 99th
Request full show: Evolution “believers” vs. “disbelievers”
100%
probability of requesting full show
probability
1
75%
.75
Evolution believer
50%
.5
.25
25%
0
0%
1st-2
-1
16th
Colored bars reflect 0.95 level of confidence
0
1 84th
50th
Science
Interest
(percentile)
science
interest
Science
Curiosity
(percentie)
Science
Curiosity
2 99th
Request full show: Evolution “believers” vs. “disbelievers”
100%
probability of requesting full show
probability
1
75%
.75
Evolution believer
50%
.5
Evolution disbeliever
.25
25%
0
0%
1st-2
-1
16th
Colored bars reflect 0.95 level of confidence
0
1 84th
50th
Science
Interest
(percentile)
science
interest
Science
Curiosity
(percentie)
Science
Curiosity
2 99th
Request full show: Evolution “believers” vs. “disbelievers”
100%
probability of requesting full show
probability
1
75%
.75
Evolution believer
Evolution believer
50%
.5
Evolution disbeliever
.25
25%
Evolution disbeliever
00%
1st-2
-1
16th
050th
1 84th
Science
Interest
(percentile)
science
interest
Science
Curiosity
(percentie)
Science
Curiosity
Logistic regression. Colored bars reflect 0.95 level of confidence
2
99th
Was the information “believable”?
Was the information “believable”?
“It seemed like the documentary supplied
strong evidence of how humans acquired
color vision.”
“I found the information in the documentary
convincing.”
1
of agreeing
Probability
probability of agreeing
of agreeing
probability of
agreeing
Probability
1
.75
.5
.25
0
.75
.5
.25
0
-2
-1
0
Science Curiosity
1
Science Curiosity
2
-2
-1
0
Science Curiosity
1
Science Curiosity
2
Was the information “believable”?
“It seemed like the documentary supplied
strong evidence of how humans acquired
color vision.”
“I found the information in the documentary
convincing.”
Evolution believer
Evolution believer
1
of agreeing
Probability
probability of agreeing
of agreeing
probability of
agreeing
Probability
1
.75
.5
.25
0
.75
.5
.25
0
-2
-1
0
Science Curiosity
1
Science Curiosity
2
-2
-1
0
Science Curiosity
1
Science Curiosity
2
Was the information “believable”?
“It seemed like the documentary supplied
strong evidence of how humans acquired
color vision.”
“I found the information in the documentary
convincing.”
Evolution believer
Evolution believer
1
.75
Evolution disbeliever
.5
.25
0
of agreeing
Probability
probability of agreeing
of agreeing
probability of
agreeing
Probability
1
.75
Evolution disbeliever
.5
.25
0
-2
-1
0
Science Curiosity
1
Science Curiosity
2
-2
-1
0
Science Curiosity
1
Science Curiosity
2
“Belief in evolution”: who you are vs. what you know
Below avg.
religiosity
Above avg.
religiosity
Colored bars reflect 0.95 level of confidence
Source: Kahan, D.M. Climate-Science Communication and the Measurement Problem. Advances in Political Psychology 36, 1-43 (2015).
Was the information “believable”?
“It seemed like the documentary supplied
strong evidence of how humans acquired
color vision.”
“I found the information in the documentary
convincing.”
Evolution believer
Evolution believer
1
.75
Evolution disbeliever
.5
.25
0
of agreeing
Probability
probability of agreeing
of agreeing
probability of
agreeing
Probability
1
.75
Evolution disbeliever
.5
.25
0
-2
-1
0
Science Curiosity
1
Science Curiosity
2
-2
-1
0
Science Curiosity
1
Science Curiosity
2
Was the information “believable”?
“It seemed like the documentary supplied
strong evidence of how humans acquired
color vision.”
“I found the information in the documentary
convincing.”
Evolution believer
Evolution believer
1
.75
Evolution disbeliever
.5
.25
of agreeing
Probability
probability of agreeing
of agreeing
probability of
agreeing
Probability
1
.75
Evolution disbeliever
.5
.25
0
0
-2
-1
0
Science Curiosity
1
2
-2
-1
0
Science Curiosity
Science Curiosity
2
11
.75.75
probability of agreeing
probability ofofagreeing
agreeing
Probability
1
agreeing
of
probabilityof
agreeing
ofagreeing
Probability
Probability
1
Science Curiosity
.75
.5
.25
.5 .5
.25.25
00
0
-2
-1
0
OSI
1
Ordinary Science
Science Intelligence
Intelligence
Ordinary
2
-2 -2
-1 -1
00
OSI
OSI
11
Ordinary Science Intelligence
22
Was the information “believable”?
“It seemed like the documentary supplied
strong evidence of how humans acquired
color vision.”
“I found the information in the documentary
convincing.”
Evolution believer
Evolution believer
1
.75
Evolution disbeliever
.5
.25
of agreeing
Probability
probability of agreeing
of agreeing
probability of
agreeing
Probability
1
.75
Evolution disbeliever
.5
.25
0
0
-2
-1
0
Science Curiosity
1
2
-2
-1
0
Science Curiosity
Science Curiosity
Evolution believer
.75.75
probability of agreeing
.75
Evolution disbeliever
.5
2
Evolution believer
11
probability ofofagreeing
agreeing
Probability
agreeing
of
probabilityof
agreeing
ofagreeing
Probability
Probability
1
1
Science Curiosity
.25
Evolution disbeliever
.5 .5
.25.25
00
0
-2
-1
0
OSI
1
Ordinary Science
Science Intelligence
Intelligence
Ordinary
2
-2 -2
-1 -1
00
OSI
OSI
11
Ordinary Science Intelligence
22
Engagement with clip: SCI vs. OSI
1
SCI
.5
Engagement
Engagement
Science Curiosity Index
0
Ordinary Science Intellience
OSI
-.5
-1
1st
16th
50th
84th
Scale percentile
bars denote 0.95 CIs
99th
Cognitive dualism
Cognitive dualism
Two objects of reason: identity & knowledge
Request full show: Evolution “believers” vs. “disbelievers”
100%
probability of requesting full show
probability
1
Krista!
75%
.75
Evolution believer
Evolution believer
50%
.5
Evolution disbeliever
.25
25%
Evolution disbeliever
00%
1st-2
-1
16th
050th
1 84th
Science
Interest
(percentile)
science
interest
Science
Curiosity
(percentie)
Science
Curiosity
Logistic regression. Colored bars reflect 0.95 level of confidence
2
99th
Evolution-science communication measurement problem:
How to disentangle
What we know
Who we are
from
Evolution-science communication measurement problem:
How to disentangle
What we know
Who we are
from
Evolution-science communication measurement problem:
How to disentangle
What we know
Who we are
from
Prob. of requesting full documentary
probability
100%
75%
Evolution believer
50%
25%
Evolution disbeliever
0%
1st
16th
Colored bars reflect 0.95 level of confidence
50th
84th
ScienceCuriosity
Interest (percentile)
Science
(percentie)
99th
Evolution-science communication disentanglement:
measurement problem:
Satisfying the curiosity of all the citizens of the liberal republic of science!
Prob. of requesting full documentary
probability
100%
75%
Evolution believer
50%
25%
Evolution disbeliever
0%
1st
16th
Colored bars reflect 0.95 level of confidence
50th
84th
ScienceCuriosity
Interest (percentile)
Science
(percentie)
99th
Climate-science communication measurement problem:
How to disentangle
What we know
Who we are
from
Climate-science communication measurement problem:
How to disentangle
What we know
Who we are
from
Evolution-science communication disentanglement:
measurement problem:
Satisfying the curiosity of all the citizens of the liberal republic of science!
Prob. of requesting full documentary
probability
100%
75%
Evolution believer
50%
25%
Evolution disbeliever
0%
1st
16th
Colored bars reflect 0.95 level of confidence
50th
84th
ScienceCuriosity
Interest (percentile)
Science
(percentie)
99th