Appendix B: Classroom Experiment
To assess whether Bush’s presidency was conditioning the presidential party
treatment effects, we re-fielded this particular portion of the experiment in the Winter of
2009, three months after Barack Obama had been elected president and after he assumed
office. In this iteration of the experiment, all the respondents were told that the president
was a Democrat. A randomly selected group were told that, “the president had a
reputation for hawkishness in international affairs; he had advocated the use of force
many times in the past when other leaders had not.” Other respondents read that, “the
president had a reputation for dovishness in international affairs; he had opposed the use
of force many times in the past when other leaders had not.” Six hundred and sixty four
surveys were administered by 83 UCLA students to others on campus between February
20 and February 26, 2009. Students administering surveys were enrolled in the Political
Science department’s course Diplomacy and War. Each student was given a packet of 8
randomly selected paper surveys to administer. Only one survey was given to each
survey respondent and students were instructed not to administer the survey to others in
the course. Once a survey was filled out by a respondent, the administering student
inputted the results into an excel file and these files were then merged to create a single
dataset. 97% of students completed the assignment resulting in 664 survey respondents.
Insert Table 1a about here.
While there were no statistically significant effects of the Hawk/Dove treatment
on presidential approval, the effect on disapproval at the War nodes is pronounced and
parallels the effect of the Republican treatment in the previous experiment. As Table 5
shows, Hawkishness increased disapproval by 19 points at the Unsuccessful War node
and by 9 points in the case of a Successful War. In this sample from a college campus,
there may be some bias in favor of doves, however, since dovishness lead to slightly
lower disapproval at the Stay Out node as well, although in this case the difference is far
from statistical significance. The effect of the Hawk-Dove treatment on disapproval is
statistically significant at the Unsuccessful War node and the differing effects of the
treatment at the Stay Out and Unsuccessful war nodes are significant using a one tailed
test and on the margin of significance using a two tailed test. The percent of participants
who responded they were dissatisfied with the president for being too willing to use
military force showed a similar pattern: it is much higher for hawks at the War nodes, and
the effect of hawkishness at the War nodes is statistically distinguishable from the effect
of hawkishness at the Peace node.1 Thus, these results of a Hawk-Dove treatment from a
survey conducted at a different time from Experiment II (after the last presidential
election) and where the president was described as a Democrat in all vignettes,
corroborate the results of Experiment II and show that they are unlikely to have been
driven largely by the specific events of the day.
1
Note that this survey did not include Concession node vignettes.
Table 1a. Explicit Hawk-Dove Reputation
LOSE
WAR
WIN
WAR
Hawkish (Dem.) Pres. 39%
(27 to 48)
15%
(7 to 22)
53%
(42 to 64)
Dovish (Dem.) Pres.
19%
(11 to 28)
52%
(41 to 63)
Hawkish (Dem.) Pres. 26%
(16 to 35)
7%
(2 to 13)
11%
(4 to 18)
Dovish (Dem.) Pres.
22%
(13 to 31)
22%
(13 to 31)
Hawkish (Dem.) Pres. 37%
(26 to 47)
78%
(69 to 87)
36%
(26 to 46)
Dovish (Dem.) Pres.
59%
(48 to 70)
27%
(17 to 36)
STAY OUT
Presidential Approval
36%
(26 to 46)
Unsure
31%
(21 to 41)
Presidential Disapproval
33%
(22 to 43)
% Respondents who believe, “the president is too willing to use the U.S. military”
Hawkish (Dem.) Pres. 9%
(2 to 15)
50%
(39 to 61)
49%
(39 to 60)
Dovish (Dem.) Pres.
14%
(7 to 22)
28%
(18 to 37)
37%
(27 to 48)
N
165
165
166
Note: Surveys administered by students of Diplomacy and War, Winter 2009. 95%
confidence intervals in parentheses.
Appendix C: Table 2a. Levels of Presidential Approval at Bargaining Outcomes by
President’s Party, Republican Voters Only
APPROVAL
President is a …
REPUBLICAN
DEMOCRAT
Difference
DISAPPROVAL
President is a …
REPUBLICAN
DEMOCRAT
Difference
Total Column N
STAY
OUT.
CONCEDE
LOSE
WAR
WIN
WAR
46%
32...
14...
94%
83...
11...
71%
67...
4...
90%
78...
12...
STAY
OUT.
CONCEDE
LOSE
WAR
WIN
WAR
46%
52 ..
-6 .
0%
4...
-4...
13%
13...
0...
0%
3...
-3...
61 ...
57...
60...
68...
The Peace column is the average of the “Stay Out” and “Concede” treatments. The War
column is the average of the two war treatments. The “Peace-War” column is the difference
between the Peace and War averages.
*** Significant difference at p<= .01 using two tailed difference of means test
** Significant difference at p<= .05 using two tailed difference of means test
* Significant difference at p<= .10 using two tailed difference of means test
Appendix C: Table 3a. Levels of Presidential Approval at Bargaining Outcomes by
President’s Party, Democratic Voters Only
APPROVAL
President is a …
REPUBLICAN
DEMOCRAT
Difference
DISAPPROVAL
President is a …
REPUBLICAN
DEMOCRAT
Difference
Total Column N
STAY
OUT.
CONCEDE
LOSE
WAR
WIN
WAR
17%
156...
-39***
44%
43...
1...
9%
31...
-22**.
31%
33...
-2...
STAY
OUT.
CONCEDE
LOSE
WAR
WIN
WAR
69%
34. ..
35***
33%
33...
0...
85%
54...
31***
56%
42...
14...
59 ...
59...
61...
62...
The Peace column is the average of the “Stay Out” and “Concede” treatments. The War
column is the average of the two war treatments. The “Peace-War” column is the difference
between the Peace and War averages.
*** Significant difference at p<= .01 using two tailed difference of means test
** Significant difference at p<= .05 using two tailed difference of means test
* Significant difference at p<= .10 using two tailed difference of means test
Appendix D: Table 4a. Presidential Approval in Peace and War by Behavior of
Opposition Party in Congress and Voter Party ID
Democratic Voters .
APPROVAL
STAY OUT.
WAR
Republican Opposition ...
SUPPORT
64%
28%
CRITICISM
50...
37...
Difference
14...
-9...
Democratic Opposition ...
SUPPORT
CRITICISM
Difference
24...
8...
16...
29...
12...
17 .
Democratic Voters .
DISAPPROVAL STAY OUT.
WAR
Republican Opposition ...
SUPPORT
36%
56%
CRITICISM
33...
37...
Difference
4...
19...
Republican Voters .
STAY OUT
WAR
Independent Voters .
STAY OUT
WAR.
36%
29...
7...
72%
73...
-1...
43%
20...
13...
39%
53...
-14...
33...
62...
-29 .
81...
80...
1...
29...
56...
-27...
46...
33...
13...
Republican Voters .
STAY OUT
WAR
Independent Voters .
STAY OUT
WAR.
43%
59...
-16...
8%
8...
0...
57%
70...
-13...
44%
12...
32...
Democratic Opposition ...
SUPPORT
CRITICISM
Difference
71...
67...
4...
61...
80...
-19 .
60...
31...
29 .
6...
7...
0...
21...
25...
-4...
50...
48...
2...
Total Column N
180
353
159
343
130
267
Appendix E. Conduct of Online Experiments
Our experiments in CCES and CCAP were completed on-line and fielded by the
survey research firm, YouGov/Polimetrix, Inc. located in Palo Alto, CA. For details on
CCES and CCAP see Vavreck and Rivers (2008) and Jackman and Vavreck (2009). Each
study begins with a large sample representative of its target population (the CCES target was
the general population and the CCAP target was registered voters). Each participating team
in the overall project gets 1,000 representative cases on which to field unique questions.
The CCES sample we use in these analyses is the product of collaboration between
four CCES teams, UCLA (Vavreck 2006a), the University of Wisconsin (Goldstein 2006),
Stanford University (Jackman 2006), and the research firm Polimetrix (Vavreck 2006b). Our
experiment ran on the post-election wave of the survey only, which was fielded on November
8th, 2006. In 2008, our CCAP experiment ran on a subset of respondents in the October wave
of CCAP, which was fielded on October 23rd, 2008. These cases were not affiliated with any
team study. The CCES experiment has nearly 8,000 cases and the CCAP experiment has
3,000 cases. The experiment analyzed here was fielded to roughly a third of the respondents
in each study.
Appendix F: Table 5a. Weighted Marginals across Six Recent Election Studies
Census
(2004
ACS)
Census
NAES
NES
CCES
2004
2004
2006
Target
Population
(2008 CPS
Reg.
Voters
Only)
CCAP 2008
V2.0
Registere
d
Voters
Registere
d Voters
NES
2008
(FTF)
NES
2008
(Internet
Panel)
General
(but
these
results
for RV
only)
General
(but
these
results
for RV
only)
General
Genera
l
Genera
l
General
HS
16.1
29.5
7.3
25.6
14.4
31.6
3.9
42.3
8.0
28.6
10.4
27.3
11.8
31.4
8.7
31.3
Some College
Through 4
Year Degree
44.5
51.7
44.1
45.1
52.5
52.6
47.2
50.2
9.9
14.3
9.9
8.7
10.9
9.8
9.6
9.7
12.3
18.1
20.5
19.3
13.6
16.1
7.7
16.1
20.3
21.6
16.3
17.9
11.5
17.2
19.4
19.2
15.8
16.9
8.6
17
25
30.8
10.6
8.1
10.3
15.5
17.4
20.6
16.9
19.2
11.1
15.9
19.7
21.9
14.6
16.8
11
18.2
15.9
21
15.3
17.6
10.5
14.2
19.8
21.3
15.9
18.3
67.3
12.8
14.1
5.8
78.2
8
5
8.8
70.9
14.8
6.3
8
71.1
10.9
12.2
5.8
76.0
11.7
7.9
4.4
76.5
11.6
7.4
4.5
74.3
12
9.1
4.6
75.6
11.5
7.7
5.2
48.9
51.1
44.7
55.3
49.5
50.5
48.2
51.8
46.6
53.4
46.9
53.1
45.2
54.8
47.2
52.8
Education
Less than HS
Advanced
Degree
Age
18-24
25-34
35-44
45-54
55-64
65+
Race
White (NonHispanic)
Black
Hispanic
Other
Gender
Male
Female
General Population
Registered Voters
Only
Note:
Marginals are weighted
principal investigators
using
weights
provided
by
survey
vendor
or