Can the Supreme Court Change Americans' Views? Comparing Experimental
and Observational Methods
Can the U.S. Supreme Court change Americans’ policy views? We surveyed a
representative sample of Americans before and after two major Supreme Court
decisions, embedding an experiment in the second survey. In contrast to much of
the prior literature, we find that the Supreme Court can lead public opinion, even on
hotly contested issues such as health care and immigration. Respondents who were
offered clear messages in support of the majority’s reasoning reacted much more
positively than respondents also reminded about the dissent. In addition, trust in
the Court, partisan affiliation, and ethnic background mediated responsiveness.
These findings invite us to revisit debates about the role of an unelected Court in a
democracy. They also contribute to debates on survey methodology, by highlighting
that both experimental and real-life treatments can yield similarly sized effects for
the population at large when events receive broad news-coverage and experimental
treatments resemble news headlines.
Katerina Linos, J.D./Ph.D.,
Assistant Professor
UC Berkeley Law School
klinos@law.berkeley.edu

Kim Twist
Ph.D. Candidate
Dept. of Political Science
UC Berkeley
kim.twist@berkeley.edu
Draft – Comments very welcome! We thank Kathy Abrams, Jack Citrin, Maria
Echaveste, Chris Edley, Chris Elmendorf, Dan Farber, Stavros Gadinis, Stephen
Goggin, Tom Goldstein, Taeku Lee, Gabriel Lenz, Ian Haney Lopez, Justin McCrary,
Victoria Plaut, Alison Post, Kevin Quinn, Bertrall Ross, David Sklansky, Fred Smith,
Laura Stoker, Cecillia Wang, Chuck Weisselberg, and John Yoo for their generous
advice. We are also very grateful to the Hellman Family Fund and the Warren
Institute for financial support, and to Annie Hilby and May Whitaker for excellent
research assistance. We also thank Samantha Luks and Ashley Grosse at
YouGov/Polymetrix for their help in fielding the surveys.
Introduction
Can the Supreme Court, a highly esteemed institution, change citizens’ views
so they support its decisions? Or do the Court’s choices mainly clarify and reinforce
citizens’ prior beliefs? As the Supreme Court takes on new and controversial
questions, including health care reform and gay marriage, the time has come to
revisit this powerful institution’s influence. Although the Court may fall in line with
public opinion (Epstein & Martin 2010; Casillas et al. 2011), when opinion is deeply
divided, the Court’s ability to move that opinion in one direction or the other is of
great importance.
Earlier studies on the impact of the Court reach conflicting conclusions.
Observational studies, which compare public opinion before and after actual Court
decisions, often find that the Court has no net impact on Americans’ views (Marshall
1987; Rosenberg 1991; Hanley 2008; Le & Citrin 2008; Luks & Salmone 2008; Gash
& Gonzalez 2008), although the Court can polarize opinion and push different
groups in opposite directions (Franklin & Kosaki 1989; Johnson & Martin 1998).
Experimental studies, in which some respondents are informed about a (usually
fictitious) Court decision from the experimenter, often find that the Court can lead
public opinion (Bartels & Mutz 2009; Hoekstra 1995; Clawson et al. 2003). This
difference could occur because actual Supreme Court decisions often come down
after controversies have received significant media exposure, after Americans have
thought about the issues and developed firm views.
Alternatively, these conflicting findings could result from severe
methodological limitations. Earlier observational studies are based on survey
questions fielded for other purposes. On issues such as flag burning, private
property takings, and even desegregation, we know what Americans thought
following critical Court decisions, but we have no baseline against which to compare
these views (Rosenberg 1991; Murakami 2008; Hanson 2008; Nadler et al. 2008).
Even when “before” and “after” data exists, the “after” data was often collected long
after the Court decision; events other than the Court decision could influence
opinion in this interval. For example, in Thomas Marshall’s pioneering research,
considered the most comprehensive observational study to date, the average time
lag between the Court decision and the post-decision survey exceeded two years
(Marshall 1987: 165; Egan & Citrin 2011: 6). Moreover, in some observational
studies, the wording of the “after” survey did not match either the wording of the
“before” survey or the exact issue before the Court (e.g., Johnson & Martin 1998:
307).
External validity limitations that often plague experimental studies could
also account for the difference between observational and experimental studies. For
example, some experiments on the influence of the Court survey college students,
who tend to be younger, better educated, and more liberal than Americans in
general (e.g. Unger 2008: 758; Hoekstra 1995). Because each of these factors is
linked to greater-than-average confidence in the Court (Stoutenborough and
Haider-Markel 2008: 38), Court decisions could influence students more than the
general population. More critically, Jason Barabas and Jennifer Jerit (2010) recently
cast significant doubts on the entire field of experimental survey research by
1
reporting that real world events had no effects on the population at large, whereas
similar experimental treatments yielded large shifts. Barabas and Jerit attributed
this difference to the fact that in survey experiments, everyone in the treatment
group is exposed to information, whereas in the real world, few people follow the
news (2010: 226). By studying events that received much greater coverage than
those examined by Barabas and Jerit, we further develop this research. We
investigate whether exposure is the critical difference between experimental and
observational studies, or whether other factors are at play, such as an
experimenter’s ability to make particular considerations salient to respondents
(Zaller 1992: 49).
Our research design combines observational and experimental techniques
and builds on the strengths of prior studies while also addressing these important
limitations. We surveyed a representative sample of Americans shortly before and
shortly after two major Supreme Court decisions. As the Court decisions focused on
health care and immigration, we asked respondents for their views on these two
issues and their confidence in the Court. Because the health care decision upheld a
law championed by Democrats, while the immigration decision upheld a law
championed by Republicans, we were able to examine how people responded both
to a liberal and to a conservative decision. We interviewed the same people at two
points in time, which greatly enhances the precision of our estimates (Mutz 2011).
In addition, if some people respond by moving towards the Court’s position, while
others respond by moving away from it, as prominent theories of backlash and
polarization suggest (Franklin & Kosaki 1989; Johnson & Martin 1998), reinterviewing the same people allows us to capture all of this movement.
To complement the before and after comparison, we embedded an
experiment in the second, post-decision survey to examine whether the type of
information individuals receive about Court decisions influences their responses.
Some Supreme Court decisions are unanimous and offer citizens a single frame
through which to understand an issue. Others contain powerful dissents, thus
offering a second, competing frame. Political communication studies suggest that
while one-sided messages can have large effects on public opinion, competing
messages can neutralize one another (Chong & Druckman 2007, 2013). To examine
whether dissenting arguments dilute the influence of the Court’s majority opinion,
we randomly assigned some respondents to be reminded of the main argument the
majority deployed to support its holding, while others were also offered the main
argument of the dissent. In addition, we monitored what information participants
received about the Court decisions through the media sources they normally use.
We found that Supreme Court rulings can change people’s attitudes and lead
public opinion, even on highly politicized and extensively debated issues such as
health care and immigration. Moreover, both experimental and real life treatments
can yield similarly sized effects when events receive broad news coverage and
experimental treatments resemble news headlines. In the case of the health care
decision, which received very broad media coverage, large portions of the
population correctly understood the holding and moved towards the court’s
position even without a reminder from us. Moreover, by repeating the holding of the
decision, and the main argument of the majority in the course of the experiment, we
2
did not further increase the size of this shift. This suggests a true information effect
at work, and a possible limit to experimental manipulations. We also found that
respondents exposed to one-sided messages in support of a decision were
significantly more likely to change their views in the direction of the Court than
respondents exposed to two-sided messages. Indeed, we found similarly sized
effects among respondents who we randomly assigned to the health care case
dissent and among respondents who were exposed to the dissent through Fox
News, which gave the dissent particular prominence.
We also found that the Court’s influence was moderated by several factors.
People with high trust in the Court before the decisions came down were more
likely to change their minds and follow the Court ruling after the decisions.
Independents shifted their views to follow the Court in both cases. Responses to the
highly politicized health care decision followed partisan lines. Democrats became
much more supportive of the individual mandate than they had been before the
Court decision, while Republicans mostly maintained their earlier opposition. We
also noted backlash in the context of the immigration decision. Hispanics, especially
those exposed to critical coverage of the decision through Spanish-language
television, became much more critical of the key immigration restriction that the
Supreme Court upheld. While our study is uniquely able to investigate all this
variation in responsiveness to the Court, we are especially confident in our results
because independent polling by several major organizations confirms that overall
support for the Affordable Care Act increased after the Supreme Court decision, and
that this increased support has persisted in subsequent months (Figure 4 below;
Campbell & Persily 2012).
In addition to their methodological implications, these findings also have
implications for important debates about the role of the judiciary in our democracy.
The Supreme Court enjoys unusually high levels of trust among the American public
(Gibson et al. 2003). In addition, Supreme Court justices control which cases they
review, when they issue decisions, what holding they reach, whether they issue
unanimous decisions or not, and how broadly or narrowly they write their opinions.
Some scholars suggest that the Court should use its broad discretion to avoid
engaging with politically controversial issues (Bickel 1986; Schauer 2006: 9). Others
argue instead that Courts should try lead public opinion to avoid a tyranny of the
majority (Ely 1980; Dorf 2010). The health care decision brought this controversy in
particularly sharp relief, as the Court chose to rule on the Obama administration’s
signature legislative achievement only months before a national presidential
election. Empirical data on whether the Court influences public opinion, whose
opinion it influences, and how dissents shape this effect are critical to developing
these major normative debates (Persily et al. 2008: 5).
The Court’s Influence on Opinion in Experimental and Real-Life Settings
We argue that we can best understand and measure the influence of a Court
decision by separating the process into three analytical components: (1) whether a
person hears about a decision and understands its main holding; (2) what she hears
about the decision, and more specifically, whether she hears one-sided or two-sided
3
arguments; and (3) what individual characteristics, such as partisanship and trust in
the Court, may mediate her responsiveness. Breaking the process into these
components also allows us to contribute to the literature on the external validity of
experimental treatments. Barabas and Jerit (2010) conclude that real-world
government announcements, unlike comparable experimental announcements,
often have no effect on the population at large, because news coverage is limited.
We test whether real life-announcements that are extensively covered in the media
yield large population wide-effects, and whether experimenters can still manipulate
these effects by making particular considerations salient to respondents.
First, for an event to influence a person’s opinion, she must hear about it (e.g.
Huber & Arceneaux 2007). Figure 1, below, shows that Supreme Court decisions
can generate very high levels of news-coverage. Following Barabas and Jerit (2010),
we searched the LexisNexis archive of major US newspapers for "health care" and
"immigration," and for both terms mentioned with either presidential candidate's
name, for the six months preceding the 2012 elections. Both issues were salient
throughout the campaign; our search averaged 88 and 29 daily stories for health
care and immigration, respectively.1 In contrast, the Barabas and Jerit study reports
just over fifty stories total for each of their two examples over a five-week period.2
Media coverage of both health care and immigration spiked after the Supreme Court
decisions to levels not seen before or since. No other actor or event, including the
presidential debates, placed as much attention on these issues as the Supreme Court
decisions.
--- FIGURE 1 HERE -Table 1, below, presents the volume of news coverage generated by the two
Supreme Court cases we studied, compared to other important Supreme Court cases
and to the two events studied by Barabas and Jerit. The tables report the number of
stories in the New York Times that referenced each case in the three weeks before
and after both the oral arguments and the decisions.3 Many Court cases received
orders of magnitude more coverage than the events studied by Barabas and Jerit.
Indeed, it is possible that coverage of Supreme Court cases might be increasing over
These totals include 29 different news sources.
Their examples focused on an announcement that the Medicare trust fund would
run out, and a change in the test that immigrants must take to become US citizens.
3 We found that more than seventy percent of coverage around Supreme Court cases
happens around the date of the decision, with the remainder occurring around oral
argument. To find coverage for the events studied by Barabas and Jerit, we
replicated the search using the terms “citizen” and “Medicare,” to cast the search net
as widely as possible, but found only two stories for each of these events in the
relevant time period. Though the New York Times tends to be more liberal than
other major publications (Ho & Quinn 2008), we believe that Table 1 gives us a
helpful first cut about the relative publicity different events received. This measure
correlates highly with another measure of case publicity, references in the Readers’
Index to Periodical Literature.
1
2
4
time, when taking into account the topic of the decisions. For example, Table 1
suggests that early abortion cases (Roe) received less coverage than later abortion
cases (Webster, Casey); early affirmative action cases (Bakke) received less coverage
than later affirmative action cases (Grutter and Gratz); early campaign finance cases
(Buckley) received less coverage than later campaign finance cases (Citizens’
United); and early gay rights cases (Bowers) received less coverage than later gay
rights cases (Lawrence).
Despite these variations, because Supreme Court decisions often receive
extensive coverage, we expect that many Americans should be able to correctly
identify the Court’s position. While an older literature suggests that Americans are
generally unaware of news developments (e.g. Delli Carpini & Keeter 1996), more
recent work (e.g. Gibson & Caldeira 2009b) suggests that people are more aware
than we had believed, and more aware of the Court, in particular. We follow the
advice of Gibson and Caldeira (2009b) in asking closed-ended questions to measure
knowledge, and expect to find levels of awareness comparable to their findings. We
expect the following hypothesis to hold true:
Hypothesis 1: Supreme Court decisions that receive widespread coverage
should influence Americans’ views and move aggregate public opinion, even
in the absence of experimental reminders.
----- TABLE 1 HERE -----Second, what people hear about a decision should matter. Sometimes,
Supreme Court decisions are presented as the final resolution of the conflict before
the Court; news media often present the only Court majority’s view. Often, however,
Court decisions have vigorous dissents, and these dissenting views also get
extensive news coverage (Ho & Quinn 2008). There are no prior studies that focus
on competing argumentation in the context of Court opinions.4 However, an
important prior experimental study suggests that merely mentioning that a decision
was unanimous or divided, without any argumentation, can change the impact of the
Court on opinion (Zink et al. 2009). Chong and Druckman (2007, 2013) report that
experiments with competing frames in other political contexts yield much smaller
shifts in opinion than experiments involving one-sided frames. As part of our
experiment, some respondents were reminded of the argument put forth of the
Court’s majority (Reminder 2), while others were also reminded of the argument of
the dissent (Reminder 3).5
Simon and Scurich (2011) examine argumentation in a different context. They
compare nuanced majority opinions (i.e. opinions acknowledging some counterarguments) to majority opinions alone, finding that nuanced majority opinions in
some cases increase the acceptability of Court decisions. In contrast, we expect and
find that dissents decrease the persuasive influence of the Court.
5 Other respondents received no reminder, while still others were only offered the
Court’s holding, with no arguments (Reminder 1).
4
5
We are also interested in comparing this experimental manipulation to
similar information some respondents likely received through real-world newssources. Most of the media coverage of the health care decision emphasized the
majority’s position. A typical headline was "Supreme Court upholds health care law"
(The Washington Post and The Boston Globe, June 28) or ""High Court backs health
care law" (The Chicago Tribune, June 28). However, Fox News heavily emphasized
the dissent, saying, “The majority is rewriting the statute and making it a tax.”6 As a
result of this news coverage, we might expect to see reduced responses to the
Court’s majority opinion not only among persons reminded about the dissent in the
course of the experiment, but also among Fox News viewers.
Hypothesis 2: Respondents who are exposed to a one-sided frame from the
Court’s majority should follow the Court more than respondents who are also
exposed to a competing frame from the dissent.
Third, individuals may respond in varied ways to the messages they receive.
Druckman (2001), among others, reports that, in other contexts, credible sources
are more persuasive. Because the Court enjoys citizens’ trust (Caldeira & Gibson
1992; Gibson & Caldeira 2009a; Hoekstra & Segal 1996), we would expect
individuals who expressed high levels of confidence in the Court before the
decisions to be particularly likely to change their opinions and follow the Court
decisions. We also investigate whether people who followed the Court’s lead on the
issues of health care and immigration were more likely to retain their trust in the
Court after these decisions. In addition, we expect partisanship to mediate
responses (Cobb & Kuklinski 1997; Lee & Schlesinger 2001; Lenz 2009). We expect
Independents to be especially likely to follow the Court, because they are less likely
to have formed firm opinions prior to the decision based on partisan cues. The
health care case was highly politicized: President Obama strongly praised the
Court’s decision on the individual mandate, while Governor Romney, the Republican
presidential nominee, sharply criticized it (Fox News Online 2012a, 2012b). In
contrast, statements following the immigration decision were more limited and
mixed in tone.7 We therefore expected large shifts among Democrats following the
“On the Record with Greta Van Susteren,” June 28. Similarly, Sean Hannity began
his Fox program that evening with the line, “Today will be remembered as the day
that the Supreme Court of the United States of America upheld the largest tax
increase in American history” (“Hannity,” June 28). In contrast, MSNBC hosts began
their evening shows by discussing “one of the great days in this country’s history,”
(“Hardball,” June 28) and the “historic news day” (“The Rachel Maddow Show,” June
28). The evening news programs on both CBS and NBC were measured in their
recounting of the day’s events; neither host offered a pro- or anti-mandate
viewpoint, and both shows presented videos and quotes from Republicans and
Democrats alike.
7 President Obama issued a statement in support of the Justices’ decision to strike
down the other aspects of the law, but spoke out about other concerns of racial
profiling that remain. In contrast, Mitt Romney made a more general statement
6
6
health care ruling, because they received positive endorsements both from the
Court and from party elites, while we expect more limited movement among
Republicans, who received both a positive message from the Court and a critical
message from party elites (including Fox News, which focused primarily on the
dissent’s arguments).
Hypothesis 3: Respondents who report high levels of trust in the Court
before the decisions, Independents, and people receiving pro-Court messages
from party elites should be particularly likely to follow the Court.
Finally, earlier studies of the Court highlight the possibility of backlash: that
subgroups of Americans become more hostile to Court’s position following a
decision. Backlash could result if a Court decision brings attention to an issue many
citizens had not carefully considered previously (Johnson & Martin 1998: 299). For
example, Franklin and Kosaki (1989: 762-3) report that, following Roe v. Wade,
Catholics’ opposition to some types of abortions solidified. Such opinion shifts are
especially likely among groups that the decision impacts directly (Hoekstra & Segal
1996) and among groups that receive critical coverage of the Court decision. Based
on the prior literature, we would anticipate backlash among Hispanics following the
Court’s immigration decision, which upheld an important Arizona restriction. The
Court decision placed Arizona’s immigration restrictions in the national spotlight
(see Figure 1 above). And in Spanish-language media, much of the coverage was
negative. For example, on the day of the decision, the headline of La Opinión, the
country’s largest Spanish-language daily, was “The Arizona Community is Full of
Doubts,” and both Univision and Telemundo aired thirty-minute, commercial-free
segments on the decision after it was handed down.8 We therefore expect to see
some negative responses to the Court decision among Hispanics, especially among
Hispanics who follow Spanish-language media.
Hypothesis 4: Citizens may move away from the Court’s position if they are
exposed to large amounts of critical news coverage about a Court decision.
Research Design
We fielded a two-wave survey shortly before and shortly after two important
Supreme Court decisions, embedding an experiment in the second wave.9 Extensive
news coverage and public opinion surveys often follow Court decisions, but the
scarcity of survey data fielded before key decisions has limited prior research. The
blaming the Obama administration for having not handled the immigration issue
earlier (Gabriel & Cooper 2012).
8 Fernández (2012). Similarly, coverage on Univision and Telemundo was very
critical. See, for example, Univision’s video archive, http://buscar.univision.com
/buscar /resultados.do?search_type=video&query=SB+1070.
9 YouGov/Polymetrix fielded the first wave in mid-May 2012, and the second wave
in late June 2012, starting on the day after each decision was announced.
7
Supreme Court announces the cases it will review early on. However, until a
decision comes down, there is significant uncertainty about every aspect of the
decision. For example, we do not know whether the Court will reach the merits of a
case or dispose of it on procedural grounds, when the decision will come down, or
which way the Court will rule. By consulting with several lawyers who closely
followed the Court and participated in related litigation, we were able to somewhat
limit this uncertainty and field questions about the major substantive provisions of
each challenged law before the Court decisions came down. Our opinion questions
examine the individual mandate provision of the Affordable Care Act and the “show
your papers” provision of Arizona’s immigration law.10
That said, the direction of the decisions came as a surprise to many
observers. On the eve of the widely anticipated health care case, futures markets
showed more than a seventy percent chance that the individual mandate would be
struck down, a prediction that was ultimately incorrect (Bingham 2012). A majority
of constitutional law professors surveyed before this decision also incorrectly
predicted its outcome (Drummond 2012). This reassures us that, though there was
significant publicity surrounding the health care case before the decision, even
experts who follow Court news closely received new information on the
constitutionality of the individual mandate on the day the Court decision was
announced.
We fielded the second wave of our survey days after each decision was
announced. Long lags between question waves are an important limitation of earlier
studies (Hoekstra 1995: 112). The short time frame between the waves, as well as
the proximity of the decisions to the second wave, gives us more confidence that the
Court’s rulings, rather than other events, drive our results. That said, we are also
interested in whether any short-term shifts persist, a point that we investigate
below by employing poll data from other sources.
We interviewed the same respondents before and after the decisions. Such
within-subjects designs are often preferable to between-subjects designs, because
they yield more precise estimates. However, a concern with these designs is that
respondents will catch on to the manipulation. Mutz (2011: 93-4) offers nationally
representative internet-based surveys, such as the one we used, as a useful solution
to this problem, largely because respondents receive multiple surveys, and are thus
unlikely to be strongly influenced by questions on any one such survey. Indeed,
given Bartels’ (1999) finding that panel effects rarely occur, the short length of the
Wave 1 survey, and the fact that it included only one question on the Supreme Court,
we believe it is unlikely that our Wave 1 questionnaire influenced respondents’
views six weeks later, when they responded to the Supreme Court decisions and
completed Wave 2.
At Time 1, survey respondents were randomly assigned to receive either the
immigration or the healthcare arm of the survey. Both sets of respondents were
asked Q1 (personal importance of immigration and health care), Q2 (news sources),
and Q4 (confidence in the court), while Q3 (opinion on immigration or health care)
varied in each survey arm. In drafting these questions, we followed earlier studies
10
See Appendix 1 for all question wording and response options.
8
where possible to enhance comparability. To ask about the personal importance of
immigration and health care, we followed the phrasing used by YouGov/Polymetrix
in past surveys. We drew on Egan and Citrin (2011) for our question on confidence
in the Court. Our media questions focused on television stations because the vast
majority of Americans still get most of their news in this way and because we expect
the most powerful and partisan stimuli to come through television news (Pew
Research Center 2012).
For the second wave of questions, fielded days after the health care and
immigration decisions were released, respondents were asked about the same topic
as in Wave 1 (health care or immigration). To disguise our purpose, and to avoid
reminding respondents that they may have answered similar questions six weeks
prior, Wave 2 began with a series of news headlines and asked respondents which, if
any, they had seen. If respondents ticked the news-headline corresponding to the
Supreme Court decision, they were then prompted to answer 1-3 questions about
the Court decisions. Following Gibson and Caldeira’s (2009b) recommendations, we
asked closed-ended rather than open-ended questions, as these more accurately
capture knowledge about the Court. These knowledge questions allow us to add
data to a debate about how much people know about Court decisions (Gibson &
Caldeira 2009b).
After completing these questions, some respondents received information
about the Court decision. Respondents were randomly assigned to one of four
groups while blocking on race, interest in the news, and viewership of Fox news in
particular. Blocking helped us to minimize error and reduce noise in our results (see
Mutz 2011: 95). For each experiment, forty percent of respondents were assigned
to the no-reminder group (n=400), and these people did not receive any information
about the content of the Court decision from the experiment. The remaining
respondents were divided into three equally sized groups and shown different
reminders about the decision (n=200 per group). Our reminders drew on the major
arguments developed by the Justices (see Appendix for the complete text). In the
health care decision, there was both a powerful majority opinion and a forceful
dissent, so we drew our arguments in favor and against the individual mandate from
these opinions. Reminder 1 offered the main holding of the decision, Reminder 2
offered this holding plus an argument favor of the decision from the Court majority,
while Reminder 3 also an argument against the decision from the Court’s dissent.
The immigration ruling, however, was unanimous on the key provision of the law.
We modified our design accordingly, and emphasized the fact that this was a
unanimous decision in Reminders 2 and 3 (but not in Reminder 1) of the
immigration experiment. Even though there was no dissenting opinion in the
immigration case, we added a consideration from the decision that pushed against
the majority’s conclusion in Reminder 3. Prior research on nuanced majority
opinions suggests that this additional consideration could work quite differently
from a dissent, and in some circumstances might increase, rather than reduce, the
influence of the majority’s position (Simon & Scurich 2011).
Results
9
We want to know whether people changed their minds about the individual
mandate and the Arizona immigration restrictions following the Court decisions.
Thus, our dependent variable takes on three values: a 1 for an increase of support in
each provision from Wave 1 to Wave 2, 0 for no change, and -1 for a decrease in
support. Because the Court upheld both laws, a 1 can also be interpreted as a shift in
the direction of the Court ruling, while a -1 can be interpreted as a shift away from
the Court ruling. This coding allows for comparability with earlier studies and
lessens the impact of outliers.11 Because this measure of change is based on the
attitudes of the very same individuals at two points in time, we have a great deal of
confidence in our descriptive statistics, which are very consistent with our
regression results.
------ TABLE 2 HERE -----Table 2 presents these first descriptive statistics. The first column shows
whether people shifted their views, broken down by issue (health care or
immigration) and by the group to which respondents were randomly assigned (no
reminder, holding only, holding + argument for, holding + arguments for and
against). For example, the first row of the first column indicates that following the
health care ruling, a net 9.5 percent of respondents who received no reminder
shifted toward the Court’s position, a shift different from zero at the 0.01 level.12
Because this group received no reminder from us, we would expect this effect to be
driven primarily by people who were aware of the Court decision, likely having
heard through the media. We thus broke down the sample depending on whether
respondents reported a high interest in following the news before the decisions
Egan and Citrin (2011), an experimental study that employs panel data, also used
this coding. We examined an alternative coding, which gives greater weight to
respondents who changed their minds radically. Because we asked respondents to
express their degree of support or opposition on a 5 point scale at two points in
time, we also constructed a nine-point scale ranging from -4 to +4, where -4
represented a move from strong support in Wave 1 to strong opposition in Wave 2,
and 4 represents the reverse. Results were generally similar across the two
measures. However, we worry that few outliers who moved from one extreme to the
other on our survey may have made a mistake or rushed through the survey. We
thus report results that give these outliers the same weight as all other respondents.
12 More specifically, 66.5 percent of our 382 respondents in this condition did not
change their views from Wave 1 to Wave 2, 12.0 percent moved away from the
Court’s position, and 21.5 percent moved towards the Court’s position. Our coding
includes people who changed their minds slightly – for example, people who moved
from opposing the individual mandate strongly to opposing the individual mandate
somewhat. If we look only to the change in support for the mandate, we see a 6
percent increase, from 29 percent support in Wave 1 to 35 percent in wave 2, for the
group that received no reminder from us.
11
10
came down.13 In both the health care and the immigration experiments, we see a
statistically significant shift towards the Court among people with a high interest in
the news, i.e., people likely to be aware of the decisions, and no such shift among
people less interested in the news. This observation, confirmed in subsequent
regression analyses, supports Hypothesis 1, that Supreme Court decisions that
receive widespread coverage influence American’s views and move aggregate public
opinion in the direction of the Court’s ruling, even in the absence of experimental
reminders.
That said, in the no-reminder groups, the shift following the health care
decision was much larger than the shift following the immigration decision. This is
likely in part because the health care case received more news coverage than the
Arizona decision. In addition, news coverage of the Arizona case was particularly
confusing, because the Court upheld the main provision of the Arizona immigration
law, while striking down other provisions.14 As a result, 46 percent of our
respondents correctly noted that the Court upheld the key Arizona immigration
restriction we studied, 14 percent incorrectly believed it had been struck down, and
another 40 percent did not know which way the Court had ruled. In comparison, 70
percent of respondents correctly identified that the Court upheld the individual
mandate, 6 percent incorrectly believed that the law had been struck down, while
another 25 percent did not know. 44 percent of all our respondents not only
correctly understood that the Court had upheld the law but were able to answer two
additional more detailed questions about the case.15 These findings are consistent
with Gibson and Caldeira’s findings that Americans are quite knowledgeable about
the Supreme Court (2009b).
We also have more detailed data on whether people correctly identified the
holding of each decision or not. The two measures are correlated (r=0.4) and results
are very similar using both methods. However, conservatives were heavily
overrepresented among people who correctly noted that the Court upheld Arizona’s
immigration restrictions. Thus, we focus on general interest in the news for the
cross-tabulations, and return to measure of knowledge of the decision in the
regression models, where we can better control for party affiliation and other
confounders.
14 Opportunity Agenda (2012) systematically reviews this confusing coverage; some
examples follow. In the hours following the announcement, the New York Times
website was one of just a few sites to suggest a favorable ruling, saying, "Justices
uphold key part of Arizona law" (June 25). In contrast, the CNN headline announced
a "blow to immigration law," with the Court telling Arizona "you went too far," and a
more reserved Fox News website said that the "Supreme Court reigns in Arizona on
immigration" (June 25). The vast majority of the coverage was somewhere in the
middle, with headlines like "Court rejects parts of Arizona immigration law" (The
Washington Post, June 25), "High Court repeals most of Ariz. immigration law" (The
Baltimore Sun, June 26), and a more-detailed "Supreme Court rejects most of
immigration law; The ruling is largely a victory for Obama, but the court leaves in
place a 'show me your papers' provision" (The Los Angeles Times, June 26).
15 See appendix for question text.
13
11
We next investigated whether we could manipulate these effects by making
particular considerations more salient to respondents. Specifically, we sought to
investigate whether respondents exposed to one-sided arguments were more likely
to follow the Court as compared to respondents exposed to two-sided arguments.
Chong and Druckman’s (2007, 2013) experimental research indicates that strong
counter-frames can significantly reduce the effect of positive endorsements, but that
weak counter-frames do not have this effect. Table 2 suggests, and subsequent
analyses confirm, that, in the health care experiment, reminding respondents of the
argument of the dissenting Justices (Reminder 3) entirely eliminated the influence
of the Court that we saw in all the other subgroups (No Reminder, Reminder 1, and
Reminder 2). We believe that the argument offered by the dissent – that
government should not be able to force Americans to buy a product – was a strong
frame because this was the main justification offered by mandate opponents.16
Thus, these patterns are consistent with Hypothesis 2 and with Chong and
Druckman’s earlier research.
In the immigration experiment, each of our reminders had a positive effect.
This is likely because, unlike in the health care experiment, many people did not
receive news of the Court decision or misunderstood the main holding due to the
confusing coverage. As we would expect, the coefficient on Reminder 2 (holding +
argument in favor) is larger than the coefficient of either of the two other reminders,
but the difference is not statistically significant. In the immigration experiment, the
Court came down unanimously on the law’s central provision, and thus we were not
able to test for the effect of a dissent. However, we were able to emphasize in
Reminders 2 and 3 that the decision was unanimous. This could drive the somewhat
larger effects of these treatments, relative to Reminder 1, which only provides the
Court’s upholding of the law.
Hypothesis 3 predicts that various individual characteristics mediate
responsiveness to the Court. Table A1 investigates these patterns for respondents
who received no reminder from us. Table A2 includes all respondents in the health
care experiment, and Table A3 includes all respondents in the immigration
experiment. For all regressions we report OLS coefficients, to facilitate the
interpretation of coefficients, especially because we include interaction effects (Ai
and Norton 2003). Results are very similar when we run ordered logit models, as
Table A4 shows.
An important mediator is trust in the Court; prior research suggests that
credible interlocutors are more persuasive. We classified people who reported a
high degree of confidence in the Court in Wave 1 as having high trust in the court,
and persons reporting only some degree of confidence or hardly any confidence as
having low trust. In both the immigration and health care experiments, people who
correctly understood the decision (labeled “aware”) and expressed high trust in the
More specifically, in January 2012, the Kaiser poll reported that 30 percent of
opponents of the mandate cited government overreach as their main concern.
Another 25 percent thought insurance would be too expense, a further 22 percent
complained about the fine, while five percent opposed the mandate because they
thought it was unconstitutional (Kaiser 2012c).
16
12
Court at Time 1 were most likely to change their opinions and follow the Court’s
lead at Time 2, controlling for a variety of other characteristics in the sample that
received no reminder from us (Table A1). In the full sample, we see a statistically
significant effect of trust on opinion change only in the immigration experiment.
In our regression analyses, we focus on trust in Time 1 only, as this measure
is independent of the Court’s eventual decisions. We saw large shifts in trust
following the health care case, because this decision was praised by Democrats and
criticized by Republicans. We only saw modest shifts in trust following the
immigration decision. An interesting question is whether people who changed their
opinions to follow the Court’s lead were more likely than others to maintain their
trust in the Court. Our preliminary investigations suggest that this is the case. Table
A5 suggests that groups that maintained or gained a high level of trust in the Court
were more likely to follow the Court’s lead than other groups. That said, this table
also suggests that these groups may be very different from one another in terms of
partisanship and other characteristics, so we set aside these findings for further
examination.17
Returning to our full sample, Table A2 analyzes the health care experiment.
Model 1 compares people who received each of the treatments (Reminders 1-3) to
people in the control (no reminder) group. Model 2 includes Reminder 3 (majority +
dissent arguments) alone. Models 3 and 4 include these reminders plus a variety of
other variables. Model 5 includes only those variables that are ever significant.
Model 6 only includes people who received reminders from us. Our main findings
are stable across specifications. Figure 2 below graphs how much each variable
contributes to opinion change, based on our final specification for all respondents
(Table A2, Model 5).
------ FIGURE 2 HERE -----Consistent with our earlier descriptive results, the regression models
confirm that people who received information about the holding of the decision
from the news (labeled “aware”) were more likely to change their minds and follow
the lead of the Court. Only one of our experimental manipulations – reminding
people of the argument of the dissent – had a significant (negative) impact on
opinion change. Further analysis of Reminder 3 suggests that the dissent was
powerful across ideologies; that is, in the group randomly assigned to this reminder,
Democrats, Republicans, and Independents all kept their original views.18 These
findings suggest that, while experimenters can manipulate real-life effects by
making certain counter-frames salient to respondents, simply repeating information
many respondents already knew, as in Reminders 1 and 2, does not shift attitudes.
This is consistent with the findings of Chong and Druckman (2007: 647), who also
find that repetition does not have an impact on attitude change, even for lowinformation subjects. Taken together, these findings suggest our result is driven by
processes of information transmission and is not an artifact of the experiment itself.
17
18
Additional analyses of shifts in trust are available upon request.
This analysis is available upon request.
13
Partisanship was an important mediator of responsiveness to the Court. Our
expectation was that Independents would often follow the Court’s lead because they
have more malleable opinions, and this expectation was confirmed. We also found
that Democrats responded more strongly to the Court’s health care decision than
did Republicans. This is not surprising, given that Democratic leaders hailed the
decision, while Republican leaders criticized it. We also found that people with very
low levels of education responded much less than did everyone else. This could
mean that people with very low levels of education process information from the
Court differently from more educated people, and do not connect the Court’s
decision about the constitutionality of the law to their policy views. It is also
possible to see this effect because our measures of knowledge of the decision and
interest in the news imperfectly captured all the variation in information about the
decision.
We found that media viewership was important. Fox News viewers kept their
original views, while non-Fox viewers moved towards the Court. Fox covered the
health care case extensively, with coverage both critical of the ruling and
emphasizing the dissent. This lack of change among Fox viewers is similar in
direction and magnitude to the experimental effect of Reminder 3: People we
exposed to the dissent, like people Fox exposed to the dissent, were not swayed by
the Court’s decision. We found that Democrats, Republicans and Independents
exposed to the dissent all kept their original views; similarly, more detailed analysis
shows that Fox viewers with diverse party affiliations all kept their original views.
That said, the Fox effect could also be selection, if, for example, people chose to
watch Fox because they had firm views on health care.
The immigration experiment also reflects findings that are consistent with
our theoretical expectations and with the health care experiment results. In Table
A3, Model 1 includes the randomly assigned reminders alone. Models 2, 3, 4, and 5
include these reminders plus a variety of other variables. Our main specification
includes only those variables that are ever significant; these results are presented
graphically in Figure 3 below.19 Model 7 includes only people who we exposed to
information about the case.
A somewhat unusual feature of our final specification of the immigration results
(Table A3, Model 6, graphed in Figure 3 and replicated using ordered logit in Table
A4) is that it includes a dummy variable labeled “aware & high trust” that takes on
the value of 1 when people were aware of the decision through the media and had
high trust in the Court, and 0 otherwise. A cursory glance might suggest that we are
inappropriately including an interaction term without its constituent parts
(Brambor et al. 2006). Models 4 and 5 in Table A3 present alternative specifications;
Model 4 separately includes awareness of the decision, and trust in the Court, which
could be thought of as the constituent parts of this interaction terms. Model 5 breaks
down people into four groups, and compares three of them (people with high
awareness and high trust, people with high awareness and low trust, and people
with low awareness and high trust) against the omitted category of low awareness
and low trust. Results across these specifications are highly consistent and suggest
that movement is concentrated only among the group that has both high awareness
19
14
------ FIGURE 3 HERE -----Each of our experimental reminders about the Court decision increased the
number of people who followed the Court’s lead, likely because coverage of the
decision through the media was confusing. Consistent with our expectations, we
found that people who both correctly identified the Court holding and had high
levels trust in the Court at the time of the first survey responded more than did
others. As in the health care experiment, people with very low education responded
less than did other people, likely for the same reasons outlined above.
As in the health care experiment, we found a particularly strong response
among Independents. This suggests that Independents have flexible views and
follow the Court after both conservative and liberal rulings. Democrats also
responded more strongly than Republicans, though we saw positive shifts in both
groups. The comparatively small positive shift among Republicans could result from
a ceiling effect – 85 percent of Republicans already supported the Arizona law’s key
provision even before the Court upheld it and 89 percent supported the law after
the decision.
We found that race and ethnicity mattered for the immigration experiment,
but not for health care. Hispanics moved in the opposite direction of the
immigration ruling from May to June. This effect is highly concentrated among
people who watch Spanish-language news on Univision or Telemundo. It is very
likely that, as with the Fox effect in the health care case, concentrated and critical
coverage of the immigration case on Univision and Telemundo led viewers to move
away from the Court’s ruling.20 This data supports Hypothesis 4: that citizens may
move away from the Court’s position if a decision generates much critical news
coverage to which they are exposed. In contrast, we did not find race to be a
significant predictor of Court-led changes in health care attitudes. Tesler (2012)
finds a racial gap between black and white Americans on Obama’s health care
legislation, which he attributes to positive associations between Obama and the
legislation among black Americans. The racial cues generated by Obama talking
about health care may not be generated when Supreme Court delivers the message
instead.
While our data only measure short-term opinion shifts, because there has
been consistent polling on the Affordable Care Act over time, we can also examine
whether the shift we observe are temporary or long lasting. Figure 4 below tracks
and high trust in the Court. Following Brambor et al. (2006: 68-69), we only omit
the constituent terms because we have strong theoretical reasons to do so (we do
not expect shifts among people who trust the Court but aren’t aware of its
decisions), and because we included the full specification with the constituent terms
and found their coefficients to be indistinguishable from 0.
20 A selection story is possible in the case of Spanish language television, but less
plausible than in the case of Fox. That is, we would have to posit that people who
had malleable views, and in particular people whose support for the Arizona law
was particularly tenuous, chose to watch Univision and Telemundo.
15
opinion on the Affordable Care Act since its enactment. While opinion on the law
was unstable immediately after enactment, it was stable for much of the two-year
period up until the Court decision. Monthly polls since the Court decision however
all note an uptick in support, suggesting that the Court’s impact on opinion was
durable. Our data suggest that both health care and immigration were very salient
to respondents, and prior work suggests that salient opinions are stable over time
(e.g. Krosnick 1988).
--- FIGURE 4 HERE ---Conclusions
We have demonstrated through two within-subjects studies that the US
Supreme Court has the ability to lead public opinion and produce large shifts in
attitudes. We observed these shifts even among respondents who only received
information through the media (with no experimental reminders from us), and even
though both health care and immigration had been hotly debated issues before the
decisions. Thus, we confirmed an important prediction of the theoretical literature
on the Court that many prior observational studies failed to confirm, and we expect
to see similar opinion movement following future high profile Court decisions, such
as the rulings on the California Proposition 8 and the federal Defense of Marriage
Act. We did this by improving on survey methodology: we asked questions about the
key issues facing the Court shortly before and shortly after its decisions, and
interviewed the same people twice to enhance the precision of our estimates.
Experimental manipulations involving the repetition of the majority’s holding and
main argument did not further increase the size of the opinion shift, suggesting a
true information effect at work. Other surveys confirm the aggregate opinion shifts
we observed following the health care case, and find that they persisted in
subsequent months.
Our study took the additional step of varying the way in which information
about the decisions was conveyed, bringing together literatures on public opinion
and framing with literatures on the impact of the Supreme Court. We found that the
degree of Court influence depends on whether respondents received a one-sided
argument in support of the Court majority, or also heard countervailing
considerations from the dissent. People we exposed to the strong dissent in the
health care case kept their original views, across the political spectrum. This
experimental effect was very similar in direction and size to the real-life effect we
observed among Fox News viewers, who also received extensive exposure to this
dissent.
We also found that trust in the Court mediated responsiveness: People who
had expressed high degrees of trust at time 1 were more likely to change their
opinions and follow the Court’s lead at time 2 in both the health care and the
immigration studies. Similarly, we expected Independents to have more flexible
opinions, and found they were especially likely to follow the Court after both
decisions. With the more liberal health care decision and the more conservative
immigration decision, we are able to show that the Court can move opinion in both
16
directions. However, we noted backlash among Hispanics after the immigration
decision, concentrated among viewers of Spanish-language television. The Supreme
Court’s review of Arizona’s immigration law likely exposed many of them to
information they did not previously have, information that was critical of the
Arizona law and of the Court’s decision to uphold a key provision.
These results lead us to revisit important theoretical debates about the role
of the Court in our democracy, especially as the Court has chosen to take on the
controversial question of gay marriage later this term. Our findings lead us to
believe, contrary to the prior observational literature, that the Court offers more
than a “hollow hope” for advocates of social change (Rosenberg 1991). The Court
has the potential to lead public opinion to protect vulnerable minorities, as some
theorists suggest it should (Ely 1980; Dorf 2010). That said, warnings from other
theorists that courts should try to avoid political controversies and write narrow
opinions also seem prescient (Bickel 1986; Schauer 2006), as we observe that
forceful dissents dramatically reduce the influence of the Court.
Finally, these results provide support for the external validity of survey
experiments and offer conditions under which we should expect results of survey
experiments to mirror responses to real-world events. Prior comparisons suggested
that, unlike experimental treatments, real-world announcements have no impact on
the population at large because most people do not follow the news (Barabas & Jerit
2010). Here, we show that, for events that receive extensive news coverage, such as
many Supreme Court decisions, real-life treatments can yield large effects for the
population at large. However, we suggest a further important consideration that
may lead to particularly large effects in experimental studies: Even when
respondents have already received ample information about an event through the
media, researchers can make particular considerations salient, thereby changing the
size of experimental effects.
17
Table 1: New York Times coverage in the six weeks surrounding events21
Case
Year
Florida v. HHS
Bush v. Gore
Brown v. Board
Webster v. RHS
Grutter v. Bollinger / Gratz v. Bollinger
Planned Parenthood v. Casey
Arizona v. US
Regents v. Bakke
Hamdan v. Rumsfeld
Citizens United
Texas v. Johnston
Lawrence v. Texas
PICS v. Seattle
Hamdi v. Rumsfeld
DC v. Heller
Boy Scouts v. Dale
Miliken v. Bradley
Boumediene v. Bush
McCreary v. ACLU / Van Orden v. Perry
Roe v. Wade
Wal-Mart v. Dukes
Kelo v. CNL
Bowers v. Hardwick
Miranda v. Arizona
Buckley v. Valeo
Loving v. Virginia
Citizenship test
Medicare warning
2012
2000
1954
1989
2003
1992
2012
1978
2006
2010
1989
2003
2007
2004
2008
2000
1974
2008
2005
1973
2011
2005
1986
1966
1976
1967
2008
2007
Case issue
Health care
2000 election
Desegregation
Abortion
Affirmative action
Abortion
Immigration
Affirmative action
Terrorism
Campaign finance
Flag burning
Gay rights
Busing
Terrorism
Second Amendment
Gay rights
Busing
Terrorism
First Amendment
Abortion
Unions
Fifth Amendment
Gay rights
Rights of the accused
Campaign finance
Race relations
NYT
stories
335
27322
138
104
77
68
62
59
50
48
44
42
34
33
32
31
26
25
24
23
22
22
19
11
11
7
2
2
For the Supreme Court cases, this includes coverage in the six weeks around the
oral arguments, as well as the six weeks around the decisions. Seventy-two percent
of articles come from the six weeks around the decisions. For comparison, the
coverage counts in the Times of the two Barabas and Jerit (2010) stories are shown
at the bottom.
22 Due to the nature of this case – with arguments coming only a day before the
decision – this total comes from the three weeks prior to the arguments and three
weeks after the ruling, rather than the twelve weeks for all other cases.
21
18
Table 2: Shifts in Opinion by Decision, Experimental Condition, and Level of Selfreported News Interest
Table 2a: % of Respondents Moving Toward Health Care Decision (net shift)
No reminder
Reminder 1 (Holding)
Reminder 2 (Holding +
argument for)
Reminder 3 (Holding +
arguments for + against)
Standard errors in parentheses
Full
sample
9.5***
(3.2)
11.4**
(4.5)
10.6**
(4.4)
0.6
(4.7)
High news
Low news
interest
interest
16.2***
2.6
(3.6)
(5.2)
10.5**
12.4
(4.9)
(7.5)
9.4*
11.7*
(5.4)
(6.7)
5.3
-4.7
(5.9)
(7.4)
*** p<0.01, ** p<0.05, * p<0.1
Table 2b: % of Respondents Moving Toward Immigration Decision (Net shift, with
standard errors in parentheses)
No reminder
Reminder 1 (Holding)
Reminder 2 (Holding +
argument for)
Reminder 3 (Holding +
arguments for + against)
Standard errors in parentheses
Full
sample
4.4
(3.0)
12.6***
(4.2)
17.8***
(4.9)
15.6***
(4.9)
High news
Low news
interest
interest
6.5*
2.4
(3.9)
(4.4)
15.6***
10.4*
(4.8)
(6.3)
16.9***
18.6**
(6.2)
(7.6)
16.0***
15.2*
(4.9)
(8.1)
*** p<0.01, ** p<0.05, * p<0.1
19
Figure 1: Coverage of Health Care and Immigration in Major US Newspapers
Source: Lexis-Nexis Archive of Major US Newspapers, which includes 29 sources.
20
Figure 2: Changes in Opinion on Health Care
* Reminder 3 is holding + argument for + argument against. Grey lines mark 90%
confidence intervals.
21
Figure 3: Changes in Opinion on Immigration
* Reminder 1 is holding only, Reminder 2 is holding + argument for, and Reminder 3
is holding + argument for + argument against. Grey lines mark 90% confidence
intervals.
22
Figure 4: Support for the Affordable Care Act since Enactment
Source: CBS/NYT polls. Question: “From what you've heard or read, do you approve
or disapprove of the health care law that was enacted in 2010?”
23
Appendix 1: Survey Wording
Time 1 Questions and Response Options
Immigration
Health Care
How important are the following issues to you?
Issues: Unemployment, Taxes, Health Care, War, Immigration (in random
order)
Q1
Personal
importance Response options: Unimportant, not very important, somewhat important,
very important.
Q2
News
sources
Q3
Opinion
In a typical week, from which of the following television news sources do you
get most of your information about politics and current events? If you do not
watch any of these, please mark “none of these.”23
Response options: NBC, ABC, CBS, Fox News, CNN, MSNBC, Univision,
Telemundo, none of these.
Do you support or oppose state laws Do you support or oppose federal
requiring police to investigate the
legislation requiring all Americans to
immigration status of any person
purchase health insurance?
during a traffic stop, based on
"reasonable suspicion" that the person
is in the country unlawfully?
Response options: strongly support,
Response options: strongly support,
somewhat support, somewhat oppose, somewhat support, somewhat oppose,
strongly oppose.
strongly oppose.
Q4
Trust
23
How much confidence do you yourself have in the United States Supreme
Court?
Response options: A great deal of confidence, only some confidence, hardly any
confidence
Respondents were allowed to select multiple options.
24
Time 2 questions and response options
Immigration
Health Care
Do you remember hearing about any of the following news stories in the last
few days? Please check all that apply. [options were randomly ordered]
Response options:
 Supreme Court rules on [federal health care legislation / Arizona
immigration law]
Q1
 Ann Curry leaves The Today Show
News
 Red Sox trade Youkilis
awareness
 Romney holds retreat in Utah for major donors
 Verdict reached in Sandusky trial
 Dick Cheney’s daughter marries partner
 New president elected in Egypt
 None of the above
Sometimes news stories contain lots of
details, many of which are hard to
remember and understand. We are
interested in what you may know of
the Supreme Court ruling on the
Arizona’s immigration legislation.
Please indicate what you remember.
Q2
Court
awareness24
Sometimes news stories contain lots of
details, many of which are hard to
remember and understand. We are
interested in what you may know of
the Supreme Court ruling on the
federal health care law. Please indicate
what you remember.
The Court held that it is...?
The Court held that it is...?
* Constitutional for U.S. States to
* Constitutional for the federal
require police to investigate the
government to require all Americans to
immigration status of anyone they stop purchase health insurance.
or arrest, based on "reasonable
suspicion" that the person is in the
country unlawfully.
* Not constitutional for U.S. States to * Not constitutional for the federal
require police to investigate the
government to require all Americans to
immigration status of anyone they stop purchase health insurance.
or arrest, based on "reasonable
suspicion" that the person is in the
country unlawfully.
* Don't remember the details of the
Court ruling.
* Don't remember the details of the
Court ruling.
These questions were only presented to respondents who indicated they had
heard about the ruling in Q1.
24
25
None
Do you remember if the Supreme
Court's ruling on the federal health
care law was a unanimous decision or
a divided decision? [options presented
in random order]
* Unanimous: all Justices agreed
* Divided: some Justices dissented
* Don't remember the details of the
Court ruling
None
Do you remember the legal basis for
the Supreme Court’s conclusion?
[options presented in random order]
* Congress has the power to regulate
commerce
* Congress has the power to raise taxes
* Congress has the power to make laws
that are necessary and proper
* Don't remember the details of the
Court ruling
Do you support or oppose federal
legislation requiring all Americans to
purchase health insurance?
Q2a
Case
awareness25
Q2b
Case
awareness26
Q3
Opinion
Do you support or oppose state laws
requiring police to investigate the
immigration status of any person
during a traffic stop, based on
"reasonable suspicion" that the person
is in the country unlawfully?
Response options: strongly support,
Response options: strongly support,
somewhat support, somewhat oppose, somewhat support, somewhat oppose,
strongly oppose.
strongly oppose.
Q4
Trust
How much confidence do you yourself have in the United States Supreme
Court?
Response options: A great deal of confidence, only some confidence, hardly any
confidence
This question was presented to all respondents who indicated they had heard
about the health care ruling (in Q1).
26 This question was presented only to respondents who correctly answered
“constitutional” about the health care ruling in Q2.
25
26
Reminder Wordings
Immigration
Health Care
The Supreme Court recently upheld a The Supreme Court recently upheld a
key provision of Arizona’s immigration key provision of the federal health care
legislation. It held that it is
legislation. It held that it is
constitutional for U.S. states to require constitutional for the federal
Reminder 1: police to investigate the immigration government to require all Americans to
Ruling only status of any person during a routine purchase health insurance.
traffic stop, based on “reasonable
suspicion” that the person is in the
country unlawfully.
The Supreme Court recently upheld a The Supreme Court recently upheld a
key provision of Arizona’s immigration key provision of the federal health care
legislation. It held that it is
legislation. It held that it is
constitutional for U.S. states to require constitutional for the federal
police to investigate the immigration government to require all Americans to
status of any person during a routine purchase health insurance.
traffic stop, based on “reasonable
Reminder 2: suspicion” that the person is in the
Many Justices on the Court accepted
Ruling + country unlawfully.
that the government can require
argument in
people to buy health insurance, so that
support Liberal and conservative Justices
people with health insurance don’t
unanimously agreed that state police have to subsidize people without it.
officers can conduct these immigration
status checks without interfering with
federal laws, and that state and federal
officials must often work together to
enforce immigration laws.
Reminder 3:
Ruling +
argument in
support +
argument
against
The Supreme Court recently upheld a The Supreme Court recently upheld a
key provision of Arizona’s immigration key provision of the federal health care
legislation. It held that it is
legislation. It held that it is
constitutional for U.S. states to require constitutional for the federal
police to investigate the immigration government to require all Americans to
status of any person during a routine purchase health insurance.
traffic stop, based on “reasonable
suspicion” that the person is in the
Many Justices on the Court accepted
country unlawfully.
that the government can require
people to buy health insurance, so that
Liberal and conservative Justices
people with health insurance don’t
unanimously agreed that state police have to subsidize people without it.
officers can conduct these immigration
status checks without interfering with However, other Justices argued that
federal laws, and that state and federal the federal government should not be
officials must often work together to able to force Americans to buy a
enforce immigration laws.
product they do not wish to buy.
27
However, the Justices disagreed about
the risks these immigration status
checks could involve. Some Justices
worried that state police might
unnecessarily prolong detentions and
violate people’s rights. They warned
state police officers to respect civil
rights and civil liberties or face further
challenges.
28
Appendix 2: Tables27
Table A1: Opinion Shifts Among Respondents Who Received No Reminders
Health Care Decision
All
Aware of
Not
Holding
Aware
Aware of
Holding
High trust
Aware & high
trust
Democrat
Independent
Age
College
Male
News
interest
Hispanic
Constant
-0.008
(0.093)
-0.091
(0.147)
0.311*
(0.173)
0.304***
(0.072)
0.238***
(0.074)
0.002
(0.002)
0.026
(0.066)
-0.033
(0.066)
0.121*
(0.073)
-0.229*
(0.128)
0.255***
(0.086)
-0.046
(0.178)
0.398***
(0.084)
0.247***
(0.075)
-0.000
(0.002)
-0.089
(0.072)
-0.091
(0.068)
0.204***
(0.078)
0.122
(0.127)
0.289*
(0.171)
0.005
(0.004)
0.299**
(0.136)
0.141
(0.168)
-0.104
(0.168)
-0.143
(0.146)
-0.469**
(0.222)
Observations
382
274
R-squared
0.082
0.128
Standard errors in parentheses
27
All
Immigration Decision
Aware of
Not
Holding
Aware
-0.072
(0.072)
-0.251**
(0.097)
0.392***
(0.131)
0.104
(0.073)
0.032
(0.065)
-0.001
(0.002)
0.041
(0.063)
0.074
(0.058)
0.008
(0.067)
-0.173*
(0.103)
0.043
(0.104)
0.148*
(0.089)
-0.238**
(0.099)
0.147
(0.106)
0.068
(0.085)
-0.003
(0.003)
0.086
(0.090)
0.105
(0.080)
-0.054
(0.097)
-0.150
(0.125)
0.015
(0.193)
0.045
(0.101)
-0.011
(0.099)
0.000
(0.002)
-0.004
(0.087)
0.018
(0.089)
0.092
(0.098)
-0.179
(0.156)
0.046
(0.128)
108
379
199
0.089
0.049
0.072
*** p<0.01, ** p<0.05, * p<0.1
180
0.047
See footnote 21 for an explanation of the interaction terms.
29
Table A2: Changes in Attitudes on Health Care
Reminder 1
(Holding only)
Reminder 2
(Holding + arg. for)
Reminder 3
(H + for + against)
Aware of holding
(1)
0.019
(0.055)
0.011
(0.054)
-0.088
(0.056)
(2)
-0.096*
(0.052)
High trust
Aware & high trust
Democrat
Independent
Black
Hispanic
Male
Age
No high
School
High school
Some college
2-yr college
4-yr college
Fox viewer
News
Interest
Constant
0.095***
(0.032)
Observations
1,000
R-squared
0.004
Standard errors in parentheses
0.103***
(0.022)
1,000
0.004
(3)
-0.012
(0.054)
0.017
(0.053)
-0.092*
(0.054)
0.105*
(0.055)
-0.069
(0.098)
0.099
(0.112)
0.220***
(0.051)
0.154***
(0.047)
0.117
(0.072)
0.055
(0.072)
0.033
(0.042)
0.001
(0.001)
-0.264**
(0.120)
-0.062
(0.069)
-0.107
(0.077)
0.010
(0.088)
-0.007
(0.073)
(4)
-0.010
(0.054)
0.018
(0.053)
-0.093*
(0.054)
0.114**
(0.053)
0.014
(0.051)
(5)
(6)
-0.091*
(0.050)
0.118**
(0.051)
-0.095*
(0.054)
0.176***
(0.066)
0.188***
0.212***
(0.054)
(0.049)
0.141***
0.147***
(0.048)
(0.048)
0.117
(0.072)
0.060
(0.072)
0.032
(0.043)
0.001
(0.001)
-0.251**
-0.196*
(0.121)
(0.103)
-0.052
(0.070)
-0.104
(0.076)
0.012
(0.088)
-0.004
(0.073)
-0.099**
-0.087**
(0.044)
(0.042)
0.032
0.043
(0.048)
(0.044)
-0.119
-0.120
-0.081
(0.103)
(0.103)
(0.051)
1,000
1,000
1,000
0.064
0.069
0.059
*** p<0.01, ** p<0.05, * p<0.1
0.175***
(0.062)
0.112*
(0.061)
-0.181
(0.123)
-0.124**
(0.054)
-0.013
(0.058)
-0.054
(0.068)
618
0.061
30
Table A3: Changes in Attitudes on Immigration
Reminder 1
(Holding only)
Reminder 2
(Holding + arg. for)
Reminder 3
(H + for + against)
Aware of holding
(1)
0.082
(0.051)
0.134**
(0.057)
0.112**
(0.057)
High trust
Aware & high trust
Democrat
Independent
Black
Hispanic
Male
Age
No high school
High school
Some college
2-yr college
4-yr college
(2)
0.082
(0.051)
0.134**
(0.055)
0.131**
(0.056)
0.024
(0.046)
-0.107
(0.080)
0.180*
(0.097)
0.092*
(0.048)
0.134***
(0.045)
-0.037
(0.071)
-0.156**
(0.074)
0.012
(0.041)
-0.003**
(0.001)
-0.244**
(0.099)
-0.088
(0.075)
-0.069
(0.080)
-0.075
(0.097)
-0.066
(0.081)
Spanish-lang. TV
viewer
High news interest
(3)
0.080
(0.051)
0.137**
(0.056)
0.130**
(0.056)
0.028
(0.047)
-0.101
(0.078)
0.183*
(0.096)
0.093*
(0.048)
0.134***
(0.045)
-0.040
(0.071)
-0.128*
(0.077)
0.019
(0.041)
-0.002*
(0.001)
-0.230**
(0.102)
-0.097
(0.076)
-0.069
(0.080)
-0.078
(0.097)
-0.064
(0.080)
(4)
0.078
(0.051)
0.133**
(0.056)
0.131**
(0.056)
0.035
(0.046)
-0.101
(0.078)
0.179*
(0.096)
0.093**
(0.047)
0.145***
(0.045)
(5)
0.078
(0.051)
0.133**
(0.056)
0.131**
(0.056)
(6)
0.080
(0.051)
0.132**
(0.056)
0.133**
(0.056)
(7)
-0.058
(0.064)
-0.003
(0.067)
0.113*
(0.059)
0.093**
(0.047)
0.145***
(0.045)
0.104**
(0.053)
0.090*
(0.047)
0.148***
(0.045)
0.133*
(0.078)
0.082
(0.061)
0.205***
(0.060)
-0.128
(0.078)
-0.128
(0.078)
-0.125
(0.077)
-0.097
(0.110)
-0.003**
(0.001)
-0.156**
(0.073)
-0.003**
(0.001)
-0.156**
(0.073)
-0.002*
(0.001)
-0.152**
(0.074)
-0.004**
(0.002)
-0.202**
(0.090)
-0.287**
(0.119)
-0.028
(0.044)
-0.279**
(0.118)
-0.010
(0.042)
-0.279**
(0.118)
-0.010
(0.042)
0.035
(0.046)
-0.293**
(0.126)
0.007
(0.040)
-0.314*
(0.178)
-0.027
(0.055)
0.100*
(0.059)
Aware & low trust
Didn’t hear & high
trust
Constant
0.044
(0.030)
Observations
1,000
R-squared
0.009
Standard errors in parentheses
0.186*
(0.111)
1,000
0.046
0.182
(0.111)
0.109
(0.078)
-0.101
(0.078)
0.109
(0.078)
1,000
1,000
1,000
0.050
0.047
0.047
*** p<0.01, ** p<0.05, * p<0.1
0.095
(0.076)
0.002
(0.110)
0.246**
(0.116)
1,000
0.044
621
0.053
31
Table A4: Ordered Logit Models of Our Final Specifications
VARIABLES
Health Care
Table A2
(5)
Reminder 1
Reminder 2
Reminder 3
Aware of holding
Aware & high trust
Democrat
Independent
No high school
Black
Hispanic
Fox viewer
-0.318*
(0.180)
0.488***
(0.181)
0.698***
(0.198)
0.501***
(0.174)
-0.745**
(0.373)
0.326
(0.257)
-0.283*
(0.147)
Spanish lang. TV viewer
Cut 1 Constant
Cut 2 Constant
-1.364***
(0.192)
1.894***
(0.193)
Immigration
Table A3
(6)
0.332*
(0.187)
0.522**
(0.208)
0.480**
(0.206)
0.344*
(0.189)
0.378**
(0.178)
0.561***
(0.164)
-0.512**
(0.256)
-0.419
(0.312)
-1.074**
(0.468)
-1.585***
(0.158)
1.766***
(0.165)
Observations
1,000
1,000
Standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1
32
Table A5: Support for Laws by Level of Trust at Time 1 and Time 2
Table A5a: Support for Immigration Laws (%) at Time 1 and Time 2
Trust at T1 & T2
High - high
High - low
Low - high
Low - low
Time 1
support (%)
71.9
71.7
69.5
50.5
Time 2
support (%)
79.7
69.4
72.2
55.2
Difference
(%)
7.8
-2.3
2.7
4.7
Table A5b: Support for Immigration Laws (%) at Time 1 and Time 2
Trust at T1 & T2
High - high
High - low
Low - high
Low - low
Time 1
support (%)
43
19.4
63.4
27.4
Time 2
support (%)
52.9
12.39
76.5
33.8
Difference
(%)
9.9
-7.01
13.1
6.4
33
References
Ai, Chunrong, and Edward C. Norton. (2003). “Interaction terms in logit and probit
models,” Economics Letters 80(1): 123-9.
Barabas, Jason, and Jennifer Jerit. (2010). “Are Survey Experiments Externally
Valid?” American Political Science Review 104(2): 226-42.
Bartels, Brandon L., and Diana C. Mutz. (2009). “Explaining Processes of Institutional
Opinion Leadership.” Journal of Politics 71(1): 249-61.
Bartels, Larry M. (1999). “Panel Effects in the American National Election Studies.”
Political Analysis 8(1): 1-20.
Bickel, Alexander M. (1986). The Least Dangerous Branch: The Supreme Court at the
Bar of Politics (2nd Ed.) New Haven: Yale University Press.
Bingham, Amy. (2012). “Health Care Games: The Odds Are Not In Obamacare's
Favor.” Available: http://abcnews.go.com/Politics/OTUS/health-care-gamesodds-obamacares-favor/story?id=16652833
Blake, Aaron. (2011). “Supreme Court inserts itself into 2012 election in a major
way.” The Washington Post. Available: http://www.washingtonpost.com
/blogs/the-fix/post/supreme-court-inserts-itself-into-2012-election-in-amajor-way/2011/12/12/gIQAXtg4pO_blog.html
Brambor, Thomas, William Roberts Clark, and Matt Golder. (2006). “Understanding
Interactions: Improving Empirical Analyses.” Political Analysis 14(1): 63-82.
Caldeira, Gregory A., and James L. Gibson. (1992). ‘‘The Etiology of Public Support
for the Supreme Court.’’ American Journal of Political Science 36(3): 635–64.
Campbell, Andrea, and Nathaniel Persily. (2012). “The Health Care Case in the Public
Mind: How the Supreme Court Shapes Opinion About Itself and the Laws It
Considers.” Working paper.
Casillas, Christopher J., Peter K. Enns, and Patrick C. Wohlfarth. (2011). “How Public
Opinion Constrains the U.S. Supreme Court.” American Journal of Political
Science 55(1): 74-88.
Chaiken, Shelly. (1980). “Heuristic Versus Systematic Information Processing and
the Use of Source Versus Message Cues in Persuasion.” Journal of Personality
& Social Psychology 39(5): 752-66.
Chong, Dennis, and James N. Druckman. (2013). “Counter-Framing Effects.” Journal
of Politics 75(1).
-----. (2007). “Framing Public Opinion in Competitive Democracies.” American
Political Science Review 101(4): 637-55.
Clawson, Rosalee A., Elizabeth R. Kegler, and Eric N. Waltenburg. (2003). ‘‘Supreme
Court Legitimacy and Group-Centric Forces: Black Support for Capital
Punishment and Affirmative Action.’’ Political Behavior 25 (4): 289–311.
Cobb, Michael D., and James H. Kuklinski. (1997). “Changing Minds: Political
Arguments and Political Persuasion.” American Journal of Political Science
41(1): 88-121.
Delli Carpini, Michael X., and Scott Keeter. (1996). What Americans Know About
Politics and Why It Matters. New Haven: Yale University Pres.
34
Dorf, Michael C. (2010). “The Majoritarian Difficulty and Theories of Constitutional
Decision Making.” University of Pennsylvania Journal of Constitutional Law 13:
283-304.
Druckman, James N. (2001). “On the limits of framing effects: Who can frame?”
Journal of Politics 63: 1041-66.
Drummond, Bob. (2012). “Obama Health Law Seen Valid, Scholars Expect Rejection.”
Available: http://www.businessweek.com/news/2012-06-22/law-expertssay-health-measure-legal-as-some-doubt-court-agrees#p1
Egan, Patrick J., and Jack Citrin. (2011). “The Limits of Judicial Persuasion and the
Fragility of Judicial Legitimacy.” Working paper.
Ely, John H. (1980). Democracy and Distrust: A Theory of Judicial Review. Cambridge,
MA: Harvard University Press.
Epstein, Lee, and Andrew D. Martin. (2010). “Does Public Opinion Influence the
Supreme Court? Possibly Yes (But We’re Not Sure Why).” University of
Pennsylvania Journal of Constitutional Law 13: 263-81.
Fernández, Valeria. (2012). “Comunidad de Arizona está llena de dudas.” Available:
http://www.laopinion.com/Comunidad_de_Arizona_esta_llena_de_dudas.
Fox News Online. (2012a). “Obama remarks on Supreme Court decision upholding
health care law.” Available http://www.foxnews.com/politics/2012/06/28
/transcript-obama-remarks-on-supreme-court-decision-upholding-hishealth-care.
-----. (2012b). “Romney remarks on high court ruling upholding Obama
health care law.” Available: http://www.foxnews.com/politics/2012/06/28
/transcript-romney-remarks-on-high-court-ruling-upholding-obama-healthcare-law.
Franklin, Charles H., and Liane C. Kosaki. (1989). “Republican Schoolmaster: The U.S.
Supreme Court, Public Opinion, and Abortion.” American Political Science
Review 83(3): 751-71.
Gabriel, Trip, and Helene Cooper. (2012). “On the Campaign Trail, Obama and
Romney React to the Justices’ Decision.” The New York Times. 26 June: A13.
Gash, Alison, and Angelo Gonzales. (2008). “School Prayer.” In Nathaniel Persily, Jack
Citrin, and Patrick J. Egan, eds., Public Opinion and Constitutional Controversy.
New York: Oxford University Press.
Gibson, James L., and Gregory A. Caldeira. (2009a) Citizens, Courts, and
Confirmations: Positivity Theory and the Judgments of the American People.
Princeton, NJ: Princeton Univ. Press.
-----. (2009b) “Knowing the Supreme Court: A Reconsideration of Public Ignorance
of the High Court.” Journal of Politics 71(2): 429-41.
Gibson, James L., Gregory A. Caldiera, and Lester Kenyatta Spence. (2003).
“Measuring Attitudes toward the United States Supreme Court.” American
Journal of Political Science 47(2): 354-67.
Hanley, John. (2008). “The Death Penalty.” In Nathaniel Persily, Jack Citrin, and
Patrick J. Egan, eds., Public Opinion and Constitutional Controversy. New York:
Oxford University Press.
Hanson, Peter. (2008). “Flag Burning.” In Nathaniel Persily, Jack Citrin, and
35
Patrick J. Egan, eds., Public Opinion and Constitutional Controversy. New York:
Oxford University Press.
Ho, Daniel E., and Kevin M. Quinn. (2008). "Measuring Explicit Political Positions of
Media." Quarterly Journal of Political Science 3(4): 353-77.
Hoekstra, Valerie J., and Jeffrey A. Segal. (1996). “The Shepherding of Local Public
Opinion: The Supreme Court and Lamb’s Chapel.” Journal of Politics 58(4):
1079-1102.
Hoekstra, Valerie J. (1995). “The Supreme Court and Opinion Change: An
Experimental Study of the Court’s Ability to Change Opinion.” American
Politics Research 23(1): 109-29.
Huber, Gregory A., and Kevin Arceneaux. (2007). “Identifying the Persuasive Effects
of Presidential Advertising.” American Journal of Political Science 51(4): 95777.
Johnson, Timothy R., and Andrew D. Martin. (1998). “The Public’s Conditional
Response to Supreme Court Decisions.” American Political Science Review
92(2): 299-309.
Kaiser Family Foundation (2012a). “Kaiser Health Tracking Poll: November 2012.”
Available: www.kff.org/kaiserpolls/ upload/8382-T.PDF
-----. (2012b) “Kaiser Health Tracking Poll: July 2012.” Available: www.kff.org/
kaiserpolls/upload/8339-T.PDF
-----. (2012c) “Kaiser Health Tracking Poll: March 2012.” Available: www.kff.org/
kaiserpolls/upload/8285-T.PDF
Krosnick, Jon A. (1988). “Attitude Importance and Attitude Change.” Journal of
Experimental Social Psychology 24: 240-55.
Le, Loan, and Jack Citrin. (2008). “Affirmative Action.” In Nathaniel Persily, Jack
Citrin, and Patrick J. Egan, eds., Public Opinion and Constitutional Controversy.
New York: Oxford University Press.
Lee, Taeku, and Mark Schlesinger. (2001). “Signaling in Context: Elite Influence and
the Dynamics of Public Support for Clinton’s Health Security Act.” Working
paper.
Lenz, Gabriel S. (2009). “Learning and Opinion Change, Not Priming: Reconsidering
the Priming Hypothesis.” American Journal of Political Science 53(4): 821-37.
Luks, Samantha, and Mike Salamone. (2008). “Abortion.” In Nathaniel Persily, Jack
Citrin, and Patrick J. Egan, eds., Public Opinion and Constitutional Controversy.
New York: Oxford University Press.
Marshall, Thomas R. (1987). “The Supreme Court As an Opinion Leader.” American
Politics Quarterly 15: 147-68.
Murakami, Michael. (2008). “Desegregation.” In Nathaniel Persily, Jack Citrin, and
Patrick J. Egan, eds., Public Opinion and Constitutional Controversy. New York:
Oxford University Press.
Mutz, Diana. (2011). Population-Based Survey Experiments. Princeton: Princeton
University Press.
Nadler, Janice, Shari Seidman Diamond, and Matthew Patton. (2008). “Government
Takings of Private Property.” In Nathaniel Persily, Jack Citrin, and Patrick J.
Egan, eds., Public Opinion and Constitutional Controversy. New York: Oxford
University Press.
36
Opportunity Agenda. (2012). “Media Analysis: News Coverage of the U.S.
Supreme Court Ruling on Arizona v. United States.” Available:
www.opportunityagenda.org/files/field_file/az_us_media_may_2012.pdf.
Pew Research Center. (2012). “State of the News Media 2012.” Available:
http://pewresearch.org/pubs/2222/news-media-network-television-cableaudioo-radio-digital-platforms-local-mobile-devices-tablets-smartphonesnative-american-community-newspapers.
Persily, Nathaniel, Jack Citrin, and Patrick Egan, eds. (2008). Public Opinion and
Constitutional Controversy. Oxford: Oxford University Press.
Rosenberg, Gerald. (1991). The Hollow Hope: Can Courts Bring About Social Change?
Chicago: University of Chicago Press.
Schauer, Frederick. (2006). “The Supreme Court, 2005 Term-Foreword: The Court's
Agenda- And the Nation's.” Harvard Law Review 120: 4.
Simon, Dan, and Nicholas Scurich. (2011). “Lay Judgments of Judicial Decision
Making.” Journal of Empirical Legal Studies 8(4): 709-27.
Stoutenborough, James W., and Donald P. Haider-Markel. (2008). “Public Confidence
in the U.S. Supreme Court: A New Look at the Impact of Court Decisions.”
Social Science Journal 45(1): 28-47.
Tesler, Michael. (2012). “The Spillover of Racialization into Health Care: How
President Obama Polarized Public Opinion by Racial Attitudes and Race.”
American Journal of Political Science 56(3): 690-704.
Unger, Michael A. (2008). “After the Supreme Word.” American Politics Research
36(50): 750-75.f
Zaller, John. (1992). The Nature and Origins of Mass Opinion. New York: Cambridge
University Press.
Zink, James R., James F. Spriggs, II, and John T. Scott. (2009). “Courting the Public:
The Influence of Decisions Attributes on Individuals’ Views of Court
Opinions.” Journal of Politics 71(3): 909-25.
37