The Impact of Online Platforms on Public Opinion
Regarding the European Union
Laura Sudulich, ULB
Leonardo Baccini, LSE
Introduction
An information environment is vital to the capacity of citizens to learn about politics, their
ability to link political preferences to parties and policies and to assess the performances of
institutions and political actors. In most instances, people gather politically relevant
information via the mass media rather than through direct interaction with political elites.
The role of the mass media is therefore key to the correct functioning of society (Castells
2000; Delli Carpini and Keeter 1996). There is no shortage of evidence showing that the
mass media exert a crucial influence on public opinion formation (Zaller (1992); for a
review of major contributions, see Bennett and Iyengar (2008)).
The mass media are particularly important when considered in relation to the formation of
opinions, the forming of political awareness and voting behaviour in the context of the
European Union, since citizens typically do not experience any direct contact with
European institutions. Thus, citizens necessarily rely on the mass media when (in)forming
their opinions about the European Union (hereafter EU). Additionally, a widespread
ignorance of EU politics and policies among members of the European public is notorious
and persistent. Over a third of Europeans are unable to name any EU institution; the
percentage of citizens claiming to know little or nothing at all about ‘the people who run
the various EU institutions and the leaders of the EU’ is a striking 73%. A similarly high
number of subjects (74%) report knowing little or nothing at all about ‘The allocation of
roles played by the various institutions (who does what?)’ (Eurobarometer 77.4). The
1
impact of radio, television and newspapers has been extensively debated in relation to
electoral behaviour and attitude formation towards the EU (De Vreese 2003; De Vreese
and Boomgaarden 2006; Schuck and De Vreese 2006), while the influence of the Internet_
is currently under-explored.1 This is in spite of a significant average penetration rate of
75.5% across member states, with several countries – The Netherlands, Sweden, Denmark
– registering over 90% coverage (World Bank 2013).2 Moreover, a real word trend indicates
that the Internet is rapidly becoming a fundamental source of information about European
matters. Between November 2011 and December 2012, the percentage of citizens
reporting that they had gathered information on the European Parliament on the World
Wide Web went up by 10%, from 33% to 43% (Eurobarometer 78.2).
This paper takes a first step to addressing whether consuming online information about the
EU is responsible for variation in public opinion regarding it and whether this process
bypasses or encompasses political knowledge. Specifically, we explore how different type
of sources affect public opinion. In so doing, we bridge studies of public opinion and
media effects to contributions addressing online source credibility (for a review see
Metzger 2007). The latter are mostly based on experimental evidence and do not directly
address political learning or opinions, while the former tends to rely on observational data
and often disregards the nuances of digital environments.
We use data from Eurobarometer 76.3 (November 2011) posing a large battery of
questions on media use for political information consumption based on a representative
Europe-wide sample of individuals. Importantly, this survey contains a unique array of
items on online political news consumption habits. In addition, we provide an in-depth
analysis of the effects of a particular type of platform - the Irish Referendum Commission
website - on electoral behaviour in the context of the Fiscal Compact Referendum held in
An exception is De Wilde et al. 2013, who content-analyse expressions of Euroscepticism in online media
across 12 member states in the 2009 European Parliament campaign. However, their analysis does not deal
with the consequences of online-based information and communication for public opinion.
2 http://data.worldbank.org/indicator/IT.NET.USER.P2
1
2
the Republic of Ireland on May 31st 2012.3 Making use of an original survey, we test the
micro-foundations of our argument by isolating the effects of online information gathered
specifically with regard to a single policy i.e. the Fiscal Compact. Specifically, we explore
the impact of online newsgathering on the knowledge of such a policy, and on actual
voting behaviour.
We find that consuming information online has an effect on the opinions of those who
consult the websites of traditional media and institutions; this is mediated by an increase in
political knowledge. Websites that are by default a mixture of vetted information and nonfact-checked information prove unable to do this. In fact, Those who gather their
information about the EU mostly via blogs, social networking sites or video sharing
platforms experience no significant effects on opinions and neither they appear to gather
knowledge. For those who actually experience an effect by visiting institutional websites or
digital versions of news media an increase in knowledge turn into positive views about the
EU. Along similar lines, the analysis on the Irish Fiscal Compact Referendum confirms
that visiting an institutional website - namely the Referendum Commission’s site –
increases self-perceived knowledge of the specific policy at stake and the likelihood of
voting in favour of the Fiscal Compact Treaty. In the mare magnum of data available on the
web, credible fact-checked sources stand out in providing viable knowledge that serves to
orient citizens’ opinions.
The article proceeds as follows: in the next section we outline the relationship between
online-based information, political knowledge and public opinion on which we base our
working hypotheses. We then discuss our main empirical strategy hinging on causal
mediation analysis and describe the data. In the following section, we outline and discuss
the results of our analysis. We then run a further test to account for the endogeneity
intrinsically associated with the relationship between media usage and public opinion. Next,
3
Called either Fiscal Compact of Fiscal Stability Referendum.
3
we explore the micro-foundations of our argument analysing the Irish Fiscal Compact
Referendum survey. We conclude by examining the implications of our findings.
Information, knowledge and the World Wide Web
Information is the data that allow individuals to acquire (politically) relevant knowledge and
to form or redefine their beliefs. It can be gathered from direct experience – attending a
candidate debate, correspondence with an MP – or by being exposed to reports. These
reports can be based on the experience of people in one’s social network – e.g. a friend
telling of his/her experience in dealing with local government – or can be provided by
mass media. Acquiring information through direct experience of EU institutions and
policies can be substantially ruled out. The same applies to information obtained via
personal networks as ‘very few citizens have first- or even second-hand contact with
Community affairs in Brussels’ (Dalton and Duval 1986, 186).
Political knowledge is the state of awareness of facts that matter to orient people’s opinions
and choices. The acquisition of knowledge depends upon the availability of information,
and so does the redefinition of what Bartels (1993) calls the ‘fund of knowledge’. Newly
acquired information allows individuals to update their pre-existing knowledge – for those
who had some – and provides new knowledge to those who had none before. The mass
media are therefore key to educating the public by making the information available and
accessible (Holtz-Bacha and Norris 2001). Information acquired via the mass media can
then translate into in-depth knowledge, superficial acquaintance, or even no knowledge as a
result of it being, unclear, noisy or redundant.
A substantial corpus of studies has unveiled differences among traditional media in their
capacity to translate information into knowledge that subsequently affects political
evaluations (for a review, see (Norris and Sanders 2003)). While several studies support the
idea of the superiority of print (Robinson and Davis 1990; Robinson et al. 1986), others
4
cast doubts on it (Graber 2001; Mondak 1995), with no ultimate consensus on which
medium carries the highest learning potential for the public. Despite the different
conclusions reached by these studies, the literature on traditional media relies on a widely
shared – if implicit – assumption: the information provided is both factual and relevant.
Media publishers and regulatory authorities act as gatekeepers to what can be broadcast or
printed, and media editors, through the process of indexing (Bennett 1990), select what is
relevant for the public to know (Flanagin and Metzger, 2000).
The multiplicity and
segmentation of information producers on the World Wide Web (Bimber 2003) cast some
doubt on the applicability of this assumption to the Internet. One of McLuhan’s (1964)
insights in formulating the well-known the medium is the message theory was that “the content
of any medium is always another medium”.
The Internet essentially contains all the other mass media that preceded it: the printed
press, TV, and radio. All major news outlets now have an online version. Additionally,
there are many other online spaces that deliver politically relevant information: the official
websites of institutions, social media sites – including social networking sites like Facebook
and Twitter as well as sharing platforms like wikis and YouTube – and blogs. Online
information can therefore take the most diverse shapes, depending on the source and
hosting platform: from credible news producers to unverifiable information posted by
individuals simply voicing their own opinions. In the words of Patterson, “the internet is at
once a gold mine of solid content and a hellhole of misinformation” (2013: 79). These
structural differences with respect to traditional media are likely to affect the opinions and
behaviours of those who have integrated online news consumption into their media usage
habits. The nature of online-based information may also affect the process through which
individuals translate information into political knowledge.
Lupia and McCubbins (1999, 25) point out a key element of the relationship between
information and knowledge: ‘although you cannot have knowledge without having
5
information, you can have information without having knowledge’. Information that does
not provide knowledge is either redundant or is simply noise. While absorbing redundant
information is a possibility when gathering information via any media, the Internet
maximizes the potential for the amplification of noise by creating an environment where
noise and signal are often indistinguishable (Ayres 1999).
Online spaces vary dramatically in the heterogeneity of their content and in the extent to
which they host unverified versus verified facts. For instance, the website of a major
newspaper is likely to publish content that is verified and has been provided by accredited
sources. At the other end of the spectrum, platforms like blogs and forums are, by
definition, aggregators of comments and opinions, often seeking to advocate more than to
report (Scott 2007). Experimental research has shown that websites credibility is best
understood as a perceptual attribute, but working with survey data rules out respondents’
self-assessment (for a review of online perceived credibility research see: Flanagin and
Matzger 2007). Therefore, we opt for a classification of sources that both builds on what
shown by these studies and accounts for the fact that platforms that host information
created by a more heterogeneous set of producers and of mixed nature are more likely to
carry greater amounts of noise. In Table 1 below we classify hosting platforms in terms of
the type of information they provide and the information producers.
[Table 1 about here]
We thus empirically analyse separately the effects of platforms that have a strong potential
for the amplification of noise from those that are most likely to maximize fact-checked
information, in order to gather a more finely-grained understanding of how the new media
affect individuals’ opinions. Formally, we test whether the following hypotheses hold true:
H1: Online platforms providing vetted information increase knowledge of the EU and significantly
6
impact upon citizens’ opinions about it.
Conversely:
H2: Online platforms providing miscellaneous information do not increase knowledge of the EU
and fail exercising an impact upon citizens’ opinions about it.
Given the large amount of variation across platforms, we expect the mechanism to work
through the knowledge channel only for those platforms that are more likely to facilitate
users in distinguishing the news from the noise. Knowledge is a well-established predictor
of attitudes; therefore increases in knowledge should filter the effects of information
acquisition, particularly so in the wake of what we know about widespread ignorance of
European citizens on EU level matters and institutions. Media effects are more
consequential where previous opinions are weak, and the second order theory (Reif and
Schmitt 1980) clearly indicates that attitudes towards the EU are weaker than attitudes
towards national government, making media effects particularly well placed to impact upon
citizens’ opinions.
Empirical strategy
To test our hypotheses we perform three empirical analyses. First, as a result of our
theoretical framework, we implement a causal mediation analysis model. This allows us to
distinguish the direct effect of online information consumption from its mediated effect
through increases in knowledge of EU matters. Causal mediation analysis is the core test of
our argument. Second, we implement instrumental variables to mitigate concerns on
reverse causality. We note that while instrumental variables are a useful tool to address the
endogeneity problem, they are substantively different from causal median analysis. In
particular, instrumental variables do not allow us to disentangle the mediated effect of
online information for its direct effect on attitudes toward the EU. Third, we validate the
7
results of our cross-country estimates with original micro-level data on the Irish Fiscal
Compact Referendum.
Let’s start describing the research design of the causal median analysis. Causal mediation
analysis allows exploration of the role of an intermediate variable that lies along the causal
paths between the treatment and the dependent variable (Hicks and Tingley 2011; Imai,
Keele, and Yamamoto 2010). In our case, knowledge about the EU is the mediator that lies
along the causal path between information gathered online and attitudes towards the EU.
The path mode is illustrated in Figure 1.4
[Figure 1 about here]
To carry out the causal mediation analysis, we rely on the STATA 13 ‘mediation’ package
developed by Hicks and Tingley (2011). The model for both the mediator variable and the
outcome variable is an ordinary least squares (OLS) regression. We include country fixed
effects to account for cross-country heterogeneity and mitigate omitted variable problems.
We use robust standard errors and we run 1000 simulations for the quasi-Bayesian
approximation of parameter uncertainty. We are unable to run an ordered logit (or probit)
– which would be more appropriate given the outcome variables – as this is not supported
by the ‘mediation’ package. However, we run simple ordered probit models – reported in
the online appendix (Table A5) - the results of which are very similar to those from the
OLS regressions. This makes us confident of the reliability of the estimations when we
implement the causal mediation analysis.
Data
We use data from Eurobarometer 76.3, which contains a large battery of items on media
use for political information on EU matters. We exclude observations from countries that
The literature on causal mediation analysis is large and fast-growing. For pioneering statistical studies, see
Baron and Kenny (1986).
4
8
were not members of the EU at the time of the data collection (November 2011).
Outcome variable
The outcome variable captures respondents’ attitudes towards the EU. The question on
which we build our dependent variable was posed as follows:
“In general, does the EU conjure up for you a very positive, fairly positive, neutral, fairly negative or
very negative image?”5
The resulting variable ranges from 0 to 4 where 0 indicates ‘very negative’ and 4 ‘very
positive’.
Treatment variables
We firstly set apart those who gathered most of their information on political matters
about the EU on the word wide web.6
To capture different types of information sources online we rely on the following dummy
variables:

A variable that scores 1 if respondents exclusively use ‘institutional and official websites
(government websites, etc.)’ to gather information on European matters, and 0 otherwise;

A variable that scores 1 if respondents exclusively use ‘information websites (websites from
newspapers, news magazines, etc.)’ to gather information on European matters, and 0
otherwise. We label this variable traditional media websites;

A variable that scores 1 if respondents exclusively use ‘online social networks’ to gather
information on European matters, and 0 otherwise;

A variable that scores 1 if respondents exclusively use ‘blogs’ to gather information on
European matters, and 0 otherwise;
5
This is one of the customary questions to measure attitudes towards the EU, as it is included in the
Eurobarometer Standard Trend Questions.
6 Verbatim of QD4: “Where do you get most of your news on national political matters” (a) Television (b)
The press (c) Radio (d) The internet (e) Other (f) You do not look for news on national political matters.
9

A variable that scores 1 if respondents exclusively use ‘video hosting websites’ to gather
information on European matters, and 0 otherwise.
These variables are generated on the basis of responses to the following question:
“On the Internet, which of the following websites do you use to get news on European political
matters? “ (a) Institutional and official websites (governmental websites, etc.) (b)
Information websites (websites from newspapers, news magazines, etc.) (c)
Online Social Networks (d) Blogs (e) Video hosting websites (f) Other.
Figure 2 shows the distribution of the country averages of our five treatments. The takehome message from this figure is that at the time of the fieldwork the websites of
traditional media outlets were, by far, the most visited platforms in every country. The
second most preferred platforms for gathering information on EU matters were institutional
and official websites, while the use of blogs, online social networks and video hosting websites was very
limited across the entire sample. As one may reasonably expect the percentage of those
who gather most of their information about the EU on Online Social Networks and
Blogs is quite small (below 5% for both types) and minuscule with regard to Video hosting
websites.
[Figure 2 about here]
Given this structural characteristic of the data, we aggregate the five dummies in two
treatments. On the one hand, we create a variable that scores 1 if respondents exclusively
use ‘institutional and official websites’ or ‘traditional media websites’. We label this fact-checked
websites. On the other hand, we create a variable that scores 1 if respondents exclusively use
‘online social networks’ or ‘blogs’ or ‘video hosting websites’. We label this miscellaneous websites. By
implementing this additional measure we minimize concerns about the results being driven
by few observations.
10
Mediator
For the mediating variable we use an index of objective political knowledge about the EU
compiled by the Eurobarometer team and available in the dataset. It ranges from 0 to 2
(bad, average, good) and it is based on the answer to the following standard question:
“For each of the following statements about the EU could you please tell me whether you think it is true or
false:
 The EU currently consists of 27 Member States
 The members of the European Parliament are directly elected by the citizens of each Member
State
 Switzerland is a member of the EU”
This index is built on items that capture the knowledge of what Barabas et al (2014) classify
as static general facts. In the Appendix (Table A6 and A7) we re-run our main models
using a variable that captures knowledge of static policy facts. This variable is based on a
question asking respondents whether the “EU budget is jointly determined by the European
Parliament and the Member states”. Finally, as detailed below, in the Fiscal Compact
Referendum Section we address the knowledge of surveillance policy facts.
In line with the estimation strategy implemented for the outcome variable we run an OLS
with country fixed effects and robust standard errors. We show the consistency of the
results from a simple ordered probit in the online Appendix (Table A5).
Control Variables
We control for the socio-economic status of the respondents (gender, age, working status,
education) and other factors that are likely to affect both media usage choice and political
opinions regarding the EU. Namely, we include controls for how frequently respondents
11
discuss both national and European political matters with friends and relatives (frequently,
occasionally, never). We then control for consumption of traditional media by including
controls for information consumption about EU matters via TV, printed press and radio.
Moreover, we include standard indicators (Gabel and Palmer 1995; Van Klingeren,
Boomgaarden, and De Vreese 2013) of subjective perception of the national and European
economies, as citizens tend to display unfavourable attitudes towards the EU if they believe
the economy – both national and European – is performing poorly (Kritzinger 2003) and a
measure of the respondents’ attitudes to immigrants. Previous studies indicate that
attitudes to immigration are strongly correlated with support for the EU. Those who are
negatively disposed towards out-groups tend to oppose the free circulation of citizens and
the opening of national borders within Europe (De Vreese and Boomgaarden 2005). We
report descriptive statistics about all the variables in use in the Appendix (Table A1), where
we also provide a link to the EB 76.3 codebook and the Fiscal Compact Referendum’s
questionnaire.
Empirical results
Table 2 below reports the results in relation to each dummy capturing a specific type of
platform. Specifically, in each model one of the types of platform is included as the
treatment, allowing us to estimate the average causal mediation effect (ACME) and direct
effect for each type of platform. Since the results of the mediation equation are the same
across the five models, we report them only once. For each and every model, we include
dummies for traditional media usage in gathering information on the EU. The results are
clear-cut: only certain types of online loci – those ensuring higher levels of fact-checked
information and produced by a more homogeneous array of producers – have a mediated
positive effect on knowledge as well as a direct positive effect on attitudes to the EU.
Specifically, traditional media websites and especially institutional and official websites are positive
12
and statistically significant (Models 2-3), whereas blogs, online social networks, and video hosting
websites (Models 4-6) have neither a statistically significant impact through knowledge
(mediated effect) nor a direct effect on attitudes to the EU (Models 4-6).7 Importantly,
ACME is positive and statistically significant only in models 2 and 3. The average causal
mediation effect accounts respectively for 12 and 11 percent of the total effect.
[Table 2 about here]
Note that the coefficients of our dummies are the same across models 2-6. We effectively
estimate the same model five times and include each dummy as a treatment to obtain the
ACME for each type of platform. The fact that all the coefficients are the same confirms
the robustness of our results.
Our findings remain similar to those presented above when using the more aggregate
treatment to capture fact-checked websites in Table 3. The results are positive and statistically
significant with regard to fact-checked websites, whereas the coefficient for miscellaneous websites
is never statistically significant. As expected, ACME is positive and statistically significant
in model 8, but not in model 9. Here, the average causal mediation effect accounts for 12
percent of the total effect.8
[Table 3 about here]
Only fact-checked websites increase the respondents’ knowledge of the EU and, in turn,
knowledge has a positive mediated effect on attitudes to it. Fact-checked websites have a
positive direct effect on attitudes to the EU, in addition to the effects through the
In the next section we show that these findings are not affected by reverse causality and/or selection into
the treatment.
8
Furthermore, the control variables have the expected coefficients, adding plausibility to our analysis (see the
online Appendix). Importantly, the results of our dummies capturing specific types of platforms have an
effect in addition to the role played by traditional media.
7
13
knowledge channel. On the contrary, websites providing a large space for less reliable
information and personal opinions affect neither the knowledge about the EU nor attitudes
to it. In sum, we find strong support for both our two hypotheses.
Finally, we perform a series of robustness checks to further validate our findings. First, we
use the alternative variable for political knowledge described above (Tables A6-A7).
Second, we re-run our main analysis using multilevel regressions to account for crosscountry heterogeneity (Table A8). Third, to balance out differences between those who
used the five aforementioned website types and those who did not use the internet as a
medium for information gathering on EU matters, we use entropy balancing (Hainmueller
2012).9 Specifically, we balance our entire set of covariates with respect to our treatments,
which captures those respondents who go online as well as the type of website they visit.10
Differences in the means between the treated and control group vanish for all the
covariates after implementing entropy balancing (pre- and post-matching descriptive
statistics for the relevant variables are reported in the online Appendix). Note that
balancing covariates with respect to the treatment is similar to controlling for such
confounding factors in a standard multivariate regression without imposing parametric
functional form or distributional assumptions. We then run all our parametric models using
the weights obtained from the entropy balance estimation as well as the entire set of
control variables to account for any residual differences between the treated and control
groups (Table A9).11 An advantage of entropy balancing over a matching technique is that
the former does not drop unmatched observations (Hainmueller 2012: 2). For all these
tests, which are reported in the Appendix, the results are very similar to the ones reported
above, adding plausibility to our empirical analysis.
We use the STATA 13 package ‘ebalance’ (Hainmueller, 2012).
A description of the covariates is provided in the following section.
11 Residual differences might come from the variance and skewness of the covariates.
9
10
14
Addressing endogeneity
The previous section has shown that consuming information online – if only via certain
platforms – has a positive effect on both knowledge about the EU and attitudes to it. One
may object that the selection into our treatments is not random, as it would be plausible to
argue that respondents who are knowledgeable about the EU are also more likely to go
online in search of information about it. Similarly, it could be claimed that respondents
who have a positive (negative) attitude to the EU are more likely to visit pro-EU (anti-EU)
websites, which in turn reinforce their priors. Although experimental research (Garrett,
Carnahan, and Lynch 2013) has shown how in online environments browsing for attitude
consonant spaces does not prevent browsing for attitude dissonant information, concerns
about self-selection are sound. To mitigate them, we implement an instrumental variable
approach for those treatments that were statistically significant in the mediation analysis.
Our instrument is a dummy that scores one if respondents have access to the Internet
either at home or at work (labelled access). This variable thus takes value 0 when
respondents said they had access neither at work nor at home. The idea behind this
identification strategy is that without access to the Internet at home or at work respondents
are less likely to go online to gather information about the EU. Indeed, the correlation
between online access and information and official websites is respectively 0.68 (p-value=0.00) and
0.54 (p-value=0.00). Clearly, respondents without online access at home or at work can still
browse the web through their smartphones/tablets, and this explains why we do not find a
perfect correlation between our instrument and our treatments.12
The problem of relying on access is that it could be correlated with our dependent variables,
i.e. knowledge about and attitudes to the EU. For instance, if respondents with Internet
access are geographically clustered, our instrument could be correlated with socioWe note that if respondents who have access to internet through smartphones/tablets are more likely to
visit fact-checking website and to have a positive attitude towards the EU, our IV estimates would be
downward biased.
12
15
economic variables (i.e. urban or rural place of residence), which consequently affect our
outcome variables. Since we control for socio-economic status, the exclusion restriction
should still hold. Moreover, acknowledging that our instrument is not randomly assigned,
we balance our entire set of covariates with respect to access in order to create a counterfactual (i.e. respondents without online access), which is as close as possible as to our
treated units, i.e. respondents with online access. In the online Appendix (Table A4) we
show the distribution of the covariates before and after balancing. Then we use the weights
obtained from entropy balancing in our instrumental variable estimations.
Armed with this identification strategy, we re-run our main models with both knowledge
about the EU and attitudes to it as dependent variables. We begin with knowledge about
the EU. The results are reported in Tables 4a and 4b and are in line with those shown in
the previous section. Specifically, institutional and official websites and traditional media websites
have a positive and statistically significant impact on knowledge about the EU.
Furthermore, all the diagnostics show that our instrumental approach is robust. In
particular, the orthogonality conditions are valid (see the Anderson-Rubin Wald test) and
our instrument is sufficiently strong (see the Cragg-Donald Wald F statistics). Moreover,
Table 4b shows the results for attitudes to the EU and indicates that our previous results
are all validated. Indeed, institutional and official websites and traditional media websites always
strengthen positive opinions about the EU. Even in this case, all the diagnostics confirm
the validity of our identification strategy.
[Tables 4a and 4b about here]
The Case of the Irish Fiscal Compact Referendum
The analysis presented above has the advantage of relying on a large comparative dataset,
but it has two shortcomings. First, the Eurobarometer survey does not contain any
16
question accounting for political knowledge of surveillance facts. If we are able to explore
knowledge of both general facts (in the main text) and of specific facts (in the Appendix),
we are still limited in our capacity to cover the temporal dimension of knowledge (Barabas
et al 2014). Second, we have no control over the content actually accessed online by
respondents. This latter issue is a factor of both the ephemerality of the medium under
scrutiny here and the limits of survey based data, particularly when secondary.
In order to account for these limitations, we integrate our analysis with a quantitative case
study - vote on the Fiscal Compact Treaty Referendum in Ireland - that is well placed to
address these concerns by (a) using a measure of policy specific surveillance fact and (b)
pinning down the effect of one particular official website whose content is known. In
doing so, we use data from an original phone survey performed by the polling company
RED C (details are provided below).
A constitutional provision in the Republic of Ireland requires a popular vote on matters
that entail amending the constitution, as ratification of EU treaties does. The issue at stake
on May the 31st 2012 was whether to ratify the European Fiscal Compact Treaty, one of
the key policy measures the European Union adopted in response to the sovereign debt
crisis. We test our main hypothesis about the effects of online information consumption by
examining the effects of a specific website on the knowledge of a single policy issue, to
then assess its impact on voting decision. This behavioural complement to our study of
attitudes implies that voting in favour of the Fiscal Compact Treaty carries an implicit
positive evaluation of the EU and its policy measures.
The Irish Referendum Commission dedicated a website to the Fiscal Compact Treaty
(www.http://www.refcom.ie/) featuring a youtube video inviting citizens to vote,
providing links to additional audio-visual material with extensive information on the Fiscal
Compact pact, and a link to the guide designed by the Referendum Commission itself to
17
provide factual and balanced information to voters.13 The Fiscal Compact Treaty website
offered relevant facts and guidance on the referendum procedures.
In line with the previous analysis, we have two outcome variables. The first dependent
variable captures respondents’ knowledge of the treaty. Specifically, respondents were
asked to self-assess their knowledge of the Treaty: How would you describe your knowledge of the
Fiscal Treaty?
(a)
I don’t know anything about it at all
(b)
I don’t know very much about it
(c)
I know quite a lot about it.
(d)
I’m extremely well informed about it.
The second dependent variable is a dummy that scores one if respondents vote ‘yes’ to the
EU Fiscal Compact Treaty.14 The question on the vote was put very straightforwardly as
follows: “Did you vote YES in favour or NO against in the Fiscal Stability Treaty referendum?”. The
treatment in this case scores one if a respondent used the official Referendum Commission
Fiscal Stability Treaty website regularly during the campaign.
Given the low number of observations, we are unable to implement causal mediation
analysis.15 Therefore, we run two separate estimations with our two outcome variables: (1)
an ordered probit for the knowledge of the Fiscal compact; (2) a probit for the probability
of voting “yes”. In line with our previous analysis we include entropy balancing weights.16
With these data it is particularly important to balance all the confounding factors with
respect to respondents who visit only the EU Commission website, which is our main
explanatory variable. Indeed, solo EU Commission website visitors are very different with
Now: http://www.refcom.ie/en/Past-Referendums/Fiscal-Stability-Treaty/
We drop those respondents who refused to answer this question. As a result, we are left with 949
observations.
15 The number of simulations has to always be lower than the number of observations in causal mediation
analysis. A low number of simulations creates problems with the convergence of the estimation and with the
reliability of the ACME.
16 Table A10 in the Appendix provides details of all the variables in use before and after matching. Moreover,
we report the verbatim of the questions from the survey included in the analysis.
13
14
18
respect to variables influencing voting behavior from respondents who also visit other
websites to gather information on the Fiscal Compact.
Furthermore, the control variables we include in the model – on top of standard socioeconomics – account for three sets of predictors of vote choice at EU level referendums.
The first type relates to the so-called ‘attitudes schools’, which contends that citizens
underlying attitudes towards the EU explain their vote choice at EU level referenda
(Franklin, Van der Eijk, and Marsh 1995). The second set accounts for evaluations of the
national level government, capturing the ‘second order school’ contention that just like at
EP elections citizens vote in EU referendums by expressing an evaluation of their national
government (Reif and Schmitt 1980). Finally, we account for the so-called ‘utilitarian’
school (Gabel 1998) by controlling for the economic consequences citizens attribute to the
referendum’s outcome.
(Table 5 about here)
Results from Table 5 indicate that visiting the Referendum Commission website increases
self-perceived knowledge about the treaty. Moreover, it shows that those who regularly
visited the website of the Referendum Commission are more likely to vote ‘yes’ than those
who did not visit this website, ceteris paribus. Therefore, online information gathering on this
fact-checked official website leads to a higher likelihood of accepting the Fiscal Compact.
Knowledge about the treaty has a positive effect on the probability of voting “yes”, though
it is (barely) not statistically significant (p-value=0.17). This result might be explained by
the relative low number of observations and by the limited variation of the outcome
variable, i.e. large percentage of “yes” vote. This finding confirms what previously seen
when using the Europe wide sample and increases the validity of the pattern identified
there. Moreover, it shows that institutional official platforms increase surveillance policy
specific knowledge of the issue at stake and this, in turn, affects the probability of
19
approving the Treaty.
Conclusions
The news media serve the purpose of supplying the information that gives citizens
awareness of institutional responsibilities, the positions and policies of political elites, and
ultimately offer them guidance when making politically relevant choices. Traditional media
dedicate relatively little attention to European matters, particularly in routine periods (Peter
and De Vreese 2004). While the emergence of a European public sphere is highly desirable,
we have not yet seen it happening. A global transnational multilingual medium like the
Internet carries the potential to supply all the information Europeans need and often do
not find on traditional media. On the other hand, the Internet also contains large amounts
of noise.
Our study has addressed the effects of online news consumption on public opinion by
digging deeper into the variety of platforms that can supply such information on the World
Wide Web. As the Internet varies enormously in terms of the types of websites hosting
information, we have set apart loci that provide fact-checked information from those
containing a mixture of opinions, non-fact-checked information and reliable facts. We also
explored the effects of a particular fact-checked website on knowledge and voting
behaviour, in relation to the specific issue of the Fiscal Stability Treaty.
Our findings indicate that only certain online spaces exert an effect on public attitudes
about the EU. Those who search for information on European matters on the websites of
traditional media or on the websites of institutions are more positive towards the EU, and
this happens via increases in levels of knowledge about the Union. Those who perform a
similar search in spaces that contain miscellaneous information do not experience changes
in attitudes and do not appear to learn from them. This may be due to the difficulties that
users experience in separating the signal from the noise when the two are simultaneously
20
consumed. The analysis of the Irish Fiscal Compact Treaty, not only provides a
confirmation that fact-checked platforms tend to increase knowledge, but also shows their
effects on actual electoral behaviour. The direction of the effects in the case of attitudes as
well as in the case of behaviour is positive, signalling that when citizens access information
on fact-checked websites their knowledge and approval of the EU experiences an increase.
The effect is robust to controlling for use of traditional media; therefore the Internet is
effective independently from and in addition to them. Here we have shown that
environments prone to containing a mixture of accredited information and fragmented
opinions do not enlighten preferences via increased awareness and they do not impact
attitudes. However, this study is limited in its capacity to reveal nuances in the process of
learning by accessing online sources, as observational data per se are not completely suitable
to such an end. Insights on how different platforms affect cognition and learning can be
gathered by means of laboratory or filed experiments. Finally, this kind of data does not
account for the effects of content or for the way content is framed. This essay expands the
debate on effects of digital sources on both politically relevant knowledge and opinions,
but does not test the effects of specific content, which is certainly part of the equation and
could also be best clarified by experimental studies.
References
Ayres, Jeffrey M. 1999. From the streets to the Internet: The cyber-diffusion of contention.
The Annals of the American Academy of Political and Social Science 566 (1): 132-143.
Barabas, Jason, Jennifer Jerit, William Pollock, and Carlisle Rainey. (2014). "The
Question(s) of Political Knowledge." American Political Science Review, 108:(4).
Baron, Reuben M, and David A Kenny. 1986. The moderator–mediator variable distinction
in social psychological research: Conceptual, strategic, and statistical considerations.
Journal of personality and social psychology 51 (6): 1173.
Bartels, Larry M. 1993. Messages received: The political impact of media exposure.
American Political Science Review: 267-285.
Bennett, W Lance. 1990. Toward a theory of press‐state relations in the United States.
Journal of Communication 40 (2): 103-127.
Bennett, W Lance, and Shanto Iyengar. 2008. A new era of minimal effects? The changing
foundations of political communication. Journal of Communication 58 (4): 707-731.
21
Bimber, Bruce Allen. 2003. Information and American democracy: Technology in the evolution of
political power. Cambridge University Press.
Castells, Manuel. 2000. Materials for an exploratory theory of the network society1. The
British journal of sociology 51 (1): 5-24.
Dalton, Russell J, and Robert Duval. 1986. The political environment and foreign policy
opinions: British attitudes toward European integration, 1972–1979. British Journal
of Political Science 16 (01): 113-134.
De Vreese, Claes. 2003. Framing Europe: television news and European integration. Aksant
Academic Pub.
De Vreese, Claes H, and Hajo G Boomgaarden. 2005. Projecting EU Referendums Fear of
Immigration and Support for European Integration. European Union Politics 6 (1):
59-82.
———. 2006. Media Effects on Public Opinion about the Enlargement of the European
Union*. JCMS: Journal of Common Market Studies 44 (2): 419-436.
Delli Carpini, Michael X , and Scott Keeter. 1996. What Americans know about politics
and why it matters. Yale UniversityPress, New Haven, CT.
Franklin, Mark N, Cees Van der Eijk, and Michael Marsh. 1995. Referendum outcomes
and trust in government: Public support for Europe in the wake of Maastricht. West
European Politics 18 (3): 101-117.
Flanagin, A. J., & Metzger, M. J. (2000). Perceptions of Internet information credibility.
Journalism & Mass Communication Quarterly, 77(3), 515-540.
Flanagin, A. J., & Metzger, M. J. (2007). The role of site features, user attributes, and
information verification behaviors on the perceived credibility of web-based
information. New Media & Society, 9(2), 319-342.
Gabel, Matthew. 1998. Public support for European integration: An empirical test of five
theories. Journal of Politics 60: 333-354.
Gabel, Matthew, and Harvey D Palmer. 1995. Understanding variation in public support
for European integration. European Journal of Political Research 27 (1): 3-19.
Garrett, R Kelly, Dustin Carnahan, and Emily K Lynch. 2013. A turn toward avoidance?
Selective exposure to online political information, 2004–2008. Political Behavior 35
(1): 113-134.
Graber, Doris A. 2001. Processing politics: Learning from television in the Internet age. University of
Chicago Press.
Hainmueller, Jens. 2012. Entropy balancing for causal effects: A multivariate reweighting
method to produce balanced samples in observational studies. Political Analysis 20
(1): 25-46.
Hicks, Raymond, and Dustin Tingley. 2011. Causal mediation analysis. Stata Journal 11 (4):
605.
Holtz-Bacha, Christina , and Pippa Norris. 2001. " To Entertain, Inform, and Educate":
Still the Role of Public Television. Political communication 18 (2): 123-140.
Imai, Kosuke, Luke Keele, and Teppei Yamamoto. 2010. Identification, inference and
sensitivity analysis for causal mediation effects. Statistical Science 25 (1): 51-71.
Kritzinger, Sylvia. 2003. The influence of the nation-state on individual support for the
European Union. European Union Politics 4 (2): 219-241.
McLuhan, Marshall. 1964. The medium is the message. Media and cultural studies: keyworks:
129-38.
Metzger, M. J. (2007). Making sense of credibility on the Web: Models for evaluating online
information and recommendations for future research. Journal of the American Society
for Information Science and Technology, 58(13), 2078-2091.
22
Mondak, Jeffery J. 1995. Newspapers and political awareness. American Journal of Political
Science: 513-527.
Newton, Kenneth. 1999. Mass media effects: mobilization or media malaise? British Journal
of Political Science 29 (04): 577-599.
Norris, Pippa, and David Sanders. 2003. Message or medium? Campaign learning during
the 2001 British general election. Political communication 20 (3): 233-262.
Peter, Jochen, and Claes H De Vreese. 2004. In Search of Europe A Cross-National
Comparative Study of the European Union in National Television News. The
Harvard International Journal of Press/Politics 9 (4): 3-24.
Reif, K., and H. Schmitt. 1980. Nine Second-Order National Elections-A Conceptual
Framework for the Analysis of European Election Results. European Journal of
Political Research 8 (1): 3-44.
Robinson, John P, and Dennis K Davis. 1990. Television news and the informed public:
An information‐processing approach. Journal of Communication 40 (3): 106-119.
Robinson, John P, Mark R Levy, Dennis K Davis, William Gill Woodall, Michael
Gurevitch, and Haluk Sahin. 1986. The main source: Learning from television news. Sage
Publications Beverly Hills, CA.
Schuck, Andreas RT, and Claes H De Vreese. 2006. Between Risk and Opportunity News
Framing and its Effects on Public Support for EU Enlargement. European Journal of
Communication 21 (1): 5-32.
Scott, D Travers. 2007. Pundits in muckrakers’ clothing: political blogs and the 2004 US
presidential election. Blogging, citizenship, and the future of media: 39-58.
Van Klingeren, Marijn, Hajo G Boomgaarden, and Claes H De Vreese. 2013. Going Soft
or Staying Soft: Have Identity Factors Become More Important Than Economic
Rationale when Explaining Euroscepticism? Journal of European Integration 35 (6):
689-704.
Zaller, John. 1992. The nature and origins of mass opinion. Cambridge university press.
23
Figure 1. Path analysis testing the mediation role of Knowledge about the
EU.
Public Opinion
Towards the EU
Mediated information
Knowledge
24
Figure 2. Distribution of online platforms by country averages.
25
Table 1. Classification of hosting platforms
Traditional Media
Websites and
Official Websites of
Institutions
Social Networks,
Blogs, and Video
Sharing Platforms
Type of information
Vetted
Miscellaneous
Set of producers
Homogenous
Heterogeneous
26
Table 2. Causal mediation analysis: OLS with country fixed effects (type of platform). Effects of different platforms on Knowledge and
Attitudes
Traditional Media
Websites
Online Social
Networks
Blogs
Video Hosting
Websites
Attitudes
(3)
Attitudes
(4)
Attitudes
(5)
Attitudes
(6)
1.373***
(0.031)
0.076**
(0.036)
0.046***
(0.017)
0.030
(0.041)
-0.064
(0.067)
-0.102
(0.098)
0.100***
(0.009)
3.371***
(0.064)
0.076**
(0.036)
0.046***
(0.017)
0.030
(0.041)
-0.064
(0.067)
-0.102
(0.098)
0.100***
(0.009)
3.371***
(0.064)
0.076**
(0.036)
0.046***
(0.017)
0.030
(0.041)
-0.064
(0.067)
-0.102
(0.098)
0.100***
(0.009)
3.371***
(0.064)
0.076**
(0.036)
0.046***
(0.017)
0.030
(0.041)
-0.064
(0.067)
-0.102
(0.098)
0.100***
(0.009)
3.371***
(0.064)
0.076**
(0.036)
0.046***
(0.017)
0.030
(0.041)
-0.064
(0.067)
-0.102
(0.098)
0.100***
(0.009)
3.371***
(0.064)
yes
0.011**
0.075**
0.086**
0.121**
yes
0.006**
0.045**
0.051**
0.118**
yes
0.003
0.028
0.032
0.062
yes
0.004
-0.066
-0.062
-0.044
yes
-0.011
-0.105
-0.116
0.079
yes
Knowledge
(1)
Institutional and Official
websites
Traditional media websites
Online social networks
Blogs
Video Hosting Websites
0.107***
(0.024)
0.062***
(0.011)
0.034
(0.028)
0.040
(0.046)
-0.112*
(0.063)
Political Knowledge
Constant
ACME
Direct Effect
Total Effect
% of Total Effect Mediated
Controls
Institutional
and Official
Websites
Attitudes
(2)
27
Country fixed-effects
yes
yes
yes
yes
yes
yes
Observations
26,038
26,038
26,038
26,038
26,038
26,038
R-squared
0.155
0.160
0.160
0.160
0.160
0.160
RMSE
0.834
0.831
0.831
0.831
0.831
0.831
Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1
Note: Tables are reported in a short format, suppressing the outputs for control variables. Tables are available in the long format upon requests.
28
Table 3. Causal mediation analysis: OLS with country fixed effects (fact-checked
vs. non-fact-checked websites). Effects of Fact-checked versus Miscellaneous
Websites on Knowledge and Attitudes
Fact-checked Websites
Knowledge
Attitudes
(7)
(8)
Fact-checked Websites
Miscellaneous Websites
0.069***
(0.011)
0.016
(0.023)
1.176***
(0.042)
0.051***
(0.016)
-0.009
(0.033)
0.100***
(0.009)
3.370***
(0.064)
0.051***
(0.016)
-0.009
(0.033)
0.100***
(0.009)
3.370***
(0.064)
yes
yes
26,038
0.155
0.834
0.007**
0.051**
0.058**
0.122**
yes
yes
26,038
0.155
0.834
0.002
-0.008
-0.006
-0.032
yes
yes
26,038
0.155
0.834
Political Knowledge
Constant
ACME
Direct Effect
Total Effect
% of Total Effect Mediated
Controls
Country fixed-effects
Observations
R-squared
RMSE
Miscellaneous
Websites
Attitudes
(9)
Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1
29
Table 4a. Instrumenting type of platform: IVREG with country fixed effects
(Knowledge about the EU) and entropy balance.
Knowledge
(11)
(12)
(10)
Institutional and Official
websites
2.622***
(1.002)
Traditional media websites
0.551***
(0.205)
Fact-checked Website
0.456***
(0.170)
Miscellaneous Website
Constant
Online Access
Anderson-Rubin Wald test
Kleibergen-Paap rk LM statistic
Cragg-Donald Wald F statistic
Controls
Country fixed-effects
Observations
RMSE
(13)
1.162***
(0.111)
1.168***
(0.110)
2.337**
(0.916)
1.210***
(0.116)
First Stage
0.023***
0.107***
(0.001)
(0.008)
0.130***
(0.008)
0.024***
(0.002)
1.195***
(0.112)
14.34***
228.74***
210.69***
yes
yes
26,398
0.609
14.34***
14.34***
395.02*** 478.16***
1147.94*** 1437.28***
yes
yes
yes
yes
26,398
26,398
0.555
0.552
Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1
30
14.34***
69.13***
195.00***
yes
yes
26,398
0.614
Table 4b. Instrumenting type of platform: IVREG with country fixed effects
(Attitudes to the EU) and entropy balance.
(14)
Institutional and Official
websites
Attitudes
(15)
(16)
4.226***
(1.603)
Traditional media websites
0.882**
(0.350)
Fact-checked Website
0.730**
(0.287)
Miscellaneous Website
Political Knowledge
Constant
Online Access
Anderson-Rubin Wald test
Kleibergen-Paap rk LM statistic
Cragg-Donald Wald F statistic
Controls
Country fixed-effects
Observations
RMSE
(17)
0.100***
(0.034)
2.802***
(0.202)
First Stage
0.022***
(0.001)
14.34***
210.76***
195.40***
yes
yes
26,038
0.945
0.102***
(0.036)
2.745***
(0.211)
0.106***
(0.008)
0.128***
(0.008)
0.024***
(0.002)
14.34***
364.66***
1091.39***
yes
yes
26,038
0.858
14.34***
445.87***
1362.79***
yes
yes
26,038
0.851
14.34***
70.10***
190.77***
yes
yes
26,038
0.946
Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1
31
0.101***
(0.035)
2.755***
(0.209)
3.706***
(1.430)
0.121***
(0.033)
2.799***
(0.213)
Table 5. Fiscal Compact Referendum. Effects of regularly visiting the
Referendum Commission Website on Knowledge of the Treaty and Vote Choice.
VARIABLES
EU Commission Website (regularly)
Orderd probit
Knowledge of the
Treaty
Probit
Pr(Voting
Yes)
0.822***
(0.195)
0.851**
(0.348)
0.206
(0.148)
Knowledge of the Treaty
cut1
-1.452***
(0.542)
0.458
(0.553)
2.417***
(0.583)
cut2
cut3
Controls
yes
Town fixed-effects
yes
Entropy balancing
yes
Observations
949
Pseudo R-squared
0.06
Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1
32
yes
yes
yes
949