Assessing Adolescents’ Communicative Self-Efficacy To Discuss Controversial Issues:
Findings from a Randomized Study of the Word Generation Program
Journal:
Teachers College Records
Manuscript ID:
In press
Manuscript
Featured article
Keywords:
Communicative self-efficacy, classroom climate, civic
education
Abstract:
Communicative self-efficacy serves as an important link
between discussing controversial issues and civic engagement
because confidence in one’s discourse skills is important to
managing conflicting perspectives and developing solutions to
community-based problems. Freely available to schools, Word
Generation is a cross-content literacy program that supports
teachers in the four main content areas –English language arts,
social studies, science, and math— to embed learning of
controversial issues through classroom discussions, subjectspecific lessons, and writing. Middle school students (N = 5,870)
from highly diverse backgrounds participated in a randomized
study of the intervention that was conducted in twelve middle
schools located in an urban school district. We analyzed survey
data based on students’ self-reported ratings on their
communicative self-efficacy, as indicated by confidence to
participate in discussions of fifteen different controversial issues
related to politics, society, and science. Paired sample t-tests
indicate that treatment students reported higher communicative
self-efficacy than control students on a set of topics immediately
covered prior to testing, but not on the set of topics covered in the
previous year. This study informs curriculum developers, policy
makers, and educators to consider the importance of incorporating
discussions of controversial issues within a framework of subjectspecific instructions.
Declines in voting and volunteering among young American adults (18-29 years old) in
the last decade affirm the need to help adolescents become civically engaged or participate in
activities to address problems in their community (Adler & Googin, 2005; Vercellotti & Matto,
2013). The most recent nationwide report on youth civic engagement indicates that adolescents
who frequently discussed controversial issues with peers and families were more likely to
become politically engaged and vote in the 2012 elections (CIRCLE, 2013). Communicative
self-efficacy serves as an important link between discussing controversial issues and civic
engagement because confidence in one’s discourse skills is important to managing conflicting
perspectives and developing solutions to community-based problems (Kim & Ball-Rokeach,
2006; Hackler, Ho, & Urquhart-Ross, 1974). Based on a randomized study of the Word
Generation program, this study seeks to understand whether a classroom intervention that
provides information about and supports discussion of controversial issues can improve
adolescents’ communicative self-efficacy.
Schools can provide students with opportunities to engage in democratic practices such as
debates and political simulations with peers and teachers who hold diverse opinions and beliefs
(Author, 2013; Guttmann & Thompson, 1996; McDevitt & Canton-Rosser, 2009; Wray-Lake &
Syvertsen, 2011; Verba, Schlozman, & Brady, 1995). Although teachers can take advantage of
this diversity of beliefs and values to foster active classroom discussions (Steiner, Brzuzy,
Gerdes, & Hurdle, 2003), discussions are rarely emphasized (Hahn, 1991; Hess, 2002;
Newmann, 1988; Nystrand, Gamoran, & Carbonaro, 1998). Nystrand and colleagues (1998)
evaluated 48 high school social studies classes and found that students participated in an average
of only half a minute per class period of classroom discussion, while 63% of the observed class
periods lacked explicit support for discussions. Controversial issues can serve as a basis for
discussion, which can help students develop stronger commitment to democratic values such as
toleration of differences and support for equality (Hess, 2002; Lockwood & Harris, 1985;
Torney-Purta, Lehmann, Oswald, & Schulz, 2001). Although there are a number of social studies
interventions that use controversial issues as a basis for discussion (Feldman, Pasek, Romer, &
Jamieson, 2007; Hess & Posselt, 2002; Venville & Dawson, 2010), a literacy intervention called
Word Generation (WG) extends learning of controversial issues by incorporating a
comprehensive framework that includes subject-specific instructions in math, English language
arts (ELA), science, and social studies.
Previous evaluations of the WG program have examined a broad range of research topics
including: writing outcomes (Mancilla-Martinez, 2010), vocabulary development (Authors et al.,
2012), and differentiated effects for English language learners (Authors et al., under review).
This is the first study of the WG program that reports on the communicative self-efficacy aspect
of classroom discussion. The purpose of this study is to determine whether participation in the
WG program has an impact on students’ communicative self-efficacy to discuss controversial
issues.
First, we examine the literature on controversial issues discussion in the classroom and
students’ development of communicative self-efficacy. Next, we report a study in which we
assessed the communicative self-efficacy of adolescents who enrolled in schools randomly
assigned to the WG program. Finally, we discuss the implications of our findings for research
and practice.
ADOLESCENTS’ EXPERIENCE OF PARTICIPATING IN STRUCTURED DISCUSSIONS
For the purposes of this study, discussion is defined as a structured form of discourse
where participants research a particular issue, adopt a position, and then debate with each other
(Maloney & Simon, 2006; Steiner et al., 2003). The key features in structured discussion include:
(a) moderation by a facilitator, (b) exploration of differences, and (c) explicit goals of increasing
shared meaning and collaboration (Ellinor & Gerard, 1998; Isaacs, 1999; Zorn, Roper, Weaver,
& Rigby, 2012). Discussions have been characterized as complex interpersonal activities because
participants deliberate on problems that contain a number of possibilities with no definitive
solution (Venville & Dawson, 2010). Schools are increasingly concerned about meeting
Common Core State Standards (CCSS) that emphasize students’ speaking and listening skills
during classroom discussions, in particular to articulating and supporting arguments with
evidence (National Governors Association Center for Best Practices, 2010; Swanson, 2013). The
Center for Information and Research on Civic Learning and Engagement (CIRCLE) conducts
research on civic education in schools and recommends exposing adolescents to classroom
debates on controversial issues (CIRCLE, 2013).
Adolescents are more likely drawn to discussions that are organized around controversial
issues (Hess & Posselt, 2002). Immigration policy and the environmental state of the world are
topic questions that have the potential of supporting rich discussions (Harwood & Hahn, 1990).
These issues challenge students to consider different perspectives, while forcing them to rethink
their own stance (Hess & Posselt, 2002). Students are naturally interested in topics that are
relevant to their social context (Harwood & Hahn, 1990). A study found that adolescents reported
more favorable attitudes towards social studies classrooms that incorporated discussions of
controversial issues (Hess, Levstik, & Tyson, 2008). In the likely event that adolescents want to
know more about controversial issues, they tend to follow up on current events through the
media and discuss political issues with their friends and family (Lee, Shah, & McLeod, 2012).
These findings indicate that controversial issues can enhance students’ interest and participation
in classroom discussions and extend opportunities for learning through self-initiated activities.
We examine teaching strategies and curricula that can help adolescents develop argumentation
skills to engage in discourse with peers and teachers.
Guided instructions and curriculum material may be required to scaffold students’
learning in regards to skillfully deliberation on controversial issues. Facilitators, such as teachers,
can guide students in understanding that competing points of views need to be equally
represented (Maloney & Simon, 2006). For example, teachers can limit contributions from
outspoken students, so that more hesitant students have opportunities to participate (Harwood &
Hahn, 1990). Certain curricula are also designed to provide turn-taking rules and arrangements
for students to adopt a specific position and role (Herrenkohl, Palincsar, DeWater, & Kawasaki,
1999). Recent studies indicate that students lacking training in argumentative discourse are more
likely to focus on supporting their own positions, while paying little attention to alternative
perspectives (Felton, 2004; Felton & Kuhn, 2001; Kuhn & Udell, 2003). Felton (2004) analyzed
classroom observation data from a study of middle school students (n = 48) who were randomly
assigned to an intervention that taught students to consider different arguments within a
controversial topic such as abortion and capital punishment. In comparison with control students
who also engaged in classroom debates, treatment students were more likely to use advanced
argumentative strategies related to counterarguments and rebuttal. These findings strongly
suggest that facilitators and curricula can provide the support required to enable adolescents to
productively engage in classroom discussions.
COMMUNICATIVE SELF-EFFICACY
Based on Bandura's theories on self-efficacy (1986, 1987), communications researchers
argue that communicative self-efficacy plays a critical role in developing civic engagement (Kim
& Ball-Rokeach, 2006). This concept refers to an individuals’ belief in their confidence level to
communicate their own words, thoughts, and feelings to others (Lonie & Rahim, 2010; Milstein,
2005; Zorn et al., 2006). It is important to note that communicative self-efficacy refers to
individuals’ beliefs regarding their capabilities to communicate effectively with other, rather than
actual capabilities (Bandura, 1986, 1997; Milstein, 2005). Communicative self-efficacy may be
determined by a number of factors that include: discussion frequency, topic, and audience. For
instance, a person may have high self-efficacy for discussing sports among friends, but low selfefficacy in matters of discussing politics with strangers (Koesten, Miller, & Hummert, 2002;
Zorn et al., 2006). Having more frequent discussion opportunities in the classroom can improve
communicative self-efficacy (Lin & Liuwood, 1997; Venville & Dawson, 2010). Liu and
Littlewood (1997) reports that East Asian college students (n = 2,156) who enrolled in English
learning courses with more frequent speaking opportunities gave themselves higher ratings on
their confidence and speaking abilities than those enrolled in courses supporting less frequent
speaking opportunities.
Organizing discussions around controversial topics can also make a difference in
participants’ communicative self-efficacy. Zorn and colleagues (2006) analyzed data from a nonrandomized study of adults (n = 59) who participated in focus groups that support discussions of
controversial issues. These focus groups consist of a small number of people (usually 6-12
participants) who are brought together by researchers for the purpose of studying their views on
particular issues. Facilitators guide the discussions so that participants have equal opportunities
to express their opinions (Zorn et al., 2006, 2012). The researchers (Zorn et al., 2006) found
significant increases in pre to post-test communicative self-efficacy scores, as indicated by
confidence and motivation to engage in public discussions of human biotechnology issues.
These findings suggest a possible relationship between participants’ experience of discussing
controversial issues and improvements in their communicative self-efficacy.
Although these studies identify the promise of discussion in improving individuals’
communicative self-efficacy, several questions remain unanswered. First, most of these
communicative self-efficacy studies examine how adults (mean age of 32 years old) experience
discussions of controversial issues (Zorn et al., 2006, 2012), but there is much less research on
adolescents. With one exception (Venville & Dawson, 2010), the studies do not include control
groups (Klosterman & Sadler, 2009; Liu & Littlewood, 1997; Zorn et al., 2006, 2012). Last,
these findings are mainly relevant to a non-diverse sample that features a low representation of
ethnic minorities including Asians, African Americans, and Latinos (Cross, Taasoobshirazi,
Hendricks, & Hickey 2008; Venville & Dawson, 2010; Zorn et al., 2006, 2012). In the next
section, we provide an overview of a cross-content literacy intervention implemented in middle
schools that organizes vocabulary instructions around discussions of controversial issues.
WORD GENERATION PROGRAM
The Word Generation (WG) is a cross-content literacy program delivered at the
classroom or grade level that instructs students to learn five all-purpose academic words, which
are embedded in brief passages covering a different controversial issue each week (Author et al.,
2009). The program features controversial topics based on a broad range of political and sciencebased controversies, such as whether the government should fund stem cell research or animal
testing. As part of the WG program, teachers in the four main content areas – ELA, social
studies, science, and math – present materials related to the controversial issue of the week and
explore academic language that is embedded in the curriculum through discourse and writing
(Author et al., 2009). For fifteen minutes a day, teachers and students engage in program
activities related to vocabulary instruction and classroom discussions. Students have books
containing general information about each controversial issue. These books also contain
examples of various opinions on a particular issue, which students can choose to defend during
classroom debates.
We present an example of the organization of one week’s activities; each week follows a
similar sequence (see http://wg.serpmedia.org/ for sample materials). On Monday, the ELA
teacher presents a text that starts with a narrative about a girl who was accidentally shot and
paralyzed, but for whom stem cell research might offer a cure; the rest of the brief text presents
arguments against (religious objections, use of embryos) and in favor of federal funding of stem
cell research (scientific and medical advances, global competitiveness). The math teacher
provides a lesson that prompts students to calculate how many stem cells are in the human body.
On Wednesday, the social studies teacher facilitates debate with the whole class, where students
can choose to defend arguments for or against funding stem cell research. Given sufficient time,
students also have opportunities to rebut the opposing group’s argument. The science teacher
assigns an experiment where students hypothesize and test whether their peers have correct
assumptions about stem cells. On Friday, the ELA teacher assigns a writing exercise that prompts
students to write a persuasive essay defending their position on the topic. The weekly sequence
was developed to provide students with recurrent exposure and opportunities to learn about the
controversial topics in various subject-specific contexts (Author et al., 2009).
RESEARCH QUESTIONS
This study seeks to assess students’ communicative self-efficacy to discuss a particular
set of topics that were covered in the WG program, rather than a general self-efficacy to discuss
all controversial issues. Given our discussion of the research on using controversial issues to
improve communicative self-efficacy, we pose the following research questions (RQ):
RQ 1: Did students in the treatment and control group differ in communicative selfefficacy on general topics of interest to adolescents that were not covered in the WG program?
RQ 2: Did treatment students report higher communicative self-efficacy on topics that
had been discussed as part of the WG program during the school year immediately prior to
testing, in comparison with control students?
RQ 3: Did treatment students report higher communicative self-efficacy on topics that
they had been exposed to as part of the WG program during the school year prior to testing, in
comparison with control students?
METHOD
PARTICIPANTS
<INSERT TABLE 1>
From Fall 2010 to Spring 2012, middle school students (N = 7,541) participated in a
randomized study of the Word Generation (WG) intervention that was conducted in an urban
school district located in a West Coast metropolitan area of the United States. Our analytical
sample (N = 5,870) was restricted to students who responded to at least three out of the fifteen
(20%) survey items analyzed in this study. Table 1 presents grade level contributions from each
middle school that participated in the randomized study. Although an equal number of schools
were randomly assigned to treatment (n = 6) and control (n = 6) conditions, the sample contains
more students who enrolled in treatment (n = 3,518) than control (n = 2,352) schools. Across the
three middle school grades, there is a near equal representation of sixth grade (31.5%), seventh
grade (32.6%), and eighth grade (35.9%) students in the treatment and control schools.
<INSERT TABLE 2>
Table 2 indicates that gender of the participants was approximately equally divided:
50.3% males and 49.7% females. The sample consists of students from extremely diverse
racial/ethnic backgrounds, which includes: Asians (54.1%), Hispanics (19.3%), and African
Americans (6.9%). As is typical in many American urban settings (Wells & Lewis, 2006), a
majority of the participants (63.8%) are from low socioeconomic (SES) homes (as indicated by
eligibility for the Free and Reduced Lunch program) and 56.8% are English language learners 1.
PROCEDURE
In the last year of the two-year study, we collected data from the twelve middle schools in
the urban school district that participated in a randomized study of the WG intervention. Prior to
randomization, state accountability data was used to rank schools on several school-level
variables based on the percentage of students identified as ethnic minorities, low-income status,
and English language learners, as well as the school’s prior academic achievement. Using
propensity score matching, we created composite scores based on the school-level variables and
used the composite scores to determine each school’s rankings. Each sequential pair of schools
formed a dyad within which randomization occurred. Paired t-tests on each variable used to
create the composite scores confirm that treatment and control schools are comparable 2.
In the six treatment schools, teachers in the four main content areas (e.g. ELA, social
studies, science and math) were required to implement the WG program, whereas teachers in the
six control schools did not use the WG or any similar program and provided “business as usual”
instruction (Author et al., 2012). In our past evaluations of the WG program, classroom
observers gave higher discussion quality ratings in classrooms using the WG schools than
classrooms in control schools (Author et al., under review) 3.
<INSERT TABLE 3>
Students in the WG program encountered a unique set of controversial topics in the first
(2010-2011) and second school year (2011-2012) of the randomized study (See Table 3). Since
we gave our efficacy survey at the end of the 2011-2012 school year, all students in WG schools
had encountered the topics that were taught that year. We refer these topics as “Treatment topics”
because all students were exposed to them during the year that we conducted the efficacy survey.
The seventh and eighth graders enrolled in the WG program also had experience with the topics
covered in the previous year (2010-2011), when they were sixth and seventh grade students. We
refer to these topics as “Follow-up topics” because at the time of testing, students had not
experienced Word Generation curriculum on those topics for more than a year. Analyzing the
Follow-up topics reveals how long any topic-specific increases in self-efficacy last. None of the
students in the treatment or control schools encountered the “Control topics”.
MEASURES
All students who participated in the randomized trial in the year of this study completed
survey items that assessed their communicative self-efficacy to discuss controversial issues.
Students were asked to respond to the guiding question, “How confident are you in being able to
participate in a discussion about the following topics?” and then assessed on a series of 15 topics
pertaining to a particular controversial issue (See Table 1 in the Appendix section). For example,
students were assessed the question, “How confident are you in being able to participate in a
discussion about the topic in regards to should colleges use affirmative action?” and they
answered based on a five-point Likert Scale of (1) not at all, (2) a little, (3) somewhat, (4) very,
and (5) extremely. Eleven survey items were based on the topics that were covered in the WG
program. The other four items also assessed topics of national interest to youth (Kettering
Foundation, 2011) that were not covered in the WG program.
ANALYSIS PLAN
The research questions posed in this study are answered by two-sample mean comparison
t -tests to compare differences between students in the treatment and control groups on the
average communicative self-efficacy scores of the Control topics (Research Question 1),
Treatment topics (Research Question 2), and Follow-Up topics (Research Question 3). Students
responding to at least 75% of the controversial issue items corresponding to the Control,
Treatment, and Follow-Up topics were included in our analysis of average scores. We also used
differences in the average scores of students in the treatment and control groups to calculate
overall effect sizes. For the purposes of providing additional information, we also report the
communicative self-efficacy scores for each controversial issue item.
RESULTS
Table 4 presents means and standard deviations of students’ communicative self-efficacy
scores on each topic item, as well as average scores in the Control, Treatment, and Follow-Up
topics.
<INSERT TABLE 4>
RQ 1: Did students in the treatment and control group differ in communicative self-efficacy on
general topics of interest to adolescents that were not covered in the WG program?
The average communicative self-efficacy scores for the Control topics were virtually
identical between treatment (M = 2.92, SD = 0.79) and control (M = 2.92, SD = 0.78) students.
No differences were found between treatment and control students for the Control topics.
Interestingly, we find that control students (M = 2.83, SD = 1.21) reported higher communicative
self-efficacy than treatment students (M = 2.75, SD = 1.19) on one of the four items in the
Control topics (i.e. “Should schools have a vocational track?”). No differences in communicative
self-efficacy scores were found between treatment and control students on the other three items.
RQ 2: Did treatment students report higher communicative self-efficacy on topics that had been
discussed as part of the WG program during the school year immediately prior to testing, in
comparison with control students?
Our results indicate that treatment students reported higher communicative self-efficacy
than control students on Treatment topics, which suggests that participating in the WG program
led to improvements in students’ confidence to discuss topics that were immediately covered
prior to testing. Based on t-test analyses, the difference of 0.10 in average scores on the
Treatment topics between treatment (M = 2.61, SD = 0.82) and control (M = 2.51, SD = 0.75)
students was statistically significant, t (5,814) = -2.67, p < 0.001. We used this difference in
average scores to calculate the average treatment effect (d = 0.13).
For the purposes of analyzing controversial issues corresponding to the Treatment topics,
treatment students report higher communicative self-efficacy than control students on four out of
the six items in the Treatment topics. For example, treatment students (M = 2.43, SD = 1.34)
report more confidence than control students (M = 2.30, SD = 1.31) to participate in a discussion
about whether the government should allow animal testing, t (5,825) = -3.51, p < 0.001.
RQ 3: Did treatment students report higher communicative self-efficacy on topics that they had
been exposed to as part of the WG program during the school year prior to testing, in
comparison with control students?
Similar to our results in RQ 1, treatment and control students had virtually identical
average communicative self-efficacy scores on Follow-Up topics. These findings suggest that
participating in the WG program did not lead to improvements on students’ confidence to discuss
Follow-Up topics. Control students reported higher communicative self-efficacy than treatment
students on two topic items. For instance, control students reported more confidence about
engaging in discussions on whether Creation should be taught in school and the difference of
0.12 was found to be statistically significant, t (3,992) = -3.37, p < 0.001. In contrast, treatment
students reported higher communicative self-efficacy on only one topic (i.e. “Should the
government fund stem cell research?”).
DISCUSSION
The results from this evaluation suggest that curricular emphasis on supporting regular
discussions of controversial issues can be effective in improving students’ communicative selfefficacy. We note, however, that program effects on students’ communicative self-efficacy were
limited to the topics discussed as part of the WG program during the school year immediately
prior to testing. These findings are congruent with past studies that found improvements in
participants’ communicative self-efficacy when they participated in structured discussions of
controversial issues (Liu & Liuwood, 1997). Although past studies suggest that adults can benefit
from discussions of controversial issues (Zorn et al., 2006, 2012), the current study extends these
findings to adolescents. In addition to displaying confidence to discuss socioscientific topics
such as stem cell research, students in the WG program reported more confidence to discuss
social issues based on the moral implications of death penalty and rap music. Although
controversial issues can serve as the basis for active discussions among adolescents, careful
facilitation and curricular support may be needed to effectively build students’ communicative
self-efficacy to discuss these issues with their peers.
We consider possible explanations of why the WG program may be effective in
improving students’ communicative self-efficacy to discuss specific controversial issues. Gist
and colleagues (1991) suggests that structured discussions provide students with opportunities to
observe similar-aged peers discuss and work towards a collective focus on developing a solution.
In addition, teachers can facilitate discussions so that adolescents have opportunities to speak in
a structured environment that aims to give respectful treatment to the speaker. These specific
discussion features can contribute to participants having positive beliefs in their ability to engage
in discussions (Zorn et al., 2006). Another explanation is that the WG program provides subjectspecific instructions that can help build students’ knowledge of controversial issues. Past studies
indicate that students learn argumentation skills more effectively when these instructions are
combined with subject specific lessons (Angeli & Valanides, 2009, Means & Voss, 1996;
Terenzinin, Springer, Pascarella, & Nora, 1995; Tsui, 1999). Means and Voss (1996) found that
knowledge is related to performance during debate, such as the number and types of reasons
generated by participants. Last, the WG program provides students with opportunities to reflect
on different arguments within a controversial issue, which helps them understand the
argumentative structure behind discourse practice (Felton, 2004; Kuhn & Udell, 2007). Students
may develop more confidence in using advanced argumentation skills when they are directed to
consider the “opponent’s assertions and the use of strategies to influence them” (Kuhn & Udell,
2007, p. 91). We acknowledge these findings as possible explanations of why the WG program
may be effective in helping students develop communicative self-efficacy.
Results from this study also indicate that treatment and control students did not differ in
their communicative self-efficacy on Control and Follow-Up topics. Whereas no difference was
expected for Control topics, the lack of difference on Follow-up topics indicates that the rise in
topical self-efficacy is of limited duration. Perhaps this finding reflects students’ need for
ongoing involvement with issues to maintain the knowledge structures that support
communicative self-efficacy (Cooper & Sweller, 1987). Controversial issues evolve rapidly and
require sustained attention to current events through the media (Lin & Salwen, 1997). It is worth
noting that the WG curriculum is not very intensive; each controversial topic gets at most 60-80
minutes of classroom attention. Extensions or improvements to the WG curriculum that includes
media and technology integration (Feldman et al., 2007), explicit critical-thinking instructions
(Angeli & Valanides, 2009), and support to engage parents in facilitating discussions at home
(Bergeron, Weemuth, Rhodes, & Rudenga, 1996) might extend treatment effects.
STRENGTHS, LIMITATIONS AND SUGGESTIONS
The strength of the study is the large-scale randomized design, which allows for rigorous
comparisons between students who participated in the WG program and those who did not.
According to past research (Fan, 2001), the tests used in this study can be considered statistically
powerful because of the large sample size that was analyzed. Also, the study extends previous
findings with its focus on mostly low-income adolescents from diverse ethnic backgrounds.
Although these design features enhance the study’s importance, we acknowledge certain
limitations.
First, the study relies on students’ self-reported confidence to engage in discussions of
controversial issues; we have not captured their actual performance during classroom
discussions of controversial issues. It is certainly possible that a student who reported having
higher confidence to discuss controversial issues will be actively engaged during classroom
discussions. The same student, however, is equally likely to have a weak performance during
classroom discussion. Past studies on self-efficacy indicate that overconfident individuals may
inflate their perceived performance levels, which results in decreased attention to understanding
the topics (Powers, 1991; Vancouver, Thompson, & Williams, 2001). Second, we did not
administer a pre-test survey, which could have provided baseline information on how students
perceived their own communication competencies prior to the randomized study. Last, it cannot
be inferred at this point whether differences in students’ communicative self-efficacy may be
attributed to other factors such as value orientation, prior knowledge, and conceptual
understanding (Ennis, 1998). Although the results of this study provide support for the
hypothesized effects, the specific processes that produced these effects require additional
analyses.
The results from our study also indicate the need for more research in the following areas
that relate to examining adolescents’ communicative self-efficacy. In understanding how students
participate in these discussions, past studies indicate the need to understand how communicative
self-efficacy interacts with other factors that relate to students’ prior knowledge, conceptual
understandings, and epistemological beliefs regarding controversial issues (Nussbaum &
Bendixen, 2003; Sadler, Chambers, & Zeidler, 2004; von Aufschnaiter, Erduran, Osborne, &
Simon, 2008). With the use of student interviews and video footage of classroom discussions, we
could also answer qualitative questions about how students respond to specific discourse
strategies, topics, and formats used in discussions (Maloney & Simon, 2006). Future studies
should also consider longitudinal development of communicative self-efficacy, which informs
how these skills may translate into future political behavior. Expanding research in these critical
areas is important to developing comprehensive assessments and teaching tools that will help
schools effectively use controversial issues as a basis for classroom instructions and discussions.
The purpose of this study was to understand whether a literacy intervention that organizes
classroom discussions around controversial issues has the potential of influencing students’
communicative self-efficacy. Findings from the study suggest that students who enroll in this
intervention are more likely to exhibit confidence in their abilities to engage in discussions of
controversial issues that are taught and discussed as part of the program. Despite these positive
findings, program effectiveness might well be improved by spending more time on each topic,
adding opportunities to revisit certain issues, and/or extending deliberation to out-of-school
contexts (e.g. home, online, and after-school programs). Nonetheless, the implications of this
study highlight the importance of combining frequent discussion opportunities of controversial
issues with instruction in subject-specific content. In facilitating discussions of controversial
issues, teachers face the daunting challenge of not only ensuring that students maintain respect
towards each other, but also making sure that competing points of views are equally represented
(Halpern, 1998; Hess, 2002). The goal of this exercise is not to encourage students to take a
dogmatic stand on a particular issue, but to help them develop confidence about their skills to
consider multiple sides of an issue as informed and critical consumers of information.
FOOTNOTES
1 Preliminary analyses (results not shown in the tables) indicate that students’
communicative self-efficacy did not vary substantially across key demographic features
(e.g. race/ethnicity, socioeconomic status, gender); thus, providing justification of not
controlling for these demographic features (Ammentorp, Sabroe, Kofoed, & Mainz,
2007; Finlay, Maughan, & Webster, 1998; Tankova, Dakovska, & Koev, 2004). We also
do not account for nesting within schools in our analysis given the small number of
schools (n = 12) that participated in the study. The intra-class coefficient (ICC ) was
found to be 0.006, which indicates no considerable differences with respect to the
proportion of between-school variance.
2 Treatment and control schools are comparable across all school-level variables, which
suggests that the randomization process was successful. In our analysis of the second
year in the randomized study, we acknowledge significant differences in academic
achievement –as indicated by CST ELA, vocabulary, and reading comprehension scores –
between WG and control schools. This finding suggests that a year of participating in the
WG program helped improve students’ academic skills.
3 The author (under review) compared classroom discussion quality in schools that
participated in a randomized study of the WG program. Observers attended training
sessions that instructed on using standardized codes for teacher and conversational
moves. Up to three observers were assigned to rate classrooms in treatment and control
schools. Participating teachers were randomized and a schedule was created identifying
specific observation periods to be conducted on a given day. The discussion quality rating
was developed to account for several measures including: the number of students
participating, frequency of teacher initiating open-ended question, and students’
intellectual contributions. In 168 observations conducted in content-area classrooms
across treatment (n = 80) and control schools (n = 88), observers gave higher classroom
discussion quality rating on classrooms using the WG program, in comparison with those
in the control schools.
REFERENCES
Ammentorp, J., Sabroe, S., Kofoed, P.E., & Mainz, J. (2007). The effect of training in
communication skills on medical doctors’ and nurses’ self-efficacy: a randomized
controlled trial. Patient education and counseling, 66(3), 270-277.
Adler, R. P., & Goggin, J. (2005). What do we mean by “civic engagement”? Journal of
Transformative Education, 3, 236–253.
Angeli, C., & Valanides, N. (2009). Instructional effects on critical thinking: Performance on illdefined issues. Learning and Instruction, 19, 322–334.
Author. (2013). Citizenship education in American schools and its role in developing civic
engagement: A review of the research. Educational Review.
Author et al. (2009). Generating knowledge of academic language among urban middle school
students. Journal of Research on Educational Effectiveness, 2(4), 325-344.
Author et al. (2012). Language proficiency, home-language status, and English vocabulary
development: A longitudinal follow-up of the Word Generation program. Bilingualism:
Language and Cognition (15), 437-451.
Author et al. (Under review). Word Generation randomized trial: Discussion mediates the impact
of program treatment on academic word learning. Reading Research Quarterly.
Author et al. (Under review). Differential effects of a systematic vocabulary intervention on
adolescent language minority students with varying levels of English proficiency.
International Journal of Bilingualism.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory.
Englewood Cliffs, NJ: Prentice-Hall.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman.
Bergeron, B. S., Weemuth, S., Rhodes, M., & Rudenga, E. A. (1996). Language development and
thematic instruction: supporting young learners at risk. Childhood Education, 72(3), 141145.
Button, M., & Ryfe, D. (2005). What can we learn from the practice of deliberative democracy? In J.
Gastil & P. Levine (Eds.), The deliberative democracy handbook. San Francisco: Josey-Bass.
Center for Information and Research on Civic Learning & Engagement (CIRCLE) (2013). All together
now: Collaboration and innovation for youth engagement. Boston, MA: Tufts University.
Cooper, G., & Sweller, J. (1987). Effects of schema acquisition and rule automation on mathematical
problem-solving transfer. Journal of Educational Psychology, 79(4), 347.
Cross, D., Taasoobshirazi, G., Hendricks, S., & Hickey, D. T. (2008). Argumentation: A strategy
for improving achievement and revealing scientific identities. International Journal of
Science Education, 30(6), 837-861.
Davies, S. R. (2009). Doing dialogue: Genre and flexibility in public engagement with science.
Science as Culture, 18(4), 397-416.
Ellinor, L., & Gerard, G. (1998). Dialogue: Rediscover the transforming power of conversation.
New York: J. Wiley & Sons.
Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed research.
Educational Researcher, 18(3), 4-10.
Fan, X. (2001). Parental involvement and students' academic achievement: A growth modeling
analysis. The Journal of Experimental Education, 70(1), 27-61.
Feldman, L., Pasek, J., Romer, D., & Jamieson, K. H. (2007). Identifying best practices in civic
education: Lessons from the Student Voices program. American Journal of Education,
114(1), 75-100.
Felton, M. K. (2004). The development of discourse strategies in adolescent argumentation. Cognitive
Development, 19(1), 35-52.
Felton, M., & Kuhn, D. (2001). The development of argumentive discourse skill. Discourse Processes,
32(2-3), 135–153.
Finlay, I., Maughan, T., & Webster, D. (1998). A randomized controlled study of portfolio learning in
undergraduate cancer education. Medical Education, 32, 172-176.
Gist, M. E., Stevens, C. K., & Bavetta, A. G. (1991). Effects of self-efficacy and post-training
intervention on the acquisition and maintenance of complex interpersonal skills. Personnel
Psychology, 44(4), 837-861.
Guttmann, A., & Thompson, D. F. (1996). Democracy and disagreement. Harvard: Harvard
University Press.
Hackler, J. C., Ho, K.-Y., & Urquhart-Ross, C. (1974). The willingness to intervene: Differing
community characteristics. Social Problems, 21(3), 328–344.
Hahn, C. L. (1991). Controversial issues in social studies. In J. Shaver (Ed.), Handbook of
research on social studies teaching and learning (pp. 470-480). New York, NY:
MacMillan Reference Books.
Halpern, D. F. (1998). Teaching critical thinking for transfer across domains: Disposition, skills,
structure training, and metacognitive monitoring. American Psychologist, 53(4), 449.
Harwood, A. M., & Hahn, C. L. (1990). Controversial issues in the classroom. Bloomington, IN: ERIC
Clearinghouse for Social Studies/Social Science Education.
Herrenkohl, L. R., Palincsar, A. S., DeWater, L. S., & Kawasaki, K. (1999). Developing scientific
communities in classrooms: A sociocognitive approach. Journal of the Learning Sciences, 8(3-4),
451-493.
Hess, D. E. (2002). Discussing controversial public issues in secondary social studies
classrooms: Learning from skilled teachers. Theory & Research in Social Education,
30(1), 10-41.
Hess, D., & Posselt, J. (2002). How high school students experience and learn from the discussion of
controversial public issues. Journal of Curriculum and Supervision, 17(4), 283-314.
Isaacs, W. (2008). Dialogue: The art of thinking together. London: Random House Digital, Inc.
Kettering Foundation. (2011). Naming and framing difficult issues to make sound decisions.
Dayton, OH: Charles F. Kettering Foundation.
Kim, Y.-C., & Ball-Rokeach, S. J. (2006). Community storytelling network, neighborhood
context, and civic engagement: A multilevel approach. Human Communication Research,
32(4), 411–439.
Kim, J., Wyatt, R. O., & Katz, E. (1999). News, talk, opinion, participation: The part played by
conversation in deliberative democracy. Political Communication, 16(4), 361-385.
Klosterman, M. L., & Sadler, T. D. (2010). Multi‐level Assessment of Scientific Content
Knowledge Gains Associated with Socioscientific Issues‐based Instruction. International
Journal of Science Education, 32(8), 1017-1043.
Koesten, J., Miller, K. I., & Hummert, M. L. (2002). Family communication, self-efficacy, and white
female adolescents' risk behavior. The Journal of Family Communication, 2(1), 7-27.
Kuhn, D., & Udell, W. (2003). The development of argument skills. Child Development, 74(5), 1245–
1260.
Kuhn, D., & Udell, W. (2007). Coordinating own and other perspectives in argument. Thinking &
Reasoning, 13(2), 90-104.
Lee, N.-J., Shah, D. V., & McLeod, J. M. (2012). Processes of political socialization: A Communication
mediation approach to youth civic engagement. Communication Research, 40(5), 669–697.
Lin, C. A., & Salwen, M. B. (1997). Predicting the spiral of silence on a controversial public issue.
Howard Journal of Communications, 8(1), 129-141.
Liu, N.F., & Littlewood, W. (1997). Why do many students appear reluctant to participate in
classroom learning discourse? System, 25(3), 371-384.
Lockwood, A. L., & Harris, D. E. (1985). Reasoning with democratic values: Ethical problems
in United States history. New York: Teachers College Press.
Long, S., & Long, R. (1975). Controversy in the classroom: Student viewpoint and educational outcome.
Teaching Political Science, 2, 275-299.
Lonie, J. M., & Rahim, H. (2010). Does the addition of writing into a pharmacy communication skills
course significantly impact student communicative learning outcomes? A pilot study. Journal of
Pharmacy Practice, 23(6), 525-530.
Maloney, J., & Simon, S. (2006). Mapping children’s discussions of evidence in science to assess
collaboration and argumentation. International Journal of Science Education, 28(15),
1817-1841.
Martocchio, J. J., & Webster, J. (1992). Effects of feedback and cognitive playfulness on performance in
microcomputer software training. Personnel Psychology, 45(3), 553-578.
Mancilla-Martinez, J. (2010). Word meanings matter: Cultivating english vocabulary knowledge in fifth
grade spanish-speaking language minority learners. TESOL Quarterly, 44(4), 669-699.
Means, M. L., & Voss, J. F. (1996). Who reasons well? Two Studies of informal reasoning among
children of different grade, ability, and knowledge levels. Cognition and Instruction, 14(2), 139–
178.
McDevitt, M. J., & Caton-Rosser, M. S. (2009). Deliberative barbarians: Reconciling the civic
and the agonistic in democratic education. InterActions: UCLA Journal of Education and
Information Studies, 5(2).
Milstein, T. (2005). Transformation abroad: Sojourning and the perceived enhancement of self-efficacy.
International Journal of Intercultural Relations, 29(2), 217-238.
National Governors Association Center for Best Practices. (2010). Common Core State
Standards (English language arts and literacy in history/social studies, science, and
technical subjects). Washington D.C.: National Governors Association Center for Best
Practices, Council of Chief State.
Newmann, F. M. (1988). Higher order thinking in high school social studies: An analysis of
classrooms, teachers, students, and leadership. Madison, WI: University of Wisconsin,
National Center on Effective Secondary Schools.
Nussbaum, M. E., & Bendixen, L. D. (2003). Approaching and avoiding arguments: The role of
epistemological beliefs, need for cognition, and extraverted personality traits.
Contemporary Educational Psychology, 28(4), 573-595.
Nystrand, M., Gamoran, A., & Carbonaro, W. (1998). Towards an ecology of learning: The case
of classroom discourse and its effects on writing in high school English and social
studies. University at Albany: Center on English Learning & Achievement.
Powers, W. T. (1991). Commentary on Bandura's" human agency.". American Psychology, 46,
152-153.
Sadler, T. D., Chambers, F. W., & Zeidler, D. L. (2004). Student conceptualizations of the nature of
science in response to a socioscientific issue. International Journal of Science Education, 26(4),
387-409.
Steiner, S., Brzuzy, S., Gerdes, K., & Hurdle, D. (2003). Using structured controversy to teach diversity
content and cultural competence. Journal of Teaching in Social Work, 23(1-2), 55-71.
Stice, E., Shaw, H., & Marti, C. N. (2006). A meta-analytic review of obesity prevention
programs for children and adolescents: The skinny on interventions that work.
Psychological Bulletin, 132(5), 667.
Swanson, K. (2013). Teaching the common core speaking and listening standards: Strategies and
digital tools. United Kingdom: Routledge.
Tankova, T., Dakovska, G., & Koev, D. (2004). Education and quality of life in diabetic patients.
Patient education and counseling, 53(3), 285-290.
Terenzini, P. T., Springer, L., Pascarella, E. T., & Nora, A. (1995). Influences affecting the
development of students’ critical thinking skills. Research in Higher Education, 36, 23–
39.
Torney-Purta, J., Lehmann, R., Oswald, H., & Schulz, W. (2001). Citizenship and education in
twenty-eight countries: Civic knowledge and engagement at age fourteen. Amsterdam:
International Association for the Evaluation of Educational Achievement.
Tsui, L. (1999). Courses and instruction affecting critical thinking. Research in Higher
Education, 40(2), 185-200.
Vancouver, J. B., Thompson, C. M., & Williams, A. A. (2001). The changing signs in the
relationships among self-efficacy, personal goals, and performance. Journal of Applied
Psychology, 86(4), 605-620.
Venville, G. J., & Dawson, V. M. (2010). The impact of a classroom intervention on grade 10
students' argumentation skills, informal reasoning, and conceptual understanding of
science. Journal of Research in Science Teaching, 47(8), 952-977.
Verba, S., Schlozman, K. L., & Brady, H. E. (1995). Voice and equality: Civic voluntarism in
American politics. Harvard: Harvard University Press.
Vercellotti, T., & Matto, E. (2013). The role of media use in the classroom and at home in
improving political knowledge. Presented at the 2013 APSA Teaching and Learning
Conference, Long Beach, CA.
von Aufschnaiter, C., Erduran, S., Osborne, J., & Simon, S. (2008). Arguing to learn and learning
to argue: Case studies of how students' argumentation relates to their scientific
knowledge. Journal of Research in Science Teaching, 45(1), 101-131.
Wells, J., & Lewis, L. (2006). Internet Access in US Public Schools and Classrooms: 1994-2005.
Highlights. NCES 2007-020. National Center for Education Statistics.
Wray‐Lake, L., & Syvertsen, A. K. (2011). The developmental roots of social responsibility in
childhood and adolescence. New Directions for Child and Adolescent Development,
2011(134), 11-25.
Zorn, T. E., Roper, J., Broadfoot, K., & Kay Weaver, C. (2006). Focus groups as sites of
influential interaction: Building communicative self-efficacy and effecting attitudinal
change in discussing controversial topics. Journal of Applied Communication Research,
34(2), 115-140.
Zorn, T. E., Roper, J., Weaver, C. K., & Rigby, C. (2012). Influence in science dialogue:
Individual attitude changes as a result of dialogue between laypersons and scientists.
Public Understanding of Science, 21(7), 848-864.
TABLES
Table 1. Grade Level Contributions by Schools Organized by Treatment Condition
6th Graders
7th Graders
8th Graders
All Grades
Schools
n
% of
school
n
% of
school
n
% of
school
N
% of sample
by condition
155
56
76
256
-107
650
43.9%
48.3%
17.7%
36.9%
-31.3%
27.6%
59
21
117
212
214
111
734
16.7%
18.1%
27.3%
30.5%
51.2%
32.5%
31.2%
139
39
236
226
204
124
968
39.4%
33.6%
55.0%
32.6%
48.8%
36.3%
41.2%
353
116
429
694
418
342
2,352
15.0%
4.9%
18.2%
29.5%
17.8%
14.5%
40.1%
Word Generation Schools
Apple
Arbola
Flower Square
Hemlock
Moon
Palm
Total
371
279
118
62
87
281
1,198
35.0%
36.4%
40.8%
36.9%
25.0%
31.7%
34.1%
352
239
72
61
135
323
1,182
33.2%
31.2%
24.9%
36.3%
38.8%
36.5%
33.6%
338
248
99
45
126
282
1,138
31.9%
32.4%
34.3%
26.8%
36.2%
31.8%
32.3%
1,061
766
289
168
348
886
3,518
30.2%
21.8%
8.2%
4.8%
9.9%
25.2%
59.9%
All Schools
1,848 31.5%
2,106 35.9%
5,870
100.0%
Control Schools
Duffie Oak
Evergreen
Honeysuckle
Maple
Rosemary
Vineland
Total
1,916 32.6%
Note. Analytic sample based on students who responded to at least 3 out of 15 survey items.
Table 2. Demographics of the Analytical Sample by Treatment Condition
Control
Schools
Female
Special Education
Free and Reduced Lunch Eligible
English Language Learner
Asian
African-American
Hispanic
White
Other
Total Students
Word
Generation
Schools
n
%
n
%
968
175
1,344
1,219
1,115
146
397
110
60
2,352
50.0%
9.0%
69.5%
63.0%
57.6%
7.6%
20.5%
5.6%
3.1%
40.1%
1,595
261
1,944
1,710
1,673
209
597
332
160
3,518
49.5%
8.1%
60.4%
53.1%
51.9%
6.5%
18.5%
10.3%
4.9%
59.9%
Note. Analytic sample based on students who responded to at least 3 out of 15 survey items.
All
Schools
N
%
2,563 49.7%
436
8.5%
3,288 63.8%
2,929 56.8%
2,788 54.1%
355
6.9%
994
19.3%
442
8.6%
220
4.3%
5,870 100.0%
Table 3. Overview of the Control, Treatment, and Follow-Up Topics
Sixth Graders
Seventh Graders
Eighth Graders
WG
Control
# of
Schools Schools Students
Instructed Instructed Tested
WG
Control
# of
Schools Schools Students
Instructed Instructed Tested
WG
Control
# of
Schools Schools Students
Instructed Instructed Tested
Treatment Topics
Should athletes be allowed to use steroids?
Should there be more strict dress codes at schools?
Is the death penalty justified?
Does rap music have a negative impact on students?
What should be done about global warming?
Should the government allow animal testing?
Y
Y
Y
Y
Y
Y
N
N
N
N
N
N
1,825
1,842
1,825
1,827
1,811
1,836
Y
Y
Y
Y
Y
Y
N
N
N
N
N
N
1,901
1,907
1,904
1,907
1,892
1,906
Y
Y
Y
Y
Y
Y
N
N
N
N
N
N
2,085
2,093
2,082
2,085
2,073
2,085
Follow-Up Topics
Should colleges use Affirmative Action?
Should the government fund stem cell research?
Should Creation be taught in schools?
Should English be the official language of the U.S.?
Is nuclear power a danger to society
N
N
N
N
N
N
N
N
N
N
1,820
1,822
1,833
1,819
1,825
Y
Y
Y
Y
Y
N
N
N
N
N
1,902
1,912
1,906
1,907
1,901
Y
Y
Y
Y
Y
N
N
N
N
N
2,081
2,091
2,092
2,087
2,079
Control Topics
Should schools protect students from cyber-bullying?
Should schools be a place for debates?
Should secret wiretapping be legal?
Should schools have a vocational track?
N
N
N
N
N
N
N
N
1,837
1,815
1,810
1,837
N
N
N
N
N
N
N
N
1,909
1,899
1,895
1,914
N
N
N
N
N
N
N
N
2,086
2,078
2,063
2,098
Note. WG = Word Generation; Y= Yes, N = No.
Table 4. Means and Standard Deviations of Students' Communicative Self-Efficacy to Participate in Discussions of Control,
Treatment, and Follow-Up Topics with Estimated Effect Size
Control
Schools
Topic Questions
Word Generation
Schools
All
Schools
t-test
Effect Size
(Cohen's d)
n
M
SD
n
M
SD
N
M
SD
Difference
Sig
Research Question # 1
Should schools protect students from cyberbullying?
Should schools be a place for debates?
Should secret wiretapping be legal?
Should schools have a vocational track?
Average Score of Control Topics
2,344
2,334
2,322
2,314
2,332
3.92
2.57
2.36
2.83
2.92
1.22
1.21
1.30
1.21
0.79
3,505
3,498
3,470
3,454
3,489
3.92
2.64
2.36
2.75
2.92
1.21
1.21
1.27
1.19
0.78
5,849
5,832
5,792
5,768
5,821
3.92
2.61
2.36
2.78
2.92
1.21
1.21
1.28
1.20
0.79
0.00
+ 0.08
0.00
- 0.08
0.00
**
ns
Research Question # 2
Should athletes be allowed to use steroids?
Should there be more strict dress codes at schools?
Is the death penalty justified?
Does rap music have a negative impact on students?
What should be done about global warming?
Should the government allow animal testing?
Average Score of Treatment Topics
2,327
2,343
2,323
2,328
2,315
2,332
2,324
1.99
2.02
2.65
2.37
3.70
2.30
2.51
1.28
1.23
1.29
1.29
1.27
1.31
0.75
3,484
3,499
3,488
3,491
3,461
3,495
3,492
2.07
2.16
2.74
2.50
3.75
2.43
2.61
1.31
1.29
1.30
1.29
1.24
1.34
0.82
5,811
5,842
5,811
5,819
5,776
5,827
5,816
2.04
2.11
2.71
2.45
3.73
2.38
2.57
1.30
1.27
1.30
1.29
1.25
1.32
0.80
+ 0.08
+ 0.14
+ 0.09
+ 0.13
+ 0.05
+ 0.13
+ 0.10
Research Question # 3
Should colleges use Affirmative Action?
Should the government fund stem cell research?
Should Creation be taught in school?
Should English be the official language of the U.S.?
Is nuclear power a danger to society?
Average Score of Follow-Up Topics
1,686
1,693
1,687
1,692
1,682
1,690
2.70
2.80
3.14
3.15
3.18
2.99
1.07
1.11
1.17
1.37
1.36
0.76
2,301
2,313
2,307
2,308
2,298
2,306
2.71
2.98
3.02
3.03
3.21
2.99
1.04
1.15
1.21
1.36
1.28
0.77
3,987
4,006
3,994
4,000
3,980
3,996
2.70
2.90
3.07
3.08
3.20
2.99
1.05
1.14
1.20
1.36
1.31
0.77
+ 0.01
+ 0.18
- 0.12
- 0.12
+ 0.03
0.00
***
***
***
***
***
+ 0.13
***
***
**
ns
Note. Students were given the prompt "How confident are you in being able to participate in a discussion about the following topics?" and followed by a series of controversial
topics. Answers based on a 5-point Likert scale of (1) "not at all" to (5) "extremely" confident in being able to participate in such discussion; Analytic sample based on students
who responded to at least 3 out of 15 survey items; Sample used to analyze Follow-Up Topics restricted to only seventh and eighth graders; Averages for Control, Treatment and
Follow-Up Topics only includes students who responded to at least 75% of the corresponding items; ** p < 0.01, *** p < 0.001.
33
APPENDIX
Survey Items Assessing Students' Communicative Self-Efficacy in Discussions of Controversial Issues
How confident are you in being able to participate in a discussion about the
following topics?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Should colleges use Affirmative Action?
Should the government fund stem cell research?
Should Creation be taught in schools?
Should schools protect students from cyber-bullying?
Should English be the official language of the U.S.?
Should there be more strict dress codes at schools?
Should schools be a place for debates?
Should secret wiretapping be legal?
Is the death penalty justified?
Should athletes be allowed to use steroids?
Does rap music have a negative impact on students?
Is nuclear power a danger to society?
What should be done about global warming?
Should schools have a vocational track?
Should the government allow animal testing?
Not at all
A Little
Somewhat
Very
Extremely
(1)
(2)
(3)
(4)
(5)