Academic vocabulary instruction across the content areas: Results from a
randomized trial of the Word Generation Program
Joshua Lawrence, Alex Lin & Anny Hwang
The Word Generation program (www. wordgeneration.org) is a freely available vocabulary
intervention developed the Strategic Educational Research Partnership. Word Generation
is a 24-week-long sequence of topics of current interest, each associated with five allpurpose academic words, and prescribed activities related to math, science, and social
studies. The basic sequence of Word Generation activities is as follows: On Monday a brief
text in which the five target words were embedded is read by the students and teacher
together, then discussed using guiding comprehension questions; this text presents
arguments on both sides of some engaging controversy or dilemma. Then the five target
words are highlighted and provided with student-friendly context-related definitions. This
activity typically occurs in the English Language Arts classroom. On Tuesday, Wednesday,
and Thursday, in an order determined by each school, the math, social studies, and science
teachers implement activities provided for them, each of which embed the same five target
words. The math teacher assigns one or two problems related in content to the dilemma of
the week; the format of these problems is modelled on the state math assessment. Math
teachers then discuss the content as well as the math procedures. The science teacher
presents a new text that focusses on science content related to the dilemma of the week;
students fill in target words left blank in the text, before the class discusses the text. The
social studies teacher organizes a debate about the dilemma of the week in one of several
possible formats (fishbowl, pairs, whole class, four corners, etc.). On Friday, the students
are asked to write a ‘taking a stand’ essay about the dilemma. A previous quasiexperimental study of the program in Boston Public School found that language-minority
students benefitted from program participation more than English-only students did
(Authors, 2009) and that gains made from program participation were sustained even a
year after instruction (Authors, in press).
This is the first presentation of data from an IES funded cluster randomized trial of Word
Generation.
An efficacy trial of Word Generation: Results from the first year of a randomized trial
Our purpose was to understand how well students participating in a vocabulary program
learned the target high leverage academic vocabulary compared to students in control
schools. Data were collected in two large urban districts in the Northeast. Twenty-eight
(28) schools participated in the randomized trial. Students (n = 943) participated in the
Word Generation program across 15 schools; the sample of students in the comparison
schools was somewhat smaller (N = 640).
We measured student vocabulary knowledge with a multiple choice assessment. For each
set of target words, we have devised a multiple choice test of knowledge of the words’
meanings in relatively neutral contexts. Student performance on these assessments suggest
acceptable levels of reliability (α = 0.87) and good distribution of incorrect answers over
the various distracters.
The general approach to our analyses was to employ random assignment of experimental units,
in this case schools, to conditions and then to employ multi-level models to address problems
of clustering that occur in instructional settings and complicate intervention research
(Raudenbush, Martinez, & Spybrook, 2007). In our analysis pretest served as a covariate at the
student and school level and the model specified that students are nested within schools.
Treatment was analyzed as a school-level predictor.
We found that students who participated in the Word Generation Program improved in
their knowledge of general academic words more than comparison students did (β = 1.16 p
<0.05). Furthermore, we found that these improvements were more likely to be realized in
struggling urban schools with larger numbers of less-proficient readers.
Developing youth civic engagement though discussion of controversial issues
Informed citizens need to develop civic engagement, which refers to a comprehensive set of
attitudes, knowledge, and participation skills necessary to participate in a democratic
society (Janmaat, 2011). Previous studies from international data on the political behavior
of 14-year-old students across 28 countries from Europe, North America, Asia, and Pacific
found that students’ perceptions about classroom-based opportunities to discuss political
issues in Civics classes were positively related to their civic knowledge (Torney-Purta
2002; Torney-Purta and Richardson, 2005). Not much is known about intervention
programs that actively engage students in discussion-based opportunities across all their
subject areas.
Students participating in the Word Generation program receive instructional material that
features opposing arguments about a controversial issue each week. Teachers in all
subjects engaged in discussion to support students’ development of reasoning and ability
to express their positions. Pilot surveys and cognitive interviews were conducted to assess
students’ experience of this aspect of the program. Students completed 30 survey questions
based on the three dimensions of civic engagement (knowledge, attitudes, and
participation) including their interest in voting, discussing political issues at home and
volunteering. Preliminary results from individual interviews reveal that the literacy
program had a positive impact on students’ civic engagement, including their interest in
helping the community and participating in discussions regarding controversial topics. This
spring, the civic engagement surveys will be distributed to all student students in one of
our participating districts (n > 6,000). We will use these data to understand the
relationship between students’ exposure to classroom-based discussion of controversial
issues and their civic engagement. The implications for this study relate to understanding
how discussion-based opportunities may be pivotal to developing students’ civic
engagement, while preparing them to become informed and active citizens.
Investigating the Role of Academic Vocabulary in Reading Comprehension for
Redesignated Fluent English Proficient Middle School Students
Redesignated Fluent English Proficient (RFEP) is a category that one of our partner
districts uses to identify students who were initially identified as English language learners
but no longer receive additional support in English. Most studies that have attempted to
understand language status and response to vocabulary instruction have not paid much
attention to difference within the RFEP category although it is a heterogeneous group.
Using detailed retrospective data from 2042 students collected from year 2 of our study,
we created precise categories of language proficiency based on the date of redesignation
and used these categories to understand the relative importance of academic word
knowledge for these groups and their responsiveness to program participation.
Preliminary analyses revealed three findings. First, the relation between vocabulary
knowledge and reading comprehension was different for the RFEP groups. For those
redesignated earlier, general and academic vocabulary equally predicted reading
comprehension (β = 0.55, p <.001 and β = 0.54, p <.001 for general and academic
vocabulary, respectively). However, for those redesignated recently, academic vocabulary
knowledge was more predictive of reading comprehension (β = 0.32, p <.001 and β = 0.47, p
<.001 for general and academic vocabulary, respectively). Second, the groups had different
academic vocabulary knowledge at the start of the program; the earlier redesignated
students knew more academic vocabulary words compared to the more recently
redesignated groups. Third, there was an overall effect of Word Generation on RFEP
students such that all RFEPs benefited. Preliminary analysis suggest that although there is
no main effect of treatment on comprehension scores, there is an effect moderate by
improvements in academic word knowledge across RFEP groups.
References
Authors. (in press). Language proficiency, home-language status, and English vocabulary
development: A longitudinal follow-up of the Word Generation program.
Bilingualism: Language and Cognition.
Authors. (2009). Generating knowledge of academic language among urban middle school
students. Journal of Research on Educational Effectiveness, 2(4), 325-344.
Janmaat, J.G. (2011). Ability grouping, segregation and civic competences among
adolescents. International Sociology, 26 (4), 455-82.
Raudenbush, S., Martinez, A., & Spybrook, J. (2007). Strategies for improving precision in
group-randomized experiments. Educational Evaluation and Policy Analysis, 29(1), 5.
Torney-Purta, J. (2002). The school’s role in developing civic engagement: A study of
adolescents in twenty-eight countries. Applied Developmental Science, 6 (4), 203-12.
Torney-Purta, J. & Richardson, W.K. (2005). Anticipated political engagement among
adolescents in Australia, England, Norway and the United States. In J. Demaine (Ed.).
Citizenship and political education today. New York: Palgrave Macmillan.