Does the Use of Technological Interventions
Improve Student Academic Achievement in
Mathematics and Language Arts for an
Identified Group of At-risk Middle School
Students?
Mark Neill
Jerry Mathews
Abstract: Computer-assisted instructional programs offer another avenue of support for student
achievement. This study was conducted to investigate the influence of two computer-assisted instructional
programs on the math and language arts academic achievement of two groups of academic at-risk middle
school students compared to a group of students who were not at-risk in a traditional instructional program.
This study utilized correlation statistics to analyze student achievement by these subjects as measured by
Rausch Unit (RIT) scores on the state mandated assessment. The results of the study indicated only a small
gap in academic achievement between the at-risk students receiving computer-assisted learning
interventions compared to those students engaged in the traditional instructional strategies. There was a
22% increase in the number of students who met or exceeded the state-mandated growth targets in
language arts and math after the first year of the computer-assisted learning interventions.
About the authors: Dr. Mark Neill is an assistant professor at Idaho State University. He is coordinator
of the Educational Leadership program. Dr. Jerry Mathews is an associate professor at Mississippi State
University. He is the graduate coordinator and program coordinator of Educational Administration in
the department of Leadership and Foundations.
Introduction
The implications established by
the provisions of the No Child Left
Behind (NCLB) legislation has left
many school districts searching for
avenues to address the specific learning
needs of select groups of learners.
Several student groups are typically
identified as at-risk of failing to meet the
proficiency requirements of state
mandated tests. These groups usually
include English as a Second Language
(ESL), special needs, or economically
disadvantaged students.
An institutional goal of a middle
school in a western state was to improve
student performance in two areas on this
western state’s achievement test
(WSAT). The WSAT is produced and
managed by the Northwest Evaluation
Association (NWEA, 2005) and is
utilized, in part, to assess the Adequate
Yearly Progress (AYP) requirements of
NCLB. This research project analyzed
Rausch Units (RIT) subscale score
academic data of 7th and 8th grade
students from three testing periods in
language arts and math. The data were
analyzed and reported primarily using
correlation measures.
Theoretical Framework: Research
Question
The proliferation of computer
technology in American schools has
resulted in a need for more rigorous
research into the effectiveness of this
technology in support of instruction and
student achievement. Research
correlating teachers’ technology skills
and their use of technology in classroom
instruction with higher academic
achievement has been conducted
(Attewell, & Battle, 1999; Mann,
Shakeshaft, Becker, & Kottkampk,
1999; Wenglinsky, 1998). Meaningful
data are needed to inform decision
makers regarding the capacity of
technological interventions to increase
student achievement and guide
instructional practice.
In an effort to better assess what
technology does, rather than what is, this
study focused on the impact of
technological interventions on
improving student achievement.
Specifically, this research assessed the
impact of selected educational
technological interventions on student
achievement in middle school language
arts and mathematics. The key research
question guiding this study was: Does
the use of technological interventions
improve student achievement in
mathematics and language arts for an
identified group of at-risk students?
The theoretical framework
supporting the design of this study was
based on one research question. The
research question was: “Is there a
statistically significant correlation in the
relationship between the RIT subscales
in language arts and math scores of
middle school students and the predictor
variables: (a) grade level [7th and 8th], (b)
instructional intervention [1 = Bridges, 2
= Fast ForWord, 3 = traditional
instruction], (c) economic status
[economically disadvantaged and noneconomically disadvantaged], (d)
gender, and (e) ethnicity [white and non-
white]. Even though technological
interventions were the focus of this
study, four other predictor variables
were included in the statistical analysis.
Methodology
Research Design
A causal-comparative research
design was used in this predictive study.
The independent variables were not
manipulated by the researchers. In the
typical public school environment
learning interventions are not
purposefully withheld from student
participants. Therefore, this study did
not utilize a true experimental design
with random selection and assignment of
participants. Rather, this correlation
study provided results that permitted the
researchers to determine the strength and
direction of the relationship between
different sets of data or to predict scores
on one variable based on the knowledge
of scores on another (Gall, Gall, & Borg,
2007). In order to address the research
question, the researchers employed a
multiple linear regression analysis
(MLR), which according to Hair, Black,
Babin, Anderson, and Tatham (2005) is
an appropriate analysis when multiple
independent variables are used for the
purpose of predicting variation in a
single dependent variable. Correlations
were determined using RIT mean scale
scores of the Western State Achievement
Test (WSAT). In order to determine
statistically significant differences in
student achievement, MLR analysis was
used to calculate independent t-tests and
analysis of variance (ANOVA) to
compare the mean scores of the 7th and
8th grade students.
Data Source
This research project included
measures of scale scores in RIT
language arts and math of 7th grade
and 8th grade students from testing
scheduled during the fall 2004,
winter 2005, and spring 2005 WSAT
testing periods. The WSAT was
produced and managed by the
Northwest Evaluation Association
(NWEA, 2005) and was utilized, in
part, to meet the federal requirements
for reporting the states’ standardized
student achievement data to the U.S.
Department of Education.
Three intact groups of students
were included in the study. Two
groups were identified as
academically at-risk students who
participated in two different
technological learning interventions.
The third intact group was students
participating in a traditionally
delivered instructional strategy. See
Table 1 for the definitions of the
variables used in the study.
Table 1
Summary of Variables in Study
Variables/Levels
Description
Data Obtained From
WSAT math RIT
Scores
Mean scores of the Western State Achievement
Test
Western State Department
of Education
WSAT language arts
RIT scores
Mean Growth Target
scores
Mean scores of the Western State Achievement
Test
Pre-set mean scores that indicate successfully
passing the WSAT subscales
Western State Department
of Education
Western State Department
of Education
7th Grade Level
Students enrolled in the 7th Grade who had math
and language achievement test scores
Western State City Middle
School
8th Grade Level
Students enrolled in the 8th Grade who had math
and language achievement test scores
Western State City Middle
School
Bridges Instruction
Bridges' online Learning Styles Inventory (LSI)
developed by the Bridges Transitions Company
Fast ForWord
Instruction
Educational products were developed by
Scientific Learning, Inc. designed to promote
academic learning
Western State City Middle
School
Western State City Middle
School
Traditional academic program, delivered through
regular class and coursework, exclusive of the
specified technological interventions.
Western State City Middle
School
Economic Status
Students defined as economically disadvantaged
or who were listed as qualifying for free or
reduced lunches. Students defined as noneconomically disadvantaged were listed as
students who did not qualify for free or reduced
lunches.
Western State City Middle
School
Gender
Male and female students
Western State City Middle
School
Traditional Instruction
Ethnicity
Students identified as minority (AfricanAmerican, Hispanic and Native American.)
Students identified as non-minority (students who
were not African-American, Hispanic and Native
American)
Participants and Data Collection
Procedures
Rausch Unit Scale Scores
Students in the school who did
not meet the individual mean growth
target were identified as at-risk for the
purpose of this study and were selected
to participate in the Fast ForWord and
Bridges computer assisted interventions.
Mean Growth Target is defined as the
average amount of the Rausch Unit
(RIT) growth observed for students in
the latest Northwest Evaluation
Association (NWEA, 2005) normed
study. The RIT scale can be compared,
in theory, to a meter stick which is made
up of equal units of measure, for
example, centimeters. RIT scores are
considered to be reliable and accurate
indicators of achievement growth over
time. Because these units do not change,
the RIT score can be used with
confidence to compare a student’s
academic growth from one year to the
next. Therefore, since the units of the
RIT scale are equal in value, reliable
comparisons and conclusions can be
made about the academic growth of a
child or a group of children (NWEA,
2005).
RIT Scores Are Grade Independent
Since WSAT tests are adaptive
and the test items are based on student
performance, not age or grade, identical
scores across grades mean the same
thing. For example, a seventh grader
who received a score of 210 and an
eighth grader who received a score of
210 are learning at the same level. This
Western State City Middle
School
allows growth to be measured
independent of grade level (NWEA,
2005). A primary objective of the
research study was to determine the
impact of technological interventions on
the academic achievement of middle
school students as determined by their
RIT scores on state mandated
assessments.
As a result of their previous
performance on state mandated
assessments, approximately 100 seventh
and eighth grade students who failed to
meet the Mean Growth Target on the
WSAT RIT language arts and math
subscales were identified as at-risk.
These students participated in the Fast
ForWord and Bridges technology
program.
FastForWord Learning Intervention
Program
A group of at-risk students were
selected to participate in the
FastForWord learning intervention
program. Fast ForWord products were
developed by Scientific Learning, Inc. to
strengthen the cognitive skills of brainbased learning maps including, (a)
memory, (b) attention, (c) processing,
and (d) sequencing. According to the
developers, Fast ForWord products are
designed to promote academic learning
success. Scientific Learning, Inc. (2004)
promotes the concept that strengthening
these skills results in improved critical
language and reading.
Bridges Learning Intervention Program
This research project also
included a different group of at-risk
students who participated in a second
technological intervention called
Bridges. This program intended to
provide assistance to students in the
development and improvement of study
skills and academic dispositions.
Bridges' online Learning Styles
Inventory (LSI) improves study habits,
attitudes and behavior, motivation, and
helps students get on a successful
academic track (Bridges Transition
Company, 2004). The LSI is used in the
education community to diagnose
middle school students’ unique learning
styles based on an analysis of their
personal preferences.
Traditional Academic Program
A third group of students who
met or exceeded the Mean Growth
Target scores and were not identified as
at-risk students, participated in the
traditional academic program as defined
by the state standards in math and
language arts. The traditional academic
program was delivered through regular
class and coursework, exclusive of the
specified technological interventions.
Approximately 400 students participated
in the traditional academic program.
Data Analysis
Several steps were taken with the
data a priori to help ensure the integrity
of the study. Data were adequately
inspected for any missing values with no
problems noted. The following
assumptions, as recommended by Hair et
al. (2005), were checked: linearity of the
phenomena measured, constant variance
of the error terms, independence of the
error terms, collinearity, and normality
of the error term distribution.
Collinearity, according to Hair et al.
(2005), refers to the fact that a predictor
variable is highly correlated with another
predictor variable. The variance inflation
factor (VIF) was examined to ensure
correlation models did not exceed a VIF
value of 10.
Reported as Pearson’s R2,
correlation coefficients were produced
using the simultaneous solution multiple
linear regression (MLR) analysis
procedure. A simultaneous solution was
used because the researchers wanted to
predict the dependent variable based on
all independent variables of interest.
According to Hair et al. (2005), MLR
allows researchers to determine a
correlation between a dependent variable
and the best linear combination of two or
more predictor or independent variables.
The correlation coefficients indicate the
strength of the correlation. An F statistic
from an ANOVA tests the significance
of the R2. The threshold for determining
significance was set a priori at an alpha
level of 0.05. Beta weights of the
standardized coefficients were examined
to determine the unique importance of
independent variables in the model.
A priori assumptions of
normality and homogeneity for the
ANOVA were examined and satisfied
(Hair et al., 2005). According to
Fraenkel and Wallen (2006), when
comparing only two groups, the F
statistic is satisfactory to reveal whether
a level of significance exists between the
comparisons. An alpha level of 0.05 was
set a priori for the ANOVA. The MLR
analysis, ANOVA, and descriptive
statistics were run using the Statistical
Package for Social Sciences 12.0®
(Norusis, 2003).
Results
The population of middle school
students was examined for the testing
periods fall 2004, winter 2005, and
spring 2005. Descriptive statistics for the
dependent variables of WSAT RIT
language arts and math subscale test
scores by intervention are shown in
Table 2. The population of testing
subjects changed from one testing cycle
to another, due to subject migration (e.g.,
Hispanic subjects). Student enrollment
fluctuated during the agricultural harvest
Table 2
cycle in the western state. Therefore,
subject participation differed in each of
the testing cycles with the highest
enrollment occurring during the peak
harvest time in the fall, dropping
dramatically during the winter testing
period, and increasing again during the
spring planting season.
Descriptive Statistics of RIT Scores by Intervention
Intervention
Bridges
Fast ForWord
Traditional
Subject
Language
Math
Language
Math
Language
Math
Fall 2004
Winter 2005
Spring 2005
n
M
SD
n
M
SD
n
M
SD
79
212.10
10.88
63
212.46
8.18
61
215.62
8.86
79
218.87
13.16
63
221.74
11.52
61
225.67
11.09
23
213.17
6.96
50
202.90
9.57
51
205.67
10.53
23
224.52
11.06
50
209.62
12.21
51
215.61
10.40
414
220.21
10.57
273
211.99
8.49
379
225.80
10.67
414
230.23
14.97
273
225.08
11.76
379
239.86
12.42
Even though the traditional
instruction groups had higher mean
scores than the at-risk Bridges and
FastForWord groups in each set of
scores, it is noted that small achievement
gaps existed between Bridges,
FastForWord, and traditional learning
groups.
An MLR analysis was conducted
to determine which of the computer
assisted interventions by grade level
accounted for a statistically significant
amount of the variation in the dependent
variable: WSAT language arts and math
subscale test scores for each of the three
testing periods. The analysis reported Rsquared values of the variation in student
achievement in language arts and math
on the basis of the predictor variables
(see Tables 3 & 4).
RIT Language Arts Subscale Analysis
The ANOVA results at an alpha
level of .05 were statistically significant
for each of the testing periods. The
researchers rejected the null hypothesis
due to the fact that predictions on the
RIT language arts subscale test scores
can be made on a better than chance
level when the predictor variables are
simultaneously entered into the model.
The Pearson’s R indicated moderate
correlations between the dependent
variable and the best linear combination
of the predictor variables (see Table 3).
The t-tests indicated if any of the
Beta coefficients of the WSAT RIT
language arts subscale were statistically
significant. As shown in the Table 3 the
predictors, gender, learning
interventions, and economic status were
statistically significant predictors of
student achievement in the fall testing
period. The predictor variables gender,
grade level and interventions were
statistically significant in the winter
2005 testing period. Gender, grade level,
intervention, and economic status were
statistically significant in the spring
2005 testing period (see Table 3).
Table 3
Summary of Multiple Linear Regression Model Analysis for the Predictors
of Student Achievement for RIT Language Arts
Predictor
β
variable
Constant
.130
Gender
.054
Grade level
.302
Intervention
-0.063
Ethnicity
Economic status -0.210
R-squared value
Adjusted
R-square
Fall 2004
t
27.37
3.17
1.31
7.31
-1.53
-5.06
p
Winter 2005
β
t
p
.000***
.002*
.190
.000***
.127
.000***
23.81
.119
.128
.200
-0.019
-0.106
Spring 2005
β
t
p
.000***
2.01
2.19
.045*
.029*
3.38
-0.318
-1.81
.001**
.750
.072
27.63
.122
.111
3.15
2.86
.434
-0.066
-0.170
11.12
-1.70
-4.36
.293
.084
.276
.082
.067
.269
000***
.002*
.004*
.000***
.091
.000***
*p<.05, **p<.01, ***p<.001
RIT Math Subscale Analysis
The ANOVA results at an alpha
level of .05 were statistically significant
for each of the testing periods. The
researchers rejected the null hypothesis
due to the fact that predictions on the
RIT math subscale test scores can be
made on a better than chance level when
the predictor variables are
simultaneously entered into the model.
The Pearson’s R indicated moderate
correlations between the dependent
variable and the best linear combination
of the predictor variables (see Table 4).
The t-tests indicated if any of the
Beta coefficients of the WSAT RIT math
subscale were statistically significant. As
shown in the Table 4 the predictor
variables gender, learning interventions,
and economic status were statistically
significant predictors of student
achievement in the fall testing period.
The predictor variables gender, grade
level, and interventions were statistically
significant in the winter 2005 testing
period. Gender, grade level, intervention,
and economic status were statistically
significant in the spring 2005 testing
period (see Table 4).
Table 4
Summary of Multiple Linear Regression Model Analysis for the
Predictors of Student Achievement for RIT Math
Predictor
variable
Constant
Gender
Grade level
Intervention
Ethnicity
Economic status
R-squared value
Adjusted
R-square
β
-0.084
-0.084
.307
-0.080
-0.214
Fall 2004
t
p
17.99
.000***
-2.11
5.32
7.62
-2.01
-5.80
.035*
.000***
.000***
.045*
.000***
.194
.186
Winter 2005
β
t
p
-0.004
-0.183
.226
.021
-0.139
14.03
.000***
-0.070
.290
3.52
.233
-2.19
.970
.004*
.001***
.739
.029*
.103
.083
Spring 2005
β
t
p
-0.128
.229
.439
-0.054
-0.164
14.03
.000***
-0.336
6.02
11.47
-0.142
-4.30
.001*
.000***
.000***
.156
.000***
.305
.298
*p<.05, **p<.01, ***p<.001
Conclusions and Implications
Students participating in the Fast
ForWord and Bridges instructional
programs were selected based on their
identification as academically at-risk
students. Historically, students identified
as academically at-risk are more likely to
achieve at a much lower level than
regular education students who are not
identified as academically at-risk. Three
key influences, Bridges, FastForWord,
and traditional instruction, were
analyzed in this study using descriptive
statistics and multiple linear regression.
It is of noteworthy practical significance
in the achievement data that the
academically at-risk students involved in
the Fast ForWord and Bridges
interventions performed at a level that
had only a small achievement gap (on
average) lower than the achievement of
the regular education students not
identified as academically at-risk (who
participated in traditional instructional
interventions in this study). This study
supports the conclusions reported by
Mann et al. (1999) that it was the at-risk
students who realized the greatest
achievement gains.
The conclusion for this study
affirmed the key research question. Data
indicated that there was an increase in
the number of student’s meeting growth
targets on the state mandated
assessment. The overall school-wide
student achievement, as measured by
WSAT scores, increased by 22%
following the first year of
implementation of the Fast ForWord
and Bridges interventions [reported by
the middle school administration]. This
The academically at-risk students who
participated in the technology
intervention programs did increase their
average academic achievement above
the Mean Growth Targets on WSAT
language arts and math subscales
achievement of the prior year. The
overall increase in achievement for the
middle school indicated a possible cause
and effect of the technology intervention
on language arts and math achievement.
In addition, the teachers and
administrators at the middle school
attributed the success of the computer
assisted learning interventions as a
diagnostic tool, and used this
information to improve student
achievement for each of the testing
periods.
Implications for Future Study
A planned future focus of the
analysis of data collected in this project
will include the effect of gender, socio-
economic status, and ethnicity on middle
school student language arts and math
achievement. The same groups of at-risk
students will be included as participants
in this analysis. This future study will be
conducted to link at-risk student
participation in the Bridges and Fast
ForWord intervention strategies with
language arts and math achievement.
References
Attewell, P., & Battle, J. (1999). Home
computers and school performance.
Information Society. 15(1), 1-10.
http://ehostvgw11.epnet.com/fulltext
.asp?resultSetId=R00000000&hitNu
m=72&
booleanTerm=is%2001972243&fuzz
yTerm=
Bridges Transitions Company (2004).
Learning styles inventory. Oroville,
WA: Author.
Fraenkel, J. & Wallen, N. (2006). How to
design and evaluate research in
education (6th ed.). McGraw-Hill,
New York.
Gall, M. D., Gall, J. P., & Borg, W. R.
(2007). Educational research: An
introduction (8thed.). Boston, MA:
Allyn and Bacon.
Hair, J. F., Black, B., Babin, B, Anderson,
R. E., & Tatham. R. L. (2005).
Multivariate data analysis (6th ed.).
Upper Saddle River, NJ: PrenticeHall, Inc.
Mann, D., Shakeshaft, C., Becker, J., &
Kottkampk, R. (1999) West Virginia
achievements gains from a state
wide comprehensive instructional
technology program.
http://www.mff.org/pubterms.taf?fil
e=http://www.mff.org/pubs/ME155.
pdf
Norusis, M. (2003). SPSS guide to data
analysis. Upper Saddle River, NJ:
Prentice-Hall, Inc.
NWEA (2005). RIT scale norms for use with
measures of academic progress and
achievement level tests. Lake
Oswego, OR: Author.
Scientific Learning Corporation (2004). Fast
ForWord literacy program.
Oakland, CA: Author.
Wenglinsky, H. (1998) Does it compute?
The relationship between
educational technology and student
achievement in mathematics.
Educational Testing Service.
Princeton, New Jersey.
ftp://etsis1.ets.org/pub/res/technolog.
pdf