Running Head: EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 1
The Effect of ST Math Software on Standardized Test Scores via Improvement in
Mathematics Expectancy
Teomara Rutherford, Briana Hinga, Arena Chang, AnneMarie M. Conley, and
Michael E. Martinez
University of California, Irvine
__________
The research reported here was supported by the Institute of Education Sciences, U.S.
Department of Education, through Grant R305A090527 to the University of California, Irvine. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S.
Department of Education.
This material is based upon work supported by the National Science Foundation Graduate
Research Fellowship under Grant No. DGE-0808392.
Correspondence concerning this article should be addressed to Teomara Rutherford, Department of Education, University of California, Irvine, Irvine, CA, 92697. Email: teomara@uci.edu

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 2
With the increasing focus on high-stakes testing in modern education, there is great value in exploring programs with potential to increase student learning as measured by standardized tests. Researchers must not only identify effective programs but also understand the mechanisms through which those programs affect test scores. This paper explores how one such promising program, Spatial-Temporal (ST) Math, a spatially-based math software, operates to improve test scores among participants within a randomized field trial. Specifically, the paper investigates whether ST Math leads to better performance on the mathematics portion of the California
Standards Test (CST), and if so, whether a child’s expectancy to perform well in their math class mediates this effect.
Math achievement
An education system tasked with preparing students for the 21st century must give them the tools to solve complex problems. Many problems critical to our society's future, including economic and environmental challenges, involve mathematics. Math skills are therefore vitally important to the education of future problem-solvers (NSF, 2010). However, students in the
United States increasingly struggle with math (NCES, 2009; NSF, 2010). International comparisons of mathematics proficiency show that U.S. students fall behind other top industrialized nations (Fleischman, Hopstock, Pelczar, & Shelley, 2010). Within the U.S., No
Child Left Behind, a national policy aimed at improving student achievement, has proved to be largely unsuccessful—despite focused efforts, students in the U.S. have not met federal goals for math achievement (Bryant, Hammond, Bocian, Rettig, Miller, & Cardullo, 2008).
Expectancy and achievement
The increasing emphasis on mathematics learning highlights the need for novel and highly effective approaches to increasing math achievement. It may not be enough to simply

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 3 teach children mathematics; schools may need to also instill motivation for math success. Within the research literature on motivation, one factor has particular value in raising academic achievement—the child’s expectancy to perform well (Wigfield, 1994).
Expectancy is defined as a self-prediction of how well one will do on an upcoming task (Wigfield & Eccles, 2000).
Because expectancy is empirically inseparable from self-efficacy (Wigfield, 1994), the current paper uses the terms interchangeably. Care is taken to apply the respective term used by researchers in discussing past work.
The relationship between expectancy or self-efficacy and achievement has been discussed in the education research literature for over three decades (e.g., Bandura, 1977). This relationship is robust: a meta-analysis of 39 studies revealed a positive and statistically significant relationship between self-efficacy beliefs and academic performance and persistence across a wide range of subjects, experimental designs, and methods (Multon, Brown, & Lent, 1991).
Research points to mechanisms through which self-efficacy influences achievement and persistence; specifically, self-efficacy influences task choice, effort, perseverance, and resilience
( Bandura, 1997; Schunk, 1995). For example, one study showed that students with high selfefficacy for math problem-solving tended to monitor their performance and persist longer on problem-solving tasks than did students with lower levels of self-efficacy; this persistence led to greater math learning (Bouffard-Bouchard, Parent, & Larivee, 1991).
Although the current literature reveals a strong link between self-efficacy, persistence, and academic achievement, few studies have explored processes by which increases in selfefficacy lead to increases in academic achievement. Most studies focus on either achievement or self-efficacy as an outcome, but not both. As an exception, one prior study of third through fifth graders found that child expectancy for writing mediated the influence of classrooms on grades:

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 4 positive classroom environments improved self-efficacy, and children who expected to do well on tasks tried harder and were more persistent (Pajares & Miller, 1994). Only one study to date has looked at the mediating role of expectancy on the relationship between classroom environment and standardized test scores (Fast, et al., 2010). Fast and colleagues (2010) found that classroom environment, mediated by self-efficacy, had a small positive indirect effect on standardized test scores in math. Both studies used survey data to draw correlational relationships between student expectancy and measures of achievement. The current study extends this research by using a randomized field trial to allow for stronger causal statements in evaluating the mediating effect of expectancy on the relationship between the ST Math intervention and CSTs.
Factors of expectancy
A strong body of research has established the factors that impact self-efficacy. Bandura
(1977, 1986, 1997) asserted that self-efficacy is influenced by four primary sources of information: 1) firsthand experience with a specific task; 2) vicarious learning through watching someone else complete a specific task; 3) verbal persuasion related to the task, including encouragement or discouragement; and 4) emotional arousal (either good or bad) when completing the task or watching the task being completed. ST Math may work to support math self-efficacy or expectancy by influencing a number of these factors.
The four sources of information are related. For example, firsthand experience can quite naturally combine with feedback to fall broadly in the category of “verbal persuasion.” Research has confirmed that providing students with process goals along with feedback on their process can lead to increases in student self-efficacy (Schunk & Schwartz, 1993a, 1993b; Schunk, 1995;
Schunk & Lilly, 1984). In Schunk and Schwartz, the interventions provided high school students

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 5 with either a general goal of doing their best, a product goal of writing paragraphs, or a process goal of learning to apply a certain strategy to their writing (1993a; 1993b). Results showed that students given the process goal had the highest self-efficacy. Also, across groups, students who received feedback versus students who did not receive feedback reported higher self-efficacy. ST
Math offers students frequent feedback both on the result of actions and on process: students are immediately shown whether they got the correct answer, and the method for finding the correct answer is illustrated after each trial with animations.
In addition to process goals and feedback, a sense of general agency has also been positively linked to self-efficacy beliefs: individuals who believe that they have control over their actions—that is, a sense of agency—tend to develop higher levels of self-efficacy (Schunk,
1982; Weiner, 1985). Self-efficacy is also enhanced when students believe they are performing well or improving at a task. If individuals believe they can perform better by working harder at the task, then even slow progress or failure will not lower self-efficacy (Schunk, 1995). ST
Math's highly scaffolded approach allows students to experience success on lower, easier levels of the game. The levels increase in difficulty by small increments, providing many more opportunities for success along the way. Although students must follow the game's predetermined playing order, their pace through the games is not dictated by the teacher or the performance of their peers but by each student’s personal record of incremental success.
Schunk and Miller (2002) noted that students need opportunities to persist through failures in their academic career if they are to learn that hard work can lead to better outcomes, a self-efficacy protecting belief. Using standard math curricula, many schools do not provide students with motivating opportunities to view mistakes as building blocks that ultimately lead to academic success. Rather, mistakes are often viewed as conclusive failures with long-lasting

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT consequences. This type of environment dampens student motivation and leads to decreased
6 persistence (e.g., Miller, Adkins, & Hooper, 1993; Miller & Blumenfeld, 1993). Conversely, the
ST Math curriculum allows students to view obstacles and mistakes as productive building blocks toward significant learning. ST Math creates opportunities for students to fail frequently as they grapple with math problems within the software while the game provides support to push through failures. Students can also receive help through teachers who have tools to monitor student individual progress—teachers receive instruction on how best to guide students whenever they are "stuck." The increments of learning provide opportunities to fail in a safe environment, where multiple attempts at success following setbacks are not overwhelming.
ST Math
The ST Math program was created by the MIND Research Institute to teach mathematical reasoning to elementary-aged children through game-like software aligned with state math content standards (Figure 1). ST Math software allows children to interact with spatial temporal problems through individualized instruction based on each student’s pace of learning.
Math problems are represented as games with increasing levels of difficulty. The first level of ST
Math allows for immediate success for students. When a student “beats” a level, they are rewarded by a slightly more difficult level, which includes math problems with a somewhat more complicated mathematical concept. The process of moving forward after completing a level allows for gradual scaffolding from simpler to more complex mathematics. In the higher-level games, children interact with more difficult math principles, larger quantities, and multistep problems. The program is designed to be integrated into the curriculum through coordination between teacher-led instruction and software design to scaffold each child’s learning at the appropriate pace and level of difficulty. Over all, the method is effective: past research has

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT found ST Math to increase student standardized test scores in math with an effect size of .37
7 when examined as aggregated changes in test scores at the grade-level (Rutherford et al., 2010).
The current study
ST Math provides students with incremental process goals, provides feedback on each student attempt, and allows students to self-pace through increasingly difficult material. Students are driven to improve upon their personal best and persist through transient failures that allow them to repeatedly experience success as the more salient aspect of mathematics learning. All these factors are associated with increased self-efficacy or expectancy. The present study explores the relationship between ST Math, expectancy, and achievement within the context of a randomized trial. It investigates whether ST Math's impact on math achievement, measured as
CST scores, is mediated by an increase in student expectancy for success in mathematics.
Considering the strong positive relationship between expectancy and important measures of academic success, any intervention that increases expectancy for success may be an effective means to improve academic outcomes for students, possibly leading to greater persistence and future learning both within the subject at hand and in academics generally.
Method
Participants and Procedures
Data were collected as part of an IES-funded project to study the impact of ST Math on
CST scores, other math achievement, and cognitive and motivational outcomes. The study includes all second through fifth grade students at 52 Orange County, California schools, each with high percentages of English Language Learners and students qualifying for free or reduced lunch.
Participating schools were randomly selected to initially implement ST Math in either 2 nd

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 8 and 3 rd
or 4 th
and 5 th
grades. Thirty-four schools began implementation in the 2008-2009 school year and 18 schools began implementation in the 2009-2010 school year. In the spring of 2010, a total of 547 students from both cohorts of schools were randomly selected (stratified by teacher) to complete motivation surveys and other cognitive measures. The sample was narrowed to 394 third through fifth graders—only those students in third grade or above would have prior year achievement measures. Descriptive statistics of the sample, divided by treatment and control, are provided in Table 1.
Testing was conducted during the school day on school grounds. Measures were administered one-on-one by trained research assistants and took approximately one hour to complete. Measures included targeted math achievement and cognitive abilities; only individual motivation measures are considered in the current paper.
Measures
Motivation measures. Eccles et al. (1993) expectancy-value scales were administered one-on-one to sampled students. The validity and reliability of these scales have been established in previous educational research, including within the context of mathematics (Wigfield &
Eccles, 2000). The items were administered using 7-point Likert-type scales and were presented both visually and read aloud to students (Figure 2). Each item was centered within the third through fifth grade sample to a mean of zero and combined into scales based on the literature.
Final scale alpha reliabilities were .82 for expectancy for math success and .65 for math value.
Of the five expectancy items, one was omitted to improve the alpha. Although it was not posited that value would have an association with ST Math, it was included in the model because of its relationship with expectancy and its place within the Eccles et al. (1993) expectancy-value framework.

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT
Standardized tests. All California 2 nd
through 5 th
graders, including our study participants, take the CST in the spring of each year. The math portion of the CST measures
9 grade-level math material aligned to the California content standards. Recent alpha reliabilities for 2 nd
and 3 rd
grade math CSTs are .93 and .94 respectively (Educational Testing Service, 2008).
School districts provided CST scores along with demographic information for study participants.
Analysis
The current analysis was conducted on 321 of the 394 third through fifth graders selected for individual testing. This number reflects those students with complete motivation data and both 2009 and 2010 CST scores. Mediation was tested by regressing 2010 CST scores on presence of ST Math, expectancy on presence of ST Math, and 2010 CST scores on both presence of ST Math and expectancy (Baron & Kenny, 1986). Control variables included ELL, gifted, and free lunch status, along with gender, grade, and previous CST score. Figure 3 provides the structure of our conceptual model.
Results
As expected (Eccles, Wigfield, Harold, & Blumenfeld, 1993), expectancy for math success declined on average from third to fifth grades among both treatment and control students. Figure 4 shows the decline in expectancy and descriptive differences between ST Math and non-ST Math students. Starting in third grade, a descriptive difference in expectancy appears between treatment and control. This difference widens for more cohorts at later grades.
Full model regression results are provided in Table 2. The total effect of ST Math on CST scores is presented in the right-most column. On average, treatment students score over 16 points higher on math CSTs than control students, d = .18. ST Math also has a positive association with math expectancy for success: treatment students rate their expectancy about one fifth of a

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 10 standard deviation above control students. These associations fulfill the requirements of paths A and C in Baron and Kenny's (1986) model for mediation. To quantify the direct effect of ST
Math on math CSTs, both treatment condition and expectancy are entered into the model. The path B requirement is met in the significant association between expectancy and CST scores.
When both are in the model, the association between ST Math and CST scores diminishes, but does not completely disappear, indicating partial mediation. Considering the difference between the effect of ST Math on CST scores before and after the addition of expectancy to the model, within this sample expectancy mediates 17% of the effect of ST Math on math CST scores.
Discussion
One goal of this study was to investigate predictors of standardized test improvement, a goal with great value in light of the high-stakes testing movement. Our results are in line with previous research in that expectancy was positively associated with performance outcomes.
Increased classroom challenge and mastery focus have been linked to positive performance, both for classroom and assessment outcomes (Wigfield, 1994), and more recently, standardized tests
(Fast et al., 2010). ST Math, a mastery-focused suite of challenging computer-based games, produces effects similar to those seen from traditional teacher-driven curricula with similar characteristics.
The effects of ST Math were partially mediated by increased expectancy for math success among treatment students. Within this model, ST Math also operates directly on performance.
Further research is needed to determine other mechanisms through which ST Math operates and to identify precursors of expectancy apart from those associated with the software. The research is ongoing as the ST Math study team collects and analyzes student data on cognitive and achievement measures and teacher data on intervention implementation and other practices. The

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 11 individualized testing procedure was slightly modified and repeated in 2011. Initial analysis of motivation data shows a positive relationship between ST Math and math expectancy for success among the new, larger sample of tested students (N > 500). Once CST scores are released by the districts, this future analysis can be compared to the current findings. Consistent, robust findings of expectancy mediation of ST Math may have implications beyond mathematics achievement by understanding more fully the important variables that predict, and functionally support, gains in academic proficiency. Participation in ST Math and similar programs might also improve a more global academic self-efficacy, which could impact achievement across domains. This exciting possibility will also be investigated in future analyses.

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 12
References
Bandura, A. (1977) Self-efficacy: Toward a unifying theory of behavioral change. Psychological
Review, 84 (2), 191-215.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory .
Englewood Cliffs, NJ: Prentice-Hall.
Bandura, A. (1997 ). Self efficacy: The exercise of control . New York: Freeman.
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. Journal of
Personality and Social Psychology , 51 (6), 1173-1182.
Bouffard-Bourchard, T., Parent, S., & Parivee, S. (1991). Influence of self-efficacy on self- regulation and performance among junior and senior high school age students.
International Journal of Behavioral Development, 14 , 153 – 164.
Bryant, M. J., Hammond, K. A., Bocian, K. M., Rettig, M. F., Miller, C. A., & Cardullo, R. A.
(2008). School performance will fail to meet legislated benchmarks. Science , 321 (5897),
1781 -1782. doi:10.1126/science.1161033
Eccles, J.,Wigfield, A., Harold, R.D., & Blumenfeld, P. (1993). Age and gender differences in children's self- and task perceptions during elementary school. Child Development , 64 ,
830-847.
Educational Testing Service (2008). California Standards Tests (CSTs) technical report, spring
2007 administration . Retrieved from http://www.cde.ca.gov/ta/tg/sr/documents/csttechrpt07.pdf
Fast, L. A., Lewis, J. L., Bryant, M. J., Bocian, K. A., Cardullo, R. A., Rettig, M., & Hammond,
K. A. (2010). Does math self-efficacy mediate the effect of the perceived classroom

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 13 environment on standardized math test performance? Journal of Educational Psychology ,
102 (3), 729-740. doi:10.1037/a0018863
Fleischman, H.L., Hopstock, P.J., Pelczar, M.P., and Shelley, B.E. (2010). Highlights from PISA
2009: Performance of U.S. 15-year old students in reading, mathematics, and science literacy in an international context (NCES 2011-004). U.S. Department of
Education, National Center for Education Statistics. Washington, DC: U.S. Government
Printing Office.
Miller, S. D., & Blumenfeld, P. C. (1993). Characteristics of tasks in basal skill instruction.
Elementary School Journal, 94 , 33-48.
Miller, S. D., Adkins, T., & Hooper, M. L. (1993). Why teachers select certain tasks and students’ reactions to them.
Journal of Reading Behavior, 25, 69-95.
Multon, K. D., Brown, S. D., & Lent, R. W. (1991). Relation of self-efficacy beliefs to academic outcomes: A meta-analytic investigation. Journal of Counseling Psychology, 38 (1), 30-
38.
National Center for Education Statistics (2009). The nation’s report card: Mathematics 2009
(NCES 2010–451). Institute of Education Sciences, U.S. Department of Education,
Washington, D.C.
National Science Foundation (2010). Preparing the Next Generation of STEM Innovators:
Identifying and Developing our Nation’s Human Capital.
Retrieved from http://www.nsf.gov/nsb/publications/2010/nsb1033.pdf
No Child Left Behind (NCLB) Act of 2001, Pub. L. No. 107-110, § 115, Stat. 1425 (2002).
OECD (2010), PISA 2009 Results: Executive Summary .
Pajares, F., & Miller, M. D. (1994). The role of self-efficacy and self-concept beliefs in

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 14 mathematical problem-solving: A path analysis. Journal of Educational Psychology , 86 ,
193-203.
Rutherford, T., Kibrick, M., Burchinal, M., Richland, L., Conley, A., Osborne, K.,…Martinez,
M. E. (2010). Spatial temporal mathematics at scale: An innovative and fully developed paradigm to boost math achievement among all learners.
Available from ERIC database.
(ED510612).
Schunk, D. H. (1982). Effects of effort attributional feedback on children’s perceived self efficacy and achievement. Educational Psychology, 74 , 548-556.
Schunk, D. H. (1995). Self-efficacy and education and instruction. In J. E. Madduz (Ed.), Self- efficacy, adaptation, and adjustment: Theory, research and application (pp. 281-3030).
New York: Plenum Press.
Schunk, D. H., & Lilly, M. W. (1984). Sex differences in self-efficacy and attributions: Influence of performance feedback. Journal of Early Adolescence, 4 , 201-213.
Schunk, D. H., & Miller, S. D. (2002). Self-efficacy and adolescents’ motivation. In F. Pajares 7
T. Urdan (Eds.), Academic motivation of adolescents (pp. 29-52). Greenwich, CT:
Information Age.
Schunk, D. H. & Schwartz, C. W. (1993a). Goals and progress feedback: Effects on self-efficacy and writing achievement. Contemporary Educational Psychology, 18 , 337-354.
Schunk, D. H. & Schwartz, C. W. (1993b). Writing strategy instruction with gifted students:
Effects of goals and feedback on self-efficacy and skills. Roeper Review, 15 , 225- 230.
Weiner, B. (1985). An attributional theory of achieveent motivation and emotion. Psychological
Review, 92 (4), 548-573.
Wigfield A. (1994). Expectancy-value theory of achievement motivation: A developmental

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT perspective. Educational. Psychoy. Review, 6 , 49–78.
Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation.
15
Contemporary Educational Psychology , 25 , 68-81.

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 16
Figure 1.
This very simple ST Math game teaches addition to third graders. The left part of the figure shows a correct answer, and the right, an incorrect answer. The blue base under 300 or 400 represents the student's answer. On the right, a student can see that they got the answer wrong because they were 200 short. The illustration gives them a method for counting along the number line and helps them to visualize the numbers corresponding to the distances.

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT
Table 1
Descriptive Statistics of 2010 Individual Testing Sample
3rd Grade 4th Grade 5th Grade
Total N = 321
Mean
2009 Math CSTs
2010 Math CSTs
Change in CSTs
Percent of Sample
Male
English Language Learner
Free/Reduced Lunch
Gifted Program
ST Math Control ST Math Control ST Math Control
376.00 371.41
384.93 379.64
8.93
54%
74%
83%
13%
8.23
57%
66%
84%
16%
387.76 370.37 405.83 386.41
406.50 377.05 425.30 374.23
18.74
50%
50%
91%
20%
6.68
68%
63%
89%
5%
19.47 -12.18
41%
28%
76%
28%
54%
49%
87%
21%
39 N 54 56 107 19 46
Note.
Demographic data as reported in the 2010 CST data provided by school districts.
17

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 18
Figure 2.
A question assessing student expectancy for math success. Trained research assistants read this and other similar questions aloud while students looked at the pictures. Students indicated their answer by giving the corresponding number.

Participated in ST Math
EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT
Expectancy
Math Performance 2010
19
Math Performance 2009
Other Covariates: ELL, Gender, Free Lunch, Gifted, Grade, & Value
Figure 3.
Conceptual framework for examining relations between ST Math, Expectancy for math and Math performance (CST).
Dashed lines indicate control variables.

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 20
Figure 4.
Descriptive (non-regression) expectancy for math success differences between treatment and control students at each grade-level.

EXPECTANCY AS MEDIATOR OF ST MATH ON ACHIEVEMENT 21
Table 2
Mediation Model of Math Performance Regressed on Math Expectancy and ST Math Condition
N = 321
ST Math
Math Expectancy
Value for Math
Math Performance (2009)
Grade (3rd-5th in 2010)
Male
Free/Reduced Lunch (2009)
Free/Reduced Lunch (2010)
Labeled as gifted (2009)
Labeled as gifted (2010)
English Language Learner (2009)
English Language Learner (2010)
Constant
Total Effect:
ST Math

Expectancy
B
0.19
*
0.65
***
0.00
***
-0.10
0.29
**
-0.03
0.16
0.16
-0.26
0.02
0.05
-1.59
***
SE
0.10
0.06
0.00
0.06
0.09
0.17
0.16
0.17
0.17
0.15
0.15
0.42
Total Effect:
ST Math

Performance
B
16.38
*
10.90
*
0.71
***
-1.07
-2.03
-4.67
-2.42
44.35
***
47.86
***
-2.10
-9.16
119.85
***
SE
6.91
4.30
0.05
4.53
6.69
12.20
11.80
12.16
12.42
11.13
11.13
30.04
Direct Effects:
ST Math & Expectancy

Performance
SE B
13.64
*
14.22
***
1.62
0.65
***
0.41
-6.13
-4.25
-4.64
42.03
***
51.54
***
-2.39
-9.92
142.46
***
6.83
4.04
4.98
0.05
4.47
6.68
11.98
11.61
11.96
12.24
10.93
10.94
30.19
Note.
* p < .05, ** p < .01, *** p < .001 Unstandardized regression coefficients. Math performance is California Standards Test (CST).
Outcome measures administered in the spring of 2010: expectancy prior to math performance. Demographic covariates as reported along wit h the
CST in the year specified; value covariate administered along with expectancy measure. Expectancy and value measures created from item responses centered to a mean of zero .